README.md

---
library_name: transformers
tags:
- SFT
- Llama
- Llama 70B Chat
license: llama2
---

<img src="https://huggingface.co/Radiantloom/radiantloom-llama-70b-instruct/resolve/main/radiantloom-llama-70b-instruct.jpeg" alt="Radiantloom Llama 70B Instruct" width="800" style="margin-left:'auto' margin-right:'auto' display:'block'"/>

## Radiantloom Llama 70B Instruct
The Radiantloom Llama 70B Instruct, a large language model (LLM) developed by Radiantloom AI, features approximately 70 billion parameters. From vibes-check evaluations, the Radiantloom Llama 70B Instruct demonstrates great performance in various applications like creative writing, multi-turn conversations, in-context learning through Retrieval Augmented Generation (RAG), and coding tasks. Its out-of-the-box performance already delivers impressive results, particularly in writing, coding, and reasoning tasks. This model produces longer form content and provides detailed explanations of its actions. To maximize its potential, consider implementing instruction tuning and Reinforcement Learning with Human Feedback (RLHF) techniques for further refinement. Alternatively, you can utilize it in its current form.

## Intended Uses & Limitations
The Radiantloom Llama 70B Instruct is versatile and can be utilized for various text generation tasks such as summarization, chat, coding, question answering, retrieval augmented generation (RAG), role play, and content generation.

It can be considered a state-of-the-art generative language model, it demonstrates competitive performance in general tasks when compared to other open and closed-source large language models such as GPT 3.5 Turbo, and Mixtral 8x7B Instruct.

## Model Usage
You can deploy it on a Runpod GPU instance using our [one-click Runpod template](https://www.runpod.io/console/gpu-secure-cloud?ref=80eh3891&template=tbkdia6qdv) (Our Referral Link. Please consider Supporting). This template provides you with an OpenAI-compatible API endpoint that you can integrate into your existing codebase designed for OpenAI APIs.

## Prompt Template
We have fine-tuned this model using the ChatML format, and you can achieve optimal performance by utilizing the ChatML format.

```
<s>[INST] <<SYS>>{system_message}<</SYS>> {user_message}[/INST]

```

## Inference Code
```python

## install dependencies
## !pip install -q -U git+https://github.com/huggingface/transformers.git
## !pip install -q -U git+https://github.com/huggingface/accelerate.git
## !pip install -q -U bitsandbytes sentencepiece

## load model
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, TextStreamer

model_name = "Radiantloom/radiantloom-llama-70b-instruct"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.bfloat16
)

# load the model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    quantization_config=bnb_config,
    device_map="auto",
)

tokenizer = AutoTokenizer.from_pretrained(
    model_name, 
    trust_remote_code=True
)

tokenizer.pad_token = tokenizer.eos_token

## inference
prompt = """
<s>[INST]<<SYS>>You are a helpful AI assistant developed by Radiantloom AI.<</SYS>>Take a deep breath and think step-by-step. Write a tale about a time-traveling historian who's determined to witness the most significant events in human history.[/INST]
"""

# tokenize the input string
inputs = tokenizer(
    prompt, 
    return_tensors="pt", 
    return_attention_mask=False
)

# generate text using the model
streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
outputs = model.generate(**inputs, max_length=2048, streamer=streamer, do_sample=True, repetition_penalty=1.1"))

# decode and print the output
text = tokenizer.batch_decode(outputs)[0]

```

Here are a few example use cases. if we query the model with

```
<s>[INST]<<SYS>>You are a helpful AI assistant developed by Radiantloom AI.<</SYS>>Take a deep breath and think step-by-step. Write a tale about a time-traveling historian who's determined to witness the most significant events in human history.[/INST]

```
the model will return

```
Story: The Time Chronicles of Professor Amelia Harcourt

Professor Amelia Harcourt was an exceptional historian, renowned for her remarkable contributions to the field of historical studies. However, her passion for understanding the past extended far beyond academics. She had always dreamt of witnessing firsthand the events that shaped the world as we know it. When she stumbled upon an ancient manuscript filled with cryptic symbols while working on a research project, she wasn't aware that it would change her life forever.

After translating the mysterious text, Amelia discovered that it contained blueprints for constructing an extraordinary device - a time machine that could transport its user across different eras. Fueled by an all-consuming fervor for exploring pivotal moments in history, Amelia spent her remaining savings on procuring the materials necessary to build the intricate apparatus. The process required tireless dedication and rigorous labor, but through a mixture of intelligence, skill, and unrelenting determination, she finally constructed a functioning time machine three years later.

With anticipation surging through her veins, Amelia embarked on a transcendent expedition to experience the most significant occurrences in human history. Her very first stop took place more than two million years ago: the East African Rift Valley, where homo habilis, our early ancestors, were believed to have emerged. In this remote corner of prehistory, Amelia studied their behavior closely and marveled at their resourcefulness. Their innovative use of tools and rudimentary social systems helped shape complex human societies of the future, making the encounter a moving lesson on the importance of adaptability.

The second destination brought Amelia even further back in time, landing directly in Earth's arctic region during the late Cretaceous period. This journey held both personal ambition and scientific curiosity; Amelia wished to better understand why dinosaurs went extinct. Upon arrival, the historian felt a wave of unease as she navigated through the treacherous environment filled with towering dinosaur species, including the voracious Tyrannosaurus rex. With immense caution, she conducted her analysis and came to the shocking conclusion that asteroid impacts, not volcanic activity, were primarily responsible for wiping out the majority of dinosaur species. As these creatures faced catastrophic conditions, mammals thrived and eventually evolved into humans.

Amelia continued her chronological pilgrimage with an extended stay at Stonehenge, one of the most mystifying historical sites she'd ever visited. After learning from several sources during her academic career, she hypothesized that the magnificent monument was built as part of a larger complex dedicated to worshipping gods tied to celestial bodies and fertility. Upon arriving in Neolithic Britain, Amelia watched in awe as people worked together to erect massive bluestones using ropes made from animal hides, wooden rollers, and levers carved from antlers. Witnessing this collaborative effort revealed a newfound appreciation for the technological ingenuity and social organization present in early human civilizations.

Throughout subsequent adventures, Amelia became captivated by the enlightening teachings of Socrates in Ancient Greece and traveled through the bustling metropolis of imperial Rome. These excursions showcased the timeless themes of wisdom, governance, justice, art, and culture. They illuminated various aspects of human nature and the inherent values shared across every epoch.

Overwhelmed by curiosity and a newfound sense of wonderment, Amelia decided to explore other transformative eras such as the discovery of the New World and the Industrial Revolution. Eventually, she found herself amidst the burgeoning civil rights movement in America during the 1960s, watching in admiration as advocates for racial equality peacefully rallied against systemic oppression and fought for equal rights.

As Professor Amelia Harcourt immersed herself in the diverse expanse of human history, her encounters revealed a profound understanding of progress. Each event she observed had contributed something unique and essential to the development, growth, and evolution of mankind. This revelation reminded her of an old saying: "Only those who look backward can truly see forward." Through her time travels, Amelia began recognizing the patterns and interconnections between different eras, gaining immense insight into the human spirit's resilience and capacity for change.

Inspired by her once-in-a-lifetime odyssey, Amelia Harcourt set forth creating The Time Chronicles-an extensive body of work featuring detailed accounts from her journeys through time. This collection transformed academic views regarding the past and offered generations to come priceless knowledge and fascinating perspectives vital to understanding our shared human story.

```

and if we query the model with

```
<s>[INST]<<SYS>>You are an AI assistant. Take a deep breath, think step-by-step and respond to following instructions from the user. <</SYS>>The cafeteria had 23 apples. If they used 20 for lunch and bought 6 more, how many apples do they have?[/INST]

```

the model will return

```
Let's break this down:
1. The cafeteria initially had 23 apples.
2. They used 20 for lunch. To find out how many apples were left after lunch, subtract the number of apples used for lunch (20) from the initial number of apples (23). This gives us 23 - 20 = 3 apples left after lunch.
3. Then they purchased 6 more apples. To find out the total number of apples now, add the number of apples left after lunch (3) to the number of apples bought recently (6). So we have 3 + 6 = 9 apples in total.

```

and if we query the model with

```
<s>[INST]<<SYS>>You are an AI assistant. Take a deep breath, think step-by-step and respond to following instructions from the user. <</SYS>>Write a function in Python that checks if a number is prime.[/INST]

```

the model will return

```
The user will provide you with a positive integer and your task is to check whether it can be divided by any other number (divisors) except for one and itself (the number).
Let's say the number given by the user is 2147483647. Your first thought process should include finding the square root of this number as that'll help narrow down potential divisors. Here, sqrt(2147483647) = 46340. Next, iterate over numbers from two up to 46340 checking if they divide evenly into 2147483647 (remainder is zero when modded). If such a divisor is found, then return False since it indicates 2147483647 has additional factors besides one and itself; thus it isn't prime. Otherwise, if no such divisor is found during iteration, then return True signaling that 2147483647 is indeed a prime number. 
But make sure you also consider edge cases like 2, because some programming languages might use zero-based indexing.

Here's a possible solution:
```

```python
import math

def is_prime(n):
   if n <= 1:  # Edge case: numbers below 2 are not considered primes
       return False
   elif n == 2 or n == 3:  # Special cases for small primes
       return True
   elif n % 2 == 0:  # Quick filter for even numbers
       return False

   root = int(math.sqrt(n))
   
   for i in range(3, root + 1, 2):  # Only odd divisors needed; start at 3
       if n % i == 0:
           return False
   return True

number = 2147483647
result = is_prime(number)
print(f"The result for {number} is:", "Prime" if result else "Not Prime")
```

## Evaluations
We are encouraged by the initial assessments conducted using the [LLM-as-a-Judge method](https://arxiv.org/abs/2306.05685), particularly with the utilization of GPT-4. Our ongoing efforts involve a more comprehensive evaluation of the model. We have also entered this model into the HuggingFace Open Leaderboard for assessment. As we progress, we will continue to provide updates on our findings and share the results on this platform.

## Ethical Considerations and Limitations
Radiantloom Llama 70B Instruct, a powerful AI language model, can produce factually incorrect output and content not suitable for work (NSFW). It should not be relied upon to provide factually accurate information and should be used with caution. Due to the limitations of its pre-trained model and the finetuning datasets, it may generate lewd, biased, or otherwise offensive content. Consequently, developers should conduct thorough safety testing prior to implementing any applications of this model.

## About Radiantloom AI
Radiantloom AI trains open-source large language models tailored for specific business tasks such as copilots, email assistance, customer support, and database operations.

Learn more about Radiantloom by visiting our [website](https://radiantloom.com). Follow us on Twitter at [Radiantloom](https://twitter.com/radiantloom) to gain early access to upcoming Radiantloom AI large language models.

## Want to Support Our Work?

We are passionate about contributing open-source AI models to the broader AI community to accelerate AI development and deployment. If you find our work valuable and would like to contribute to its continued success, consider buying us a few coffees!

Your support goes a long way in helping us cover training costs, fund development tools, and keep the project thriving. Every coffee you buy is a small but meaningful contribution that fuels our dedication to providing a top-notch experience for users like you.

[Buy us a coffee](https://www.buymeacoffee.com/aigeek_) and be a part of our journey towards making this project even more awesome. We appreciate your support!