README.md

---
license: llama3.1
tags:
- code
---

Model Overview
Model Name: Llama 3.1 180M Untrained
Model Size: 180M parameters
Tensor Type: F32
License: MIT
Model Type: Untrained Language Model
Framework: PyTorch

Model Description
The Llama 3.1 180M Untrained model is a lightweight, untrained language model designed to serve as a starting point for research and experimentation in natural language processing. With 180 million parameters, this model is suitable for fine-tuning on specific tasks or domains, offering a balance between model complexity and computational efficiency.

Intended Use
This model is intended for research purposes and fine-tuning on specific tasks such as text classification, sentiment analysis, or other NLP tasks. As the model is untrained, it requires fine-tuning on relevant datasets to achieve desired performance.

Fine-Tuning Requirements
GPU Requirements:
Full Fine-Tuning: This model requires a GPU with at least 24 GB of VRAM for full fine-tuning at a sequence length of 4096.
Supported GPUs: NVIDIA RTX 3090, A100, or equivalent.
Training Data
This model has not been trained on any data. Users are encouraged to fine-tune the model on datasets that are appropriate for their specific use case.

Evaluation
As the model is untrained, it has not been evaluated on any benchmark datasets. Performance metrics should be determined after fine-tuning.

Limitations
Untrained: The model is untrained and will not perform well on any task until it has been fine-tuned.
Ethical Considerations: Users should be mindful of the ethical implications of deploying fine-tuned models, especially in sensitive applications.



from transformers import AutoModelForCausalLM, AutoTokenizer

# Load the fine-tuned model and tokenizer
model = AutoModelForCausalLM.from_pretrained("oktrained/llama3.1_180M_untrained")
tokenizer = AutoTokenizer.from_pretrained("oktrained/llama3.1_180M_untrained")

# Sample input text
input_text = "Once upon a time"

# Tokenize input
inputs = tokenizer(input_text, return_tensors="pt")

# Generate output
output = model.generate(**inputs, max_length=50)

# Decode output
generated_text = tokenizer.decode(output[0], skip_special_tokens=True)

print(generated_text)