README.md

---
title: "Training PyThia-160M from scratch on AWS Sagemaker"
emoji: 📈
colorFrom: indigo
colorTo: yellow
sdk: gradio
sdk_version: 4.9.0
app_file: app.py
pinned: false
license: mit
---

# Training PyThia-160M from scratch on AWS Sagemaker
[**Repo Link**](https://github.com/RaviNaik/ERA-SESSION22/)

This is an implementation of Pythia-160M [LitGPT](https://github.com/Lightning-AI/lit-gpt) by LightningAI.  
Dataset used to train: [RedPajama](https://huggingface.co/datasets/togethercomputer/RedPajama-Data-1T).

### Tasks:
1. :heavy_check_mark: Once done, share: screenshot of the logs showing your submission result and clearly showing PageMaker in the URL [800 pts]
2. :heavy_check_mark: 4 sample predictions [200 pts]
3. ✔️ The model moved to HuggingFace Spaces as an app. [1000 pts]

### Training Log on AWS Sagemaker:
![image](https://github.com/RaviNaik/ERA-SESSION22/assets/23289802/b549772f-847e-4f12-a6f6-6661abd36fc8)

### Sample Results:
![image](https://github.com/RaviNaik/ERA-SESSION22/assets/23289802/6ce0cccf-9694-4020-8e98-e2175c0e2261)

![image](https://github.com/RaviNaik/ERA-SESSION22/assets/23289802/a21addc7-2d13-4a96-977c-2eaff2e8d414)

![image](https://github.com/RaviNaik/ERA-SESSION22/assets/23289802/bdf88403-b430-49d3-a706-1210728c39ee)

![image](https://github.com/RaviNaik/ERA-SESSION22/assets/23289802/27493648-9273-47a0-bc5f-42dcbf74cb87)