andrewatef (Andrew Atef)

updated a dataset 7 days ago

andrewatef/testtesttesttesttest02

Viewer • Updated 7 days ago • 5k • 11

liked a dataset 6 months ago

ClusterlabAi/101_billion_arabic_words_dataset

Viewer • Updated Jun 16 • 33.1M • 1.79k • 64

updated 3 models 7 months ago

liked a model 7 months ago

microsoft/Phi-3-mini-4k-instruct

Text Generation • Updated Sep 20 • 1.5M • • 1.08k

Reacted to macadeliccc's post with 🔥 7 months ago

Post

4501

Fine tune Phi-3 using samatha themed dataset and Huggingface SFT trainer!

In this colab, we simply apply a supervised finetune to phi-3 using the sharegpt format.

def formatting_prompts_func(examples):
    convos = examples["conversations"]
    texts = []
    mapper = {"system": "system\n", "human": "\nuser\n", "gpt": "\nassistant\n"}
    end_mapper = {"system": "", "human": "", "gpt": ""}
    for convo in convos:
        text = "".join(f"{mapper[(turn := x['from'])]} {x['value']}\n{end_mapper[turn]}" for x in convo)
        texts.append(f"{text}{EOS_TOKEN}")  
    return {"text": texts}

dataset = dataset.map(formatting_prompts_func, batched=True)
print(dataset['text'][8])

Opus Samantha consists of 1848 samples with the samantha personality. The dataset covers a wide variety of topics such as logical reasoning, mathematics, legal, and rp.

This notebook serves as a viable option to finetune Phi-3 until Unsloth supports phi-3, which should be very soon. When that happens check out AutoSloth for both SFT, DPO, and langfuse format RAG fine tuning on free tier colab hardware.

Resources:
Dataset: macadeliccc/opus_samantha
Colab: https://colab.research.google.com/drive/1e8LILflDQ2Me52hwS7uIfuJ9DxE2oQzM?usp=sharing
AutoSloth: https://colab.research.google.com/drive/1Zo0sVEb2lqdsUm9dy2PTzGySxdF9CNkc#scrollTo=bpimlPXVz-CZ