tangled-llama-b-128k-base-v0.1 / scripts /prepare_finetune_dataset.py
mtasic85's picture
pretrain model
81565a0
raw
history blame
No virus
793 Bytes
"""
# sft
https://huggingface.co/datasets/HuggingFaceH4/no_robots
https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k
https://huggingface.co/datasets/HuggingFaceH4/deita-10k-v0-sft
https://huggingface.co/datasets/Open-Orca/slimorca-deduped-cleaned-corrected
https://huggingface.co/datasets/arcee-ai/EvolKit-20k
https://huggingface.co/datasets/ise-uiuc/Magicoder-Evol-Instruct-110K
https://huggingface.co/datasets/WizardLMTeam/WizardLM_evol_instruct_V2_196k
https://huggingface.co/datasets/ai2-adapt-dev/olmoe-commercial
# dpo
https://huggingface.co/datasets/allenai/ultrafeedback_binarized_cleaned
https://huggingface.co/datasets/kyujinpy/orca_math_dpo
https://huggingface.co/datasets/argilla/OpenHermesPreferences
https://huggingface.co/datasets/ProlificAI/social-reasoning-rlhf
"""