MMLU-by-task-Leaderboard / details_data_processor.py

Commit History

updated pipeline and init
7f2d984

Corey Morris commited on

removed commented code
2f457d8

Corey Morris commited on

shortened file name
7622af3

Corey Morris commited on

shortened file name
38d88f9

Corey Morris commited on

using URL as file name
25b87bf

Corey Morris commited on

WIP. Updated download file. Can now download all files. Need to integrate that code to loop through all files to download or combine files first into a single dataframe and then save that
0a77c60

Corey Morris commited on

Replicating 404 error with a test so I can troubleshoot
9adae3c

Corey Morris commited on

Updated download_file method
b58e1f0

Corey Morris commited on

Build URL from file path is working
cc32c4f

Corey Morris commited on

moved methods to better match flow
f228d38

Corey Morris commited on

removed most commented out code from details processor
74822dd

Corey Morris commited on

Find files is working as expected
30fa96a

Corey Morris commited on

WIP commit. Finding files can be identical as the method in results_data_processor.
c32735e

Corey Morris commited on

added mostly hardcoded generate url method and test
83a34f0

Corey Morris commited on

Added download file method and test
513e813

Corey Morris commited on

Added basic structure of details data processing and testing. For downloading huggingface details dataset files
ee9e25e

Corey Morris commited on