Spaces:

upstage

/

open-ko-llm-leaderboard

like 478

Running on CPU Upgrade

Hello, i use the code [https://huggingface.co/spaces/upstage/open-ko-llm-leaderboard/tree/main]

1. Fork the above code
2. run the below code

from src.load_from_hub import get_evaluation_queue_df, get_leaderboard_df, is_model_on_hub, load_all_info_from_hub
from src.display_models.utils import (
    AutoEvalColumn,
    EvalQueueColumn,
    fields,
    styled_error,
    styled_message,
    styled_warning,
)

QUEUE_REPO = "open-ko-llm-leaderboard/requests"
RESULTS_REPO = "open-ko-llm-leaderboard/results"
EVAL_REQUESTS_PATH = "eval-queue"
EVAL_RESULTS_PATH = "eval-results"
eval_queue_private, eval_results_private = None, None
eval_queue, requested_models, eval_results, users_to_submission_dates = load_all_info_from_hub(
    QUEUE_REPO, RESULTS_REPO, EVAL_REQUESTS_PATH, EVAL_RESULTS_PATH
)

COLS = [c.name for c in fields(AutoEvalColumn) if not c.hidden]
BENCHMARK_COLS = [
    c.name
    for c in [
        AutoEvalColumn.arc,
        AutoEvalColumn.hellaswag,
        AutoEvalColumn.mmlu,
        AutoEvalColumn.truthfulqa,
        AutoEvalColumn.commongen_v2,
        # TODO: Uncomment when we have results for these
        # AutoEvalColumn.ethicalverification,
    ]
]
df = get_leaderboard_df(eval_results, eval_results_private, COLS, BENCHMARK_COLS)

if you get TypeError: 'NoneType' object is not subscriptable error when run above code, change src/display_models/get_model_metadata.py line 82
except AttributeError: --> except

This method is outdated. Please tell me how to get a CSV now?