Only allow 1 instance of a particular model name to be in the list.

New cloned models of the toppers in the leaderboard will obviously continue appearing in the leaderboard as duplicates, since we are adding models in the skip list manually.

This method attempts to avoid that process for models named identically to the existing ones.

You might like to also check the creation date of a model and choose the earliest one with a specific date to display.

app.py

@@ -827,12 +827,12 @@ def get_mteb_data(tasks=["Clustering"], langs=[], datasets=[], fillna=True, add_
                 res["Sequence Length"] = EXTERNAL_MODEL_TO_SEQLEN.get(model, "")
             df_list.append(res)
     
+    models_processed_names = []
     for model in models:
         if model.modelId in MODELS_TO_SKIP: continue
-        print("MODEL", model)
-        readme_path = hf_hub_download(model.modelId, filename="README.md")
-        meta = metadata_load(readme_path)
-        # meta['model-index'][0]["results"] is list of elements like:
+        if model.modelId.split('/')[1] in models_processed_names: continue
+        models_processed_names.append(model.modelId.split('/')[1])
+        print("MODEL", model)    # meta['model-index'][0]["results"] is list of elements like:
         # {
         #    "task": {"type": "Classification"},
         #    "dataset": {