How to use similarity settings  other then "BM25" in Elasticsearch index ?
{'login': 'nsankar', 'id': 431890, 'node_id': 'MDQ6VXNlcjQzMTg5MA==', 'avatar_url': 'https://avatars.githubusercontent.com/u/431890?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/nsankar', 'html_url': 'https://github.com/nsankar', 'followers_url': 'https://api.github.com/users/nsankar/followers', 'following_url': 'https://api.github.com/users/nsankar/following{/other_user}', 'gists_url': 'https://api.github.com/users/nsankar/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/nsankar/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/nsankar/subscriptions', 'organizations_url': 'https://api.github.com/users/nsankar/orgs', 'repos_url': 'https://api.github.com/users/nsankar/repos', 'events_url': 'https://api.github.com/users/nsankar/events{/privacy}', 'received_events_url': 'https://api.github.com/users/nsankar/received_events', 'type': 'User', 'site_admin': False}
[]
[]
['Datasets does not use elasticsearch API to define custom similarity. If you want to use a custom similarity, the best would be to run a curl request directly to your elasticsearch instance (see sample hereafter, directly from ES documentation), then you should be able to use `my_similarity` in your configuration passed to datasets\r\n\r\n```\r\ncurl -X PUT "localhost:9200/index?pretty" -H \'Content-Type: application/json\' -d\'\r\n{\r\n  "settings": {\r\n    "index": {\r\n      "similarity": {\r\n        "my_similarity": {\r\n          "type": "DFR",\r\n          "basic_model": "g",\r\n          "after_effect": "l",\r\n          "normalization": "h2",\r\n          "normalization.h2.c": "3.0"\r\n        }\r\n      }\r\n    }\r\n  }\r\n}\r\n\'\r\n\r\n```']
**QUESTION : How should we use other similarity algorithms supported by Elasticsearch other than "BM25"  ?**
**ES Reference**
https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-similarity.html
**HF doc reference:**
https://huggingface.co/docs/datasets/faiss_and_ea.html

**context :**
========

I used the latest Elasticsearch server  version 7.9.2
When I set DFR  which is one of the other similarity algorithms supported by elasticsearch  in the mapping, I get an error

For example DFR that I had tried in the first instance in mappings as below.,
`"mappings": {"properties": {"text": {"type": "text", "analyzer": "standard", "similarity": "DFR"}}},`

I get the following error 
RequestError: RequestError(400, 'mapper_parsing_exception', 'Unknown Similarity type [DFR] for field [text]')

The other thing as another option I had tried was to declare "similarity": "my_similarity" within settings and then assigning "my_similarity" inside the mappings as below 

`es_config = {
        "settings": {
            "number_of_shards": 1,
             **"similarity":  "my_similarity"**: {
          "type": "DFR",
          "basic_model": "g",
          "after_effect": "l",
          "normalization": "h2",
          "normalization.h2.c": "3.0"
        } ,
            "analysis": {"analyzer": {"stop_standard": {"type": "standard", " stopwords": "_english_"}}},
            
        },
        "mappings": {"properties": {"text": {"type": "text", "analyzer": "standard", "similarity": "my_similarity"}}},
    }`

For this , I got the following error
RequestError: RequestError(400, 'illegal_argument_exception', 'unknown setting [index.similarity] please check that any required plugins are installed, or check the breaking changes documentation for removed settings')


{'url': 'https://api.github.com/repos/huggingface/datasets/issues/709/reactions', 'total_count': 0, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}