Add a metric for the MATH dataset (competition_math).
{'login': 'hacobe', 'id': 91226467, 'node_id': 'MDQ6VXNlcjkxMjI2NDY3', 'avatar_url': 'https://avatars.githubusercontent.com/u/91226467?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/hacobe', 'html_url': 'https://github.com/hacobe', 'followers_url': 'https://api.github.com/users/hacobe/followers', 'following_url': 'https://api.github.com/users/hacobe/following{/other_user}', 'gists_url': 'https://api.github.com/users/hacobe/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/hacobe/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/hacobe/subscriptions', 'organizations_url': 'https://api.github.com/users/hacobe/orgs', 'repos_url': 'https://api.github.com/users/hacobe/repos', 'events_url': 'https://api.github.com/users/hacobe/events{/privacy}', 'received_events_url': 'https://api.github.com/users/hacobe/received_events', 'type': 'User', 'site_admin': False}
[]
[]
["I believe the only failed test related to this PR is tests/test_metric_common.py::LocalMetricTest::test_load_metric_competition_math. It gives the following error:\r\n\r\nImportError: To be able to use this dataset, you need to install the following dependencies['math_equivalence'] using 'pip install git+https://github.com/hendrycks/math.git' for instance'\r\n\r\nIt fails along with (these fail with ImportError as well):\r\ntest_load_metric_bertscore\r\ntest_load_metric_bleurt\r\ntest_load_metric_comet\r\ntest_load_metric_coval\r\n\r\nLet me know if there is anything I need to change."
 'Hi ! The script looks all good thanks :)\r\n\r\nTo fix the CI you just need to merge `master` into your branch\r\n```\r\ngit fetch upstream/master\r\ngit merge upstream/master\r\n```\r\n\r\nThen you also need to add `math_equivalence` to the list of git packages installed for the tests in `additional-tests-requirements.txt`\r\nhttps://github.com/huggingface/datasets/blob/ba831e4bcd175ae3d52afbf7d12c4f625bf541b0/additional-tests-requirements.txt#L1-L3'
 "I ran:\r\n\r\ngit fetch upstream\r\ngit merge upstream/master\r\n\r\nAnd I also added  math_equivalence to the list of git packages installed for the tests in additional-tests-requirements.txt\r\n\r\ntests/test_metric_common.py fails with the same errors as before. tests/test_dataset_cards.py also fails, but it doesn't look related to this PR (it's an issue datasets/ami/README.md)."
 '@lhoestq Anything else I can do? I re-merged again and am getting the same test failures as described in the previous comment.']
{'url': 'https://api.github.com/repos/huggingface/datasets/pulls/3020', 'html_url': 'https://github.com/huggingface/datasets/pull/3020', 'diff_url': 'https://github.com/huggingface/datasets/pull/3020.diff', 'patch_url': 'https://github.com/huggingface/datasets/pull/3020.patch', 'merged_at': datetime.datetime(2021, 10, 22, 10, 29, 31)}
This metric computes accuracy for the MATH dataset (https://arxiv.org/abs/2103.03874) after canonicalizing the prediction and the reference (e.g., converting "1/2" to "\\\\frac{1}{2}").
{'url': 'https://api.github.com/repos/huggingface/datasets/issues/3020/reactions', 'total_count': 0, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}