chenxwh commited on
Commit
4c00394
1 Parent(s): 2de3de7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -16
README.md CHANGED
@@ -129,27 +129,31 @@ python -m src.prediction.evaluate_veracity
129
 
130
  ## Format for submission files
131
 
132
- To facilitate human evaluation, the submission file should include the text of the evidence documents used, retrieved through the `url` field. If external knowledge is utilized, please provide the scraped text. If our provided knowledge store is used, this can be achieved by running the following code block (see [veracity_with_scraped_text.py](https://huggingface.co/chenxwh/AVeriTeC/blob/main/src/prediction/veracity_with_scraped_text.py) for adding the text to the previous prediction file. An example output for the dev set is [here](https://huggingface.co/chenxwh/AVeriTeC/blob/main/data_store/dev_veracity_prediction_for_submission.json).
133
  ```bash
134
  python -m src.prediction.veracity_with_scraped_text --knowledge_store_dir <directory_of_the_knowledge_store>
135
  ```
136
 
137
- Each line of the final submission file is a json object with the following information:
138
  ```json
139
- {
140
- "claim_id": "The ID of the sample.",
141
- "claim": "The claim text itself.",
142
- "pred_label": "The predicted label of the claim.",
143
- "evidence": [
144
- {
145
- "question": "The text of the generated question.",
146
- "answer": "The text of the answer to the generated question.",
147
- "url": "The source URL for the answer.",
148
- "scraped_text": "The text scraped from the URL."
149
- }
150
- ]
151
- }
152
-
 
 
 
 
153
  ```
154
 
155
  ## Citation
 
129
 
130
  ## Format for submission files
131
 
132
+ To facilitate human evaluation, the submission file should include the text of the evidence documents used, retrieved through the `url` field. If external knowledge is utilized, please provide the scraped text. If our provided knowledge store is used, this can be achieved by running the following code block (see [veracity_with_scraped_text.py](https://huggingface.co/chenxwh/AVeriTeC/blob/main/src/prediction/veracity_with_scraped_text.py) for adding the text to the previous prediction file). An example output for the dev set is [here](https://huggingface.co/chenxwh/AVeriTeC/blob/main/data_store/dev_veracity_prediction_for_submission.json).
133
  ```bash
134
  python -m src.prediction.veracity_with_scraped_text --knowledge_store_dir <directory_of_the_knowledge_store>
135
  ```
136
 
137
+ The final submission file should include the following information:
138
  ```json
139
+ [
140
+ {
141
+ "claim_id": "The ID of the sample.",
142
+ "claim": "The claim text itself.",
143
+ "pred_label": "The predicted label of the claim.",
144
+ "evidence": [
145
+ {
146
+ "question": "The text of the generated question.",
147
+ "answer": "The text of the answer to the generated question.",
148
+ "url": "The source URL for the answer.",
149
+ "scraped_text": "The text scraped from the URL."
150
+ }
151
+ ]
152
+ },
153
+ {
154
+ ...
155
+ },
156
+ ]
157
  ```
158
 
159
  ## Citation