region:us
---
dataset_info:
  features:
  - name: image
    dtype: image
  - name: text
    dtype: string
  splits:
  - name: train
    num_bytes: 132661697.28
    num_examples: 1265
  download_size: 118220818
  dataset_size: 132661697.28
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
---
# Dataset Card for "wildreceipts_ocr_train"



Dataset Summary
-----------------------------
This is collection of receipts images with enhanced text information source from Wildreceipts and additional curated receipt images. 
It contains photo and OCRs information of each image including words, bounding box, labels and key information extraction data in json and xml format.

Features and Data Structure
-----------------------------

visual data
- Receipt image represent complex layouts, the effects are well demonstrated on each image.

text data  
- ocr_json   - represent extracted receipt key information data in json format
- ocr_boxes  - represent up-to-date ocr scan result as grouth truth in raw format
- ocr_words  - represent ocr detected and recognized words from the receipt image 
- ocr_labels - represent original mapping of labels class and text position (may deviate from actual ocr scan result)
- ocr_xml    - represent xml format of the key information
- ocr_kie    - represent extraction of key information from the receipt image

Languages
The language of the data is primarily English.

Data Instances
A data instance in this dataset represents entries from the Receipt collection which have been augmented. 

Data Samples
-----------------------------
Image:
file_name: receipt_0.jpeg

Sample: ocr_words
-----------------------------
['CHO EUN', 'KOREAN RESTAURANT', '2621 ORANGETHORPE AVE,FULLERTON.', '714879-3574', 'THANKYOU!!', 'DATE12/30/2016 FRI', 'TIME19:19', 'BIBIM.OCTOPU T1', '$13.99', 'S-FOODP.CAKT1', '$14.99', 'PORK DUMPLIN T1', '$8.99', 'LA BEEF RIB T1', '$17.99', '4.00xITEMS', 'SUBTOTAL', '$55.96', 'TAX1', '$4.48', 'TOTAL', '$60.44', '$60AA']

Sample: ocr_json
-----------------------------
{"store_name": "CHOEUN KOREANRESTAURANT", "store_addr": "2621ORANGETHORPEAVE,FULLERTON.", "telephone": "(714)879-3574", "date": "12/30/2016FRI", "time": "19:19", "subtotal": "$55.96", "tax": "$4.48", "total": "$60.44", "ignore": " ", "tips": "", "line_items": [{"item_key": "", "item_name": "BIBIM.OCTOPUT1", "item_value": "$13.99", "item_quantity": "1"}, {"item_key": "", "item_name": "S-FOODP.CAKT1", "item_value": "$14.99", "item_quantity": "1"}, {"item_key": "", "item_name": "PORKDUMPLINT1", "item_value": "$8.99", "item_quantity": "1"}, {"item_key": "", "item_name": "LABEEFRIBT1", "item_value": "\uffe517.99", "item_quantity": "1"}, {"item_key": "4.00xITEMS", "item_name": "", "item_value": "", "item_quantity": ""}]}

Sample: ocr_xml
-----------------------------
<s_receipt><s_total>$60.44</s_total><s_tips></s_tips><s_time>19:19</s_time><s_telephone>(714)879-3574</s_telephone><s_tax>$4.48</s_tax><s_subtotal>$55.96</s_subtotal><s_store_name>CHOEUN KOREANRESTAURANT</s_store_name><s_store_addr>2621ORANGETHORPEAVE,FULLERTON.</s_store_addr><s_line_items><s_item_value>$13.99</s_item_value><s_item_quantity>1</s_item_quantity><s_item_name>BIBIM.OCTOPUT1</s_item_name><s_item_key></s_item_key><sep/><s_item_value>$14.99</s_item_value><s_item_quantity>1</s_item_quantity><s_item_name>S-FOODP.CAKT1</s_item_name><s_item_key></s_item_key><sep/><s_item_value>$8.99</s_item_value><s_item_quantity>1</s_item_quantity><s_item_name>PORKDUMPLINT1</s_item_name><s_item_key></s_item_key><sep/><s_item_value>￥17.99</s_item_value><s_item_quantity>1</s_item_quantity><s_item_name>LABEEFRIBT1</s_item_name><s_item_key></s_item_key><sep/><s_item_value></s_item_value><s_item_quantity></s_item_quantity><s_item_name></s_item_name><s_item_key>4.00xITEMS</s_item_key></s_line_items><s_ignore> </s_ignore><s_date>12/30/2016FRI</s_date></s_receipt>

Sample: ocr_kie
-----------------------------
[{'label': 'Store_name_value', 'transcription': 'CHOEUN'}, {'label': 'Store_name_value', 'transcription': 'KOREANRESTAURANT'}, {'label': 'Store_addr_value', 'transcription': '2621ORANGETHORPEAVE,FULLERTON.'}, {'label': 'Tel_value', 'transcription': '(714)879-3574'}, {'label': 'Others', 'transcription': 'THANKYOU!!'}, {'label': 'Date_key', 'transcription': 'DATE'}, {'label': 'Date_value', 'transcription': '12/30/2016FRI'}, {'label': 'Time_value', 'transcription': '19:19'}, {'label': 'Prod_item_value', 'transcription': 'BIBIM.OCTOPUT1'}, {'label': 'Prod_item_value', 'transcription': 'S-FOODP.CAKT1'}, {'label': 'Prod_item_value', 'transcription': 'PORKDUMPLINT1'}, {'label': 'Prod_item_value', 'transcription': 'LABEEFRIBT1'}, {'label': 'Prod_price_value', 'transcription': '$13.99'}, {'label': 'Prod_price_value', 'transcription': '$14.99'}, {'label': 'Prod_price_value', 'transcription': '$8.99'}, {'label': 'Prod_price_value', 'transcription': '￥17.99'}, {'label': 'Prod_item_key', 'transcription': '4.00xITEMS'}, {'label': 'Subtotal_key', 'transcription': 'SUBTOTAL'}, {'label': 'Tax_key', 'transcription': 'TAX1'}, {'label': 'Total_key', 'transcription': 'TOTAL'}, {'label': 'Subtotal_value', 'transcription': '$55.96'}, {'label': 'Tax_value', 'transcription': '$4.48'}, {'label': 'Total_value', 'transcription': '$60.44'}, {'label': 'Ignore', 'transcription': ''}, {'label': 'Ignore', 'transcription': ''}, {'label': 'Time_key', 'transcription': 'TIME'}]

Sample: ocr_labels
-----------------------------
[{'label': 'Store_name_value', 'transcription': 'CHOEUN', 'points': [[114.0, 19.0], [230.0, 19.0], [230.0, 1.0], [114.0, 1.0]]}, {'label': 'Store_name_value', 'transcription': 'KOREANRESTAURANT', 'points': [[97.0, 35.0], [236.0, 35.0], [236.0, 19.0], [97.0, 19.0]]}, {'label': 'Store_addr_value', 'transcription': '2621ORANGETHORPEAVE,FULLERTON.', 'points': [[29.0, 56.0], [295.0, 56.0], [295.0, 34.0], [29.0, 34.0]]}, {'label': 'Tel_value', 'transcription': '(714)879-3574', 'points': [[48.0, 73.0], [280.0, 73.0], [280.0, 54.0], [48.0, 54.0]]}, {'label': 'Others', 'transcription': 'THANKYOU!!', 'points': [[79.0, 92.0], [259.0, 92.0], [259.0, 74.0], [79.0, 74.0]]}, {'label': 'Date_key', 'transcription': 'DATE', 'points': [[22.0, 130.0], [61.0, 130.0], [61.0, 112.0], [22.0, 112.0]]}, {'label': 'Date_value', 'transcription': '12/30/2016FRI', 'points': [[70.0, 131.0], [192.0, 131.0], [192.0, 112.0], [70.0, 112.0]]}, {'label': 'Time_value', 'transcription': '19:19', 'points': [[263.0, 128.0], [307.0, 128.0], [307.0, 111.0], [263.0, 111.0]]}, {'label': 'Prod_item_value', 'transcription': 'BIBIM.OCTOPUT1', 'points': [[19.0, 168.0], [157.0, 168.0], [157.0, 149.0], [19.0, 149.0]]}, {'label': 'Prod_item_value', 'transcription': 'S-FOODP.CAKT1', 'points': [[17.0, 190.0], [158.0, 190.0], [158.0, 171.0], [17.0, 171.0]]}, {'label': 'Prod_item_value', 'transcription': 'PORKDUMPLINT1', 'points': [[14.0, 214.0], [158.0, 214.0], [158.0, 192.0], [14.0, 192.0]]}, {'label': 'Prod_item_value', 'transcription': 'LABEEFRIBT1', 'points': [[14.0, 236.0], [151.0, 236.0], [151.0, 215.0], [14.0, 215.0]]}, {'transcription': '$13.99', 'points': [[254.0, 168.0], [312.0, 168.0], [312.0, 149.0], [254.0, 149.0]]}, {'transcription': '$14.99', 'points': [[257.0, 189.0], [314.0, 189.0], [314.0, 170.0], [257.0, 170.0]]}, {'transcription': '$8.99', 'points': [[268.0, 212.0], [316.0, 212.0], [316.0, 191.0], [268.0, 191.0]]}, {'transcription': '￥17.99', 'points': [[261.0, 234.0], [318.0, 234.0], [318.0, 213.0], [261.0, 213.0]]}, {'label': 'Prod_item_key', 'transcription': '4.00xITEMS', 'points': [[118.0, 260.0], [217.0, 260.0], [217.0, 239.0], [118.0, 239.0]]}, {'label': 'Subtotal_key', 'transcription': 'SUBTOTAL', 'points': [[8.0, 285.0], [91.0, 285.0], [91.0, 264.0], [8.0, 264.0]]}, {'label': 'Tax_key', 'transcription': 'TAX1', 'points': [[8.0, 312.0], [49.0, 312.0], [49.0, 291.0], [8.0, 291.0]]}, {'label': 'Total_key', 'transcription': 'TOTAL', 'points': [[8.0, 336.0], [61.0, 336.0], [61.0, 316.0], [8.0, 316.0]]}, {'label': 'Subtotal_value', 'transcription': '$55.96', 'points': [[263.0, 283.0], [325.0, 283.0], [325.0, 260.0], [263.0, 260.0]]}, {'label': 'Tax_value', 'transcription': '$4.48', 'points': [[274.0, 308.0], [326.0, 308.0], [326.0, 286.0], [274.0, 286.0]]}, {'label': 'Total_value', 'transcription': '$60.44', 'points': [[267.0, 334.0], [328.0, 334.0], [328.0, 310.0], [267.0, 310.0]]}, {'label': 'Ignore', 'transcription': '', 'points': [[269.0, 347.0], [328.0, 347.0], [328.0, 336.0], [269.0, 336.0]]}, {'label': 'Ignore', 'transcription': '', 'points': [[11.0, 347.0], [50.0, 347.0], [50.0, 342.0], [11.0, 342.0]]}, {'label': 'Time_key', 'transcription': 'TIME', 'points': [[215.0, 128.0], [253.0, 128.0], [253.0, 112.0], [215.0, 112.0]]}]

Sample: ocr_boxes
-----------------------------
[[[[113.0, 0.0], [228.0, 3.0], [227.0, 20.0], [113.0, 17.0]], ('CHO EUN', 0.9466678500175476)], [[[96.0, 17.0], [236.0, 21.0], [236.0, 38.0], [96.0, 33.0]], ('KOREAN RESTAURANT', 0.9685913324356079)], [[[28.0, 32.0], [293.0, 37.0], [292.0, 56.0], [28.0, 51.0]], ('2621 ORANGETHORPE AVE,FULLERTON.', 0.951709508895874)], [[[48.0, 53.0], [279.0, 56.0], [279.0, 73.0], [47.0, 70.0]], ('714879-3574', 0.9919183850288391)], [[[81.0, 75.0], [256.0, 75.0], [256.0, 89.0], [81.0, 89.0]], ('THANKYOU!!', 0.9518492817878723)], [[[24.0, 113.0], [191.0, 113.0], [191.0, 127.0], [24.0, 127.0]], ('DATE12/30/2016 FRI', 0.9638745784759521)], [[[214.0, 111.0], [305.0, 109.0], [306.0, 125.0], [215.0, 128.0]], ('TIME19:19', 0.9523274898529053)], [[[18.0, 150.0], [156.0, 149.0], [156.0, 167.0], [18.0, 168.0]], ('BIBIM.OCTOPU T1', 0.9491282105445862)], [[[253.0, 147.0], [312.0, 144.0], [313.0, 166.0], [254.0, 168.0]], ('$13.99', 0.9204174876213074)], [[[16.0, 172.0], [157.0, 170.0], [157.0, 187.0], [16.0, 189.0]], ('S-FOODP.CAKT1', 0.9633263945579529)], [[[255.0, 168.0], [313.0, 168.0], [313.0, 189.0], [255.0, 189.0]], ('$14.99', 0.9975371956825256)], [[[15.0, 194.0], [157.0, 192.0], [157.0, 210.0], [15.0, 212.0]], ('PORK DUMPLIN T1', 0.9503927826881409)], [[[265.0, 190.0], [317.0, 188.0], [318.0, 209.0], [266.0, 212.0]], ('$8.99', 0.9171518087387085)], [[[12.0, 217.0], [149.0, 213.0], [149.0, 233.0], [12.0, 236.0]], ('LA BEEF RIB T1', 0.925663948059082)], [[[258.0, 213.0], [319.0, 210.0], [320.0, 232.0], [259.0, 235.0]], ('$17.99', 0.9976120591163635)], [[[119.0, 237.0], [217.0, 237.0], [217.0, 258.0], [119.0, 258.0]], ('4.00xITEMS', 0.9557921290397644)], [[[9.0, 264.0], [90.0, 262.0], [90.0, 284.0], [9.0, 286.0]], ('SUBTOTAL', 0.9968011379241943)], [[[263.0, 261.0], [324.0, 259.0], [325.0, 281.0], [264.0, 283.0]], ('$55.96', 0.9971590042114258)], [[[8.0, 289.0], [50.0, 289.0], [50.0, 311.0], [8.0, 311.0]], ('TAX1', 0.9973537921905518)], [[[273.0, 286.0], [326.0, 283.0], [328.0, 306.0], [274.0, 309.0]], ('$4.48', 0.991606593132019)], [[[9.0, 315.0], [61.0, 315.0], [61.0, 337.0], [9.0, 337.0]], ('TOTAL', 0.9985822439193726)], [[[266.0, 312.0], [328.0, 309.0], [328.0, 331.0], [267.0, 333.0]], ('$60.44', 0.9942547678947449)], [[[269.0, 334.0], [326.0, 334.0], [326.0, 347.0], [269.0, 347.0]], ('$60AA', 0.7674070596694946)]]


Curation Rationale
-----------------------------

The curated dataset was created to provide a source of OCR augmented text data for own personal AI research use. The datapoints are intended primarily to provide an enhancement of the core Receipt Image Collection data which relies upon the key information extraction from receipt image. 

Data Source and Prepratation
-----------------------------
1) This dataset use the great work from WildReceipt is a large receipt dataset collected from document images of unseen templates in the wild. It contains 25 key information categories, a total of about 69000 text boxes. Offical dataset: https://download.openmmlab.com/mmocr/data/wildreceipt.tar
2) OCR text data is generated using techniques OCR scaned on each image. 
3) Additional Post progressing OCR result into XML, JSON and Words format


License: 
Please check out the license of each subset in our curated dataset. 


[More Information needed](https://github.com/huggingface/datasets/blob/main/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)