<commit_before><div id="main_column" class="xxlarge-11 xlarge-10 large-9 medium-8 columns" style="padding-top:1rem;height:11489px" data-equalizer-watch="">

<!-- main column content - comment used to remove left column and center content on some pages -->  
	


<h1>Guide for producers</h1>

<p>This guide explains why and how producers can directly integrate photos and product data into the Open Food Facts database.</p>

<ul>
<li><a href="#introduction">Introduction</a></li>
<li><a href="#pourquoi_ajouter_les_donnees_de_ses_produits_dans_open_food_facts">Why add your product data to Open Food Facts</a></li>
<ul>
<li><a href="#exemples_de_producteurs_qui_transmettent_les_photos_et_donnees_de_leurs_produits_a_open_food_facts">Examples of producers who submit photos and product data to Open Food Facts</a></li>
</ul>
<li><a href="#comment_ajouter_les_donnees_de_ses_produits_dans_open_food_facts">How to add data for your products in Open Food Facts</a></li>
<ul>
<li><a href="#licence">Licence</a></li>
<li><a href="#quelles_donnees">What data ?</a></li>
<ul>
<li><a href="#une_base_dans_toutes_les_langues">A database in all languages</a></li>
<li><a href="#photos_et_donnees">Photos and Data</a></li>
</ul>
<li><a href="#ajout_manuel">Manual addition</a></li>
<li><a href="#ajout_automatise">Automated addition</a></li>
<ul>
<li><a href="#format_des_images">Picture format</a></li>
<ul>
<li><a href="#format_des_fichiers_images">Format of image files</a></li>
<li><a href="#nom_des_fichiers_images">Name of the image files</a></li>
</ul>
<li><a href="#format_des_donnees">Data format</a></li>
<ul>
<li><a href="#champs_obligatoires">Required fields</a></li>
<li><a href="#champs_optionnels">Optional fields</a></li>
</ul>
<li><a href="#envoi_des_photos_et_des_donnees">Sending photos and data</a></li>
</ul>
</ul>
<li><a href="#questions_frequentes">常見問答</a></li>
<ul>
<li><a href="#est_ce_que_le_referencement_des_produits_est_gratuit">Is the listing of products free?</a></li>
<li><a href="#les_donnees_des_producteurs_vont_elles_ecraser_les_donnees_saisies_par_les_contributeurs">Will producer data overwrite data entered by contributors?</a></li>
<li><a href="#est_il_possible_de_supprimer_les_produits_qui_ne_sont_plus_vendus">Is it possible to delete products that are no longer sold?</a></li>
<li><a href="#est_ce_que_les_produits_pour_animaux_les_produits_non_alimentaires_les_cosmetiques_et_les_autres_produits_peuvent_etre_integres_a_open_food_facts">Can animal products, non-food products, cosmetics and other products be integrated into Open Food Facts?</a></li>
</ul>
</ul>

<h2 id="introduction">Introduction</h2>

<p>The photos and data on the food products referenced in the Open Food Facts database come from 2 sources:</p>
<ul>
<li>crowdsourcing through mobile apps and the Open Food Facts website: contributors scan barcodes, send product photos, ingredient lists and nutrition charts and then enter the data in a structured way.</li>
  <li>the addition of data directly from the producers, either manually or automatically.</li>
</ul>


<h2 id="pourquoi_ajouter_les_donnees_de_ses_produits_dans_open_food_facts">Why add product data to Open Food Facts</h2>
<p>There are many benefits to importing date directly from manufacturers:</p>

<ul>
<li>The data is complete, up to date, and without risk of error.</li>
<li>The photos are of better quality than the photos taken by the contributors in very variable conditions.</li>
<li>The data is then available in more than 100 applications.</li>
<li>You can easily obtain statistics about your products and export the data of your products supplemented by indicators calculated by Open Food Facts such as the Nutri-Score.</li>
</ul>
  
  
<h3 id="exemples_de_producteurs_qui_transmettent_les_photos_et_donnees_de_leurs_produits_a_open_food_facts">Examples of producers who submit photos and product data to Open Food Facts</h3>

<p>A growing number of high level French producers are sending us the data of their products: Fleury Michon, Sodebo, System U, Carrefour, Casino, Franprix, Monoprix.</p>

<h2 id="comment_ajouter_les_donnees_de_ses_produits_dans_open_food_facts">How to add the data of your products in Open Food Facts</h2>

<h3 id="licence">Licence</h3>
  
<p>All data sent to Open Food Facts is published under the Open Database License (ODbL), and the photos under the Creative Commons License CC-BY-SA. These licenses allow the data to be freely reusable by all and for all purposes. For more information, see the <a href="https://fr.openfoodfacts.org/conditions-d-utilisation">Terms of Use, Contribution, and Reuse.</a></p>
  
<h3 id="quelles_donnees">What data</h3>

<p>All the data you have can be integrated into Open Food Facts: ingredients and nutritional information of course, but also labels, materials used for packaging, traceability of ingredients, instructions for preparation and recycling, ecological footprint etc.</p>
<p>We regularly add new fields to the database based on the information on the packages or provided by the manufacturers.</p>
We have recently added fields on the carbon footprint, and the packaging of products, which are already highlighted on Open Food Facts and in applications that help to change the behavior in terms of recycling.<p></p>

 <h4 id="une_base_dans_toutes_les_langues">A database in all languages</h4>
  
<p>The Open Food Facts database, site and mobile applications are also fully multilingual and available in all countries. If you have photos and data in multiple languages, we can also integrate them into Open Food Facts.</p>

<h4 id="photos_et_donnees">Photos and Data</h4>

<p>One of the fundamental principles of Open Food Facts is that the user or contributor must be able to verify the information from a photo.</p>
<p>We therefore ask you to send us photos of the product and packaging (possibly as a flat file of the packaging) to support the structured data you send.</p>

<h3 id="ajout_manuel">Manual addition</h3>

<p>If you have a small number of products whose formulation does not change often, you can create an account on Open Food Facts and add yourself photos and data of your products.</p>
<p>Highly recommended method: Add photos and enter information from the <a href="https://world.openfoodfacts.org">Open Food Facts website</a> .</p>
<p>Alternative method: Use the Android mobile application to scan and then photograph and finally enter a product completely.</p>
<p>Thank you in advance for telling us if you add your products yourself so that we can identify your account as a producer account, and indicate that the information comes directly from your company.</p>

<h3 id="ajout_automatise">Automated addition</h3>

To make importing data easier and faster, we have defined a standard format for images and data:
<h4 id="format_des_images">Picture format</h4>

<p>At least 3 images are required:</p>
<ul>
<li>Front view of product on white background (avoid three-quarter views that do not allow visual recognition of products without barcode scanning)</li>
<li>成分列表</li>
<li>Nutritional table</li>
</ul>
  
<p>The front view, ingredients and nutrition chart must be provided in at least the main language of the product. It is recommended that you also provide images for other languages ​​when they appear on the product.</p>

<p>For example, a product intended for the Belgian market may have photos showing the product written in French and written in Dutch.</p>

  <p>Other images may also be included:</p>
<ul>
<li>Verso and other views of the product (they allow in particular to automatically identify labels logos etc.)</li>
<li>Flat view of the packaging</li>
</ul>
  
<h5 id="format_des_fichiers_images">Format of image files</h5>
<ul>
  <li>JPEG or PNG format</li>
  <li>Size at least 2000 pixels wide or tall</li>
</ul>

<h5 id="nom_des_fichiers_images">Name of the image files</h5>


<pre>[barcode]_[type of image]_[language code].[jpg or png]</pre>

  <p>For example :</p>

<p>3660140808750_front_en.jpg for the front image of the product whose barcode is 3660140808750</p>

  <p>The types of images are:</p>

<ul>
  <!-- Do not translate field names, just their description -->
  <li>front: front image</li>
  <li>ingredients: list of ingredients</li>
  <li>nutrition: nutritional chart</li>
  <li>back: version</li>
  <li>any name you see fit (characters from a to e, no spaces) for other views</li>
</ul>
  

<h4 id="format_des_donnees">Data format</h4>

<p>The data must be provided in a table in CSV format, with tabulation as field separator character and UTF-8 character encoding.</p>
<p>The first line contains the field names, and each of the products occupies one of the following lines. The order of the fields does not matter, and not all columns need to be present.</p>
<p>Some fields are mandatory and others are optional. We encourage you to fill in as many fields as possible so that the data can be used by as many applications as possible. If in doubt, or if you have data for which there are not yet fields in the Open Food Facts database, you can contact us so that we can add fields and make the best possible match.</p>

  <h5 id="champs_obligatoires">Required fields</h5>

<ul>  
  <!-- Do not translate field names, just their description -->
  <li>code: product barcode as it appears on the packaging (EAN-13, EAN-8 or other)</li>
  <li>lang: 2-letter code of the main language (most present on the packaging) of the product (example: fr for French)</li>
  <li>product_name_ [2-letter code of the main language]: product name (in the field product_name_en for the name in French)</li>
  <li>generic_name_ [2-letter code of the main language]: Generic product name </li>
  <li>brands: brands (several possible values for brands and sub-brands, to be separated by commas)</li>
  <li>quantity: quantity, specify the unit, and possibly add in parentheses if the product contains several units (example: 120 g (4 x 30 g))</li>
  <li>categories: product category, in the main product language (example: chocolate cakes)</li>
  <li>countries: country where the product is sold, in the main language of the product (several possible values, to be separated by commas)</li>
  <li>ingredients_text_ [2-letter code of the main language]: list of ingredients (in the ingredients_text_fr field for the list in French). Allergens may be indicated in CAPITAL LETTERS or between </li>
  <li>allergens: allergens, in the main language of the product, separated by commas</li>
  <li>traces: traces of allergens, in the main language of the product, separated by commas</li>
</ul>

<p>Among the mandatory fields are the nutritional values that appear on the package, per 100g / 100ml of product as sold.</p>
<p>All values must have no unit (the unit is kJ for energy and g for all other fields).</p>

<ul>
  <!-- Do not translate nutrient field names -->
  <li>energy_100g: energy, in kJ</li>
  <li>fat_100g: fat</li>
  <li>saturated-fat_100g: saturated fat</li>
  <li>carbohydrates_100g: carbohydrates</li>
  <li>sugars_100g: sugars</li>
  <li>proteins_100g: proteins</li>
  <li>salt_100g: salt</li>
  <!-- Do not translate nutrient field names -->
</ul>

<p>It is strongly recommended to also indicate the fibers and the % of fruits and vegetables so that this information can be taken into account in the calculation of Nutri-Score:</p>

  <ul>
    <!-- Do not translate nutrient field names -->
  <li>fiber_100g: fibers</li>
  <li>fruits-vegetables-nuts_100g</li>
    <!-- Do not translate nutrient field names -->
  </ul>
  
  <h5 id="champs_optionnels">Optional fields</h5>

  <ul>
    <li>labels: labels, in the main language of the product (several possible values, to be separated by commas) (example: AB Bio, MSC, Max Havelaar). Note: the labels must appear on the packaging of the product.</li>
    <li>origins: origins of the ingredients, in the main language of the product (several possible values, to be separated by commas) (example: Peru, Périgord)</li>
    <li>packaging: materials and forms of the product packaging, in the main language of the product (several possible values, to be separated by commas) (example: can, aluminum)</li>
    <li>manufacturing_places: manufacturing locations, in the main language of the product (several possible values, to be separated by commas: city names, countries)</li>
    <li>emb_codes: packager codes and European sanitary stamps (several possible values, to be separated by commas)</li>
    <li>stores: places of sale (several possible values, to be separated by commas: names of supermarket chains)</li>
    <li>nutri_score: Nutri-Score rating from A to E, only in case it is indicated on the package</li>
    <li>serving_size: quantity of a serving, specify the unit</li>
  </ul>

<p>Nutritional values can also be indicated for the prepared product (with water, milk etc.). Then add "_prepared" to the name of the field. For example energy_prepared_100g.</p>
<p>It is also possible to indicate the values for other nutrients:</p>

<ul>
  <!-- Do not translate nutrient field names -->
  
<li>energy-from-fat_100g</li>
<li>butyric-acid_100g</li>
<li>caproic-acid_100g</li>
<li>caprylic-acid_100g</li>
<li>capric-acid_100g</li>
<li>lauric-acid_100g</li>
<li>myristic-acid_100g</li>
<li>palmitic-acid_100g</li>
<li>stearic-acid_100g</li><commit_msg>New translations producers.html (Chinese Traditional, Hong Kong)<commit_after><div id="main_column" class="xxlarge-11 xlarge-10 large-9 medium-8 columns" style="padding-top:1rem;height:11489px" data-equalizer-watch="">

<!-- main column content - comment used to remove left column and center content on some pages -->  
	


<h1>生產商指南</h1>

<p>This guide explains why and how producers can directly integrate photos and product data into the Open Food Facts database.</p>

<ul>
<li><a href="#introduction">簡介</a></li>
<li><a href="#pourquoi_ajouter_les_donnees_de_ses_produits_dans_open_food_facts">為甚麼要將產品數據加入Open Food Facts</a></li>
<ul>
<li><a href="#exemples_de_producteurs_qui_transmettent_les_photos_et_donnees_de_leurs_produits_a_open_food_facts">Examples of producers who submit photos and product data to Open Food Facts</a></li>
</ul>
<li><a href="#comment_ajouter_les_donnees_de_ses_produits_dans_open_food_facts">How to add data for your products in Open Food Facts</a></li>
<ul>
<li><a href="#licence">Licence</a></li>
<li><a href="#quelles_donnees">甚麼數據？</a></li>
<ul>
<li><a href="#une_base_dans_toutes_les_langues">A database in all languages</a></li>
<li><a href="#photos_et_donnees">照片和數據</a></li>
</ul>
<li><a href="#ajout_manuel">手動添加</a></li>
<li><a href="#ajout_automatise">自動添加</a></li>
<ul>
<li><a href="#format_des_images">圖片格式</a></li>
<ul>
<li><a href="#format_des_fichiers_images">圖片檔案格式</a></li>
<li><a href="#nom_des_fichiers_images">圖片檔案名</a></li>
</ul>
<li><a href="#format_des_donnees">數據格式</a></li>
<ul>
<li><a href="#champs_obligatoires">必填項目</a></li>
<li><a href="#champs_optionnels">Optional fields</a></li>
</ul>
<li><a href="#envoi_des_photos_et_des_donnees">Sending photos and data</a></li>
</ul>
</ul>
<li><a href="#questions_frequentes">常見問答</a></li>
<ul>
<li><a href="#est_ce_que_le_referencement_des_produits_est_gratuit">Is the listing of products free?</a></li>
<li><a href="#les_donnees_des_producteurs_vont_elles_ecraser_les_donnees_saisies_par_les_contributeurs">Will producer data overwrite data entered by contributors?</a></li>
<li><a href="#est_il_possible_de_supprimer_les_produits_qui_ne_sont_plus_vendus">Is it possible to delete products that are no longer sold?</a></li>
<li><a href="#est_ce_que_les_produits_pour_animaux_les_produits_non_alimentaires_les_cosmetiques_et_les_autres_produits_peuvent_etre_integres_a_open_food_facts">Can animal products, non-food products, cosmetics and other products be integrated into Open Food Facts?</a></li>
</ul>
</ul>

<h2 id="introduction">Introduction</h2>

<p>The photos and data on the food products referenced in the Open Food Facts database come from 2 sources:</p>
<ul>
<li>crowdsourcing through mobile apps and the Open Food Facts website: contributors scan barcodes, send product photos, ingredient lists and nutrition charts and then enter the data in a structured way.</li>
  <li>the addition of data directly from the producers, either manually or automatically.</li>
</ul>


<h2 id="pourquoi_ajouter_les_donnees_de_ses_produits_dans_open_food_facts">Why add product data to Open Food Facts</h2>
<p>There are many benefits to importing date directly from manufacturers:</p>

<ul>
<li>The data is complete, up to date, and without risk of error.</li>
<li>The photos are of better quality than the photos taken by the contributors in very variable conditions.</li>
<li>The data is then available in more than 100 applications.</li>
<li>You can easily obtain statistics about your products and export the data of your products supplemented by indicators calculated by Open Food Facts such as the Nutri-Score.</li>
</ul>
  
  
<h3 id="exemples_de_producteurs_qui_transmettent_les_photos_et_donnees_de_leurs_produits_a_open_food_facts">Examples of producers who submit photos and product data to Open Food Facts</h3>

<p>A growing number of high level French producers are sending us the data of their products: Fleury Michon, Sodebo, System U, Carrefour, Casino, Franprix, Monoprix.</p>

<h2 id="comment_ajouter_les_donnees_de_ses_produits_dans_open_food_facts">How to add the data of your products in Open Food Facts</h2>

<h3 id="licence">Licence</h3>
  
<p>All data sent to Open Food Facts is published under the Open Database License (ODbL), and the photos under the Creative Commons License CC-BY-SA. These licenses allow the data to be freely reusable by all and for all purposes. For more information, see the <a href="https://fr.openfoodfacts.org/conditions-d-utilisation">Terms of Use, Contribution, and Reuse.</a></p>
  
<h3 id="quelles_donnees">What data</h3>

<p>All the data you have can be integrated into Open Food Facts: ingredients and nutritional information of course, but also labels, materials used for packaging, traceability of ingredients, instructions for preparation and recycling, ecological footprint etc.</p>
<p>We regularly add new fields to the database based on the information on the packages or provided by the manufacturers.</p>
We have recently added fields on the carbon footprint, and the packaging of products, which are already highlighted on Open Food Facts and in applications that help to change the behavior in terms of recycling.<p></p>

 <h4 id="une_base_dans_toutes_les_langues">A database in all languages</h4>
  
<p>The Open Food Facts database, site and mobile applications are also fully multilingual and available in all countries. If you have photos and data in multiple languages, we can also integrate them into Open Food Facts.</p>

<h4 id="photos_et_donnees">照片和數據</h4>

<p>One of the fundamental principles of Open Food Facts is that the user or contributor must be able to verify the information from a photo.</p>
<p>We therefore ask you to send us photos of the product and packaging (possibly as a flat file of the packaging) to support the structured data you send.</p>

<h3 id="ajout_manuel">手動添加</h3>

<p>If you have a small number of products whose formulation does not change often, you can create an account on Open Food Facts and add yourself photos and data of your products.</p>
<p>Highly recommended method: Add photos and enter information from the <a href="https://world.openfoodfacts.org">Open Food Facts website</a> .</p>
<p>Alternative method: Use the Android mobile application to scan and then photograph and finally enter a product completely.</p>
<p>Thank you in advance for telling us if you add your products yourself so that we can identify your account as a producer account, and indicate that the information comes directly from your company.</p>

<h3 id="ajout_automatise">自動添加</h3>

To make importing data easier and faster, we have defined a standard format for images and data:
<h4 id="format_des_images">圖片格式</h4>

<p>At least 3 images are required:</p>
<ul>
<li>Front view of product on white background (avoid three-quarter views that do not allow visual recognition of products without barcode scanning)</li>
<li>成分列表</li>
<li>Nutritional table</li>
</ul>
  
<p>The front view, ingredients and nutrition chart must be provided in at least the main language of the product. It is recommended that you also provide images for other languages ​​when they appear on the product.</p>

<p>For example, a product intended for the Belgian market may have photos showing the product written in French and written in Dutch.</p>

  <p>Other images may also be included:</p>
<ul>
<li>Verso and other views of the product (they allow in particular to automatically identify labels logos etc.)</li>
<li>Flat view of the packaging</li>
</ul>
  
<h5 id="format_des_fichiers_images">圖片檔案格式</h5>
<ul>
  <li>JPEG or PNG format</li>
  <li>Size at least 2000 pixels wide or tall</li>
</ul>

<h5 id="nom_des_fichiers_images">圖片檔案名</h5>


<pre>[barcode]_[type of image]_[language code].[jpg or png]</pre>

  <p>For example :</p>

<p>3660140808750_front_en.jpg for the front image of the product whose barcode is 3660140808750</p>

  <p>The types of images are:</p>

<ul>
  <!-- Do not translate field names, just their description -->
  <li>front: front image</li>
  <li>ingredients: list of ingredients</li>
  <li>nutrition: nutritional chart</li>
  <li>back: version</li>
  <li>any name you see fit (characters from a to e, no spaces) for other views</li>
</ul>
  

<h4 id="format_des_donnees">數據格式</h4>

<p>The data must be provided in a table in CSV format, with tabulation as field separator character and UTF-8 character encoding.</p>
<p>The first line contains the field names, and each of the products occupies one of the following lines. The order of the fields does not matter, and not all columns need to be present.</p>
<p>Some fields are mandatory and others are optional. We encourage you to fill in as many fields as possible so that the data can be used by as many applications as possible. If in doubt, or if you have data for which there are not yet fields in the Open Food Facts database, you can contact us so that we can add fields and make the best possible match.</p>

  <h5 id="champs_obligatoires">必填項目</h5>

<ul>  
  <!-- Do not translate field names, just their description -->
  <li>code: product barcode as it appears on the packaging (EAN-13, EAN-8 or other)</li>
  <li>lang: 2-letter code of the main language (most present on the packaging) of the product (example: fr for French)</li>
  <li>product_name_ [2-letter code of the main language]: product name (in the field product_name_en for the name in French)</li>
  <li>generic_name_ [2-letter code of the main language]: Generic product name </li>
  <li>brands: brands (several possible values for brands and sub-brands, to be separated by commas)</li>
  <li>quantity: quantity, specify the unit, and possibly add in parentheses if the product contains several units (example: 120 g (4 x 30 g))</li>
  <li>categories: product category, in the main product language (example: chocolate cakes)</li>
  <li>countries: country where the product is sold, in the main language of the product (several possible values, to be separated by commas)</li>
  <li>ingredients_text_ [2-letter code of the main language]: list of ingredients (in the ingredients_text_fr field for the list in French). Allergens may be indicated in CAPITAL LETTERS or between </li>
  <li>allergens: allergens, in the main language of the product, separated by commas</li>
  <li>traces: traces of allergens, in the main language of the product, separated by commas</li>
</ul>

<p>Among the mandatory fields are the nutritional values that appear on the package, per 100g / 100ml of product as sold.</p>
<p>All values must have no unit (the unit is kJ for energy and g for all other fields).</p>

<ul>
  <!-- Do not translate nutrient field names -->
  <li>energy_100g: energy, in kJ</li>
  <li>fat_100g: fat</li>
  <li>saturated-fat_100g: saturated fat</li>
  <li>carbohydrates_100g: carbohydrates</li>
  <li>sugars_100g: sugars</li>
  <li>proteins_100g: proteins</li>
  <li>salt_100g: salt</li>
  <!-- Do not translate nutrient field names -->
</ul>

<p>It is strongly recommended to also indicate the fibers and the % of fruits and vegetables so that this information can be taken into account in the calculation of Nutri-Score:</p>

  <ul>
    <!-- Do not translate nutrient field names -->
  <li>fiber_100g: fibers</li>
  <li>fruits-vegetables-nuts_100g</li>
    <!-- Do not translate nutrient field names -->
  </ul>
  
  <h5 id="champs_optionnels">可填項目</h5>

  <ul>
    <li>labels: labels, in the main language of the product (several possible values, to be separated by commas) (example: AB Bio, MSC, Max Havelaar). Note: the labels must appear on the packaging of the product.</li>
    <li>origins: origins of the ingredients, in the main language of the product (several possible values, to be separated by commas) (example: Peru, Périgord)</li>
    <li>packaging: materials and forms of the product packaging, in the main language of the product (several possible values, to be separated by commas) (example: can, aluminum)</li>
    <li>manufacturing_places: manufacturing locations, in the main language of the product (several possible values, to be separated by commas: city names, countries)</li>
    <li>emb_codes: packager codes and European sanitary stamps (several possible values, to be separated by commas)</li>
    <li>stores: places of sale (several possible values, to be separated by commas: names of supermarket chains)</li>
    <li>nutri_score: Nutri-Score rating from A to E, only in case it is indicated on the package</li>
    <li>serving_size: quantity of a serving, specify the unit</li>
  </ul>

<p>Nutritional values can also be indicated for the prepared product (with water, milk etc.). Then add "_prepared" to the name of the field. For example energy_prepared_100g.</p>
<p>It is also possible to indicate the values for other nutrients:</p>

<ul>
  <!-- Do not translate nutrient field names -->
  
<li>energy-from-fat_100g</li>
<li>butyric-acid_100g</li>
<li>caproic-acid_100g</li>
<li>caprylic-acid_100g</li>
<li>capric-acid_100g</li>
<li>lauric-acid_100g</li>
<li>myristic-acid_100g</li>
<li>palmitic-acid_100g</li>
<li>stearic-acid_100g</li>
{'fraction_non_alphanumeric': 0.10221852048156281, 'fraction_numerical': 0.011963352767471794, 'mean_word_length': 5.1876317638791285, 'pattern_counts': {'":': 0, '<': 691, '<?xml version=': 0, '>': 691, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6062976', 'n_tokens_mistral': 8539, 'n_tokens_neox': 8091, 'n_words': 3244}