<commit_before>openLCA Reference Data and Mapping Format
=========================================
The openLCA reference data format defines a simple CSV format for exchanging
reference data like flows, units, categories etc. and mappings to entities of
other formats and databases. In general all CSV files should have the following
properties:

* files should be utf-8 encoded
* columns should be separated by semicolon: ;
* strings should be enclosed in double quotes if it is necessary: "
* the decimal separator of numbers should be a point: .
* the files should not contain column headers


Locations
---------
File:       `locations.csv`
Columns:

0. reference ID (UUID, required)
1. name (string, required)
2. description (string, optional) 
3. code (string, required)
4. latitude (double, required)
5. longitude (double, required)

Categories
----------
File:       `categories.csv`      
Columns:

0. reference ID (UUID; required)
1. name (string; required)
2. description (string; optional)
3. model type of the category (enumeration: "PROJECT", "PRODUCT_SYSTEM", 
   "IMPACT_METHOD", "PROCESS", "FLOW", "FLOW_PROPERTY", "UNIT_GROUP"; required)
4. reference ID of the parent category (UUID, optional)


Units
-----
File:       `units.csv`
Columns:

0. reference ID (UUID; required)
1. name (string; required)
2. description (string; optional)
3. conversion factor (double; required)
4. synonyms (string: list separated by semicolon; optional)
5. reference ID of unit group (UUID, required)


Unit groups
-----------
File:       `unit_groups.csv`
Columns:

0. reference ID (UUID, required)
1. name (string, required)
2. description (string, optional)
3. category ID (UUID, optional)
4. default flow property ID (UUID, optional)
5. reference unit ID (UUID, required)


Flow properties
---------------
File:       `flow_properties.csv`
Columns:

0. reference ID (UUID, required)
1. name (string, required)
2. description (string, optional)
3. category ID (UUID, optional)
4. unit group ID (UUID, required)
5. flow property type (integer enum: 0=economic, 1=physical; required)


Flows
-----
File:       `flows.csv`
Columns:

0. reference ID (UUID, required)
1. name (string, required)
2. description (string, optional)
3. category ID (UUID, optional)
4. flow type (enumeration: 'ELEMENTARY_FLOW', 'PRODUCT_FLOW', 'WASTE_FLOW'; required)
5. CAS number (string, optional)
6. formula (string, optional)
7. reference flow property (UUID, required)


Flow property factors 
---------------------
(relations between flows and flow properties)
File:       `flow_property_factors.csv`

0. flow ID (UUID, required)
1. flow property ID (UUID, required)
2. factor (double, required) (1 if it is the reference flow property)


LCIA methods
------------
File:       `lcia_methods.csv`
Columns:

0. reference ID (UUID, required)
1. name (string, required)
2. description (string, optional)
3. category ID (UUID, optional)


LCIA categories
---------------
File:       `lcia_categories.csv`
Columns:

0. reference ID (UUID, required)
1. name (string, required)
2. description (string, optional)
3. reference unit (string, required)
4. LCIA method ID (UUID, required)


LCIA factors
------------
File:       `lcia_factors.csv`

Columns:

0. reference ID of the LCIA category (UUID, required)
1. reference ID of the flow (UUID, required)
2. reference ID of the flow's property (UUID, required)
3. reference ID of the flow's unit (UUID, required)
4. value of the factor (double, required)


Default mappings
================


Default mappins: Unit import
----------------------------
File:		`unit_import_map.csv`

Columns:

0. external name of the unit
1. openLCA reference ID of the unit (UUID)
2. openLCA name of the unit (string)
3. openLCA reference ID of the flow property (UUID)
4. openLCA name of the flow property


EcoSpold 1
==========
<commit_msg>added nw-sets to reference data definition<commit_after>openLCA Reference Data and Mapping Format
=========================================
The openLCA reference data format defines a simple CSV format for exchanging
reference data like flows, units, categories etc. and mappings to entities of
other formats and databases. In general all CSV files should have the following
properties:

* files should be utf-8 encoded
* columns should be separated by semicolon: ;
* strings should be enclosed in double quotes if it is necessary: "
* the decimal separator of numbers should be a point: .
* the files should not contain column headers


Locations
---------
File:       `locations.csv`
Columns:

0. reference ID (UUID, required)
1. name (string, required)
2. description (string, optional) 
3. code (string, required)
4. latitude (double, required)
5. longitude (double, required)

Categories
----------
File:       `categories.csv`      
Columns:

0. reference ID (UUID; required)
1. name (string; required)
2. description (string; optional)
3. model type of the category (enumeration: "PROJECT", "PRODUCT_SYSTEM", 
   "IMPACT_METHOD", "PROCESS", "FLOW", "FLOW_PROPERTY", "UNIT_GROUP"; required)
4. reference ID of the parent category (UUID, optional)


Units
-----
File:       `units.csv`
Columns:

0. reference ID (UUID; required)
1. name (string; required)
2. description (string; optional)
3. conversion factor (double; required)
4. synonyms (string: list separated by semicolon; optional)
5. reference ID of unit group (UUID, required)


Unit groups
-----------
File:       `unit_groups.csv`
Columns:

0. reference ID (UUID, required)
1. name (string, required)
2. description (string, optional)
3. category ID (UUID, optional)
4. default flow property ID (UUID, optional)
5. reference unit ID (UUID, required)


Flow properties
---------------
File:       `flow_properties.csv`
Columns:

0. reference ID (UUID, required)
1. name (string, required)
2. description (string, optional)
3. category ID (UUID, optional)
4. unit group ID (UUID, required)
5. flow property type (integer enum: 0=economic, 1=physical; required)


Flows
-----
File:       `flows.csv`
Columns:

0. reference ID (UUID, required)
1. name (string, required)
2. description (string, optional)
3. category ID (UUID, optional)
4. flow type (enumeration: 'ELEMENTARY_FLOW', 'PRODUCT_FLOW', 'WASTE_FLOW'; required)
5. CAS number (string, optional)
6. formula (string, optional)
7. reference flow property (UUID, required)


Flow property factors 
---------------------
(relations between flows and flow properties)
File:       `flow_property_factors.csv`

0. flow ID (UUID, required)
1. flow property ID (UUID, required)
2. factor (double, required) (1 if it is the reference flow property)


LCIA methods
------------
File:       `lcia_methods.csv`
Columns:

0. reference ID (UUID, required)
1. name (string, required)
2. description (string, optional)
3. category ID (UUID, optional)


LCIA categories
---------------
File:       `lcia_categories.csv`
Columns:

0. reference ID (UUID, required)
1. name (string, required)
2. description (string, optional)
3. reference unit (string, required)
4. LCIA method ID (UUID, required)


LCIA factors
------------
File:       `lcia_factors.csv`

Columns:

0. reference ID of the LCIA category (UUID, required)
1. reference ID of the flow (UUID, required)
2. reference ID of the flow property of the factor (UUID, required)
3. reference ID of the unit of the factor (UUID, required)
4. value of the factor (double, required)


Normalisation and weighting sets
--------------------------------
File:		`nw_sets.csv`

Columns:

0. reference ID (UUID, required)
1. name (string, required)
2. description (string, optional)
3. weighted score unit (string, optional)
4. LCIA method ID (UUID, required)


Normalisation and weighting set factors
---------------------------------------
File:		`new_set_factors.csv`

Columns:

0. reference ID of the normalisation and weighting set (UUID, required)
1. reference ID of the LCIA category (UUID, required)
2. nomalisation factor (double, optional)
3. weighting factor (double, optional)


Default mappings
================


Default mappins: Unit import
----------------------------
File:		`unit_import_map.csv`

Columns:

0. external name of the unit
1. openLCA reference ID of the unit (UUID)
2. openLCA name of the unit (string)
3. openLCA reference ID of the flow property (UUID)
4. openLCA name of the flow property


EcoSpold 1
==========

{'fraction_non_alphanumeric': 0.14641744548286603, 'fraction_numerical': 0.016415049125329498, 'mean_word_length': 3.492465016146394, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5152246', 'n_tokens_mistral': 2477, 'n_tokens_neox': 2437, 'n_words': 1060}