<commit_before># Object Relation Mapper for ETSource

The object relation mapper for ETSource is designed to:

1. Provide a common way to (mass) manipulate objects in ETSource
2. Run validations
3. Easier import of ETSource

## 1 Common way to (mass) manipulate update

e.g.

```Ruby
Gquery.all.each do |gquery|
  gquery.unit = "percentage" if gquery.unit == "%"
  gquery.save!
end
```

And hurray! You have changed all those gqueries.

## 2 Run validations

When you are updating/manipulating an object, you can (or must) run validations.

When an object is invalid, you can ask the object which 'parts' are invalid.

```Ruby
gquery = ETSource::Gquery.find(:co2_emissions_total)
gquery.unit = nil

gquery.save!
=> false

gquery.errors.message
=> {:unit=>["can't be blank"]}
```

Validation will be run at three levels:

1. We can have the Continuous Integration server checking whether all objects
are valid, thus to prevent malicious data.
2. You will not be able to save the File through the ORM when the object is not
valid.
3. ETengine import will not proceed when an object is invalid.<commit_msg>updating documentation on ORM<commit_after># Object Relation Mapper for ETSource

The object relation mapper for ETSource is designed to:

1. Provide a common way to (mass) manipulate objects in ETSource
2. Run validations
3. Easier import of ETSource

## Easy scripts to (mass) manipulate objects

E.g. when you would like to update all the percentage units for gqueries:

```Ruby
Gquery.all.each do |gquery|
  gquery.unit = "percentage" if gquery.unit == "%"
  gquery.save!
end
```

And hurray! You have changed all those gqueries.

## Run validations

When you are updating/manipulating an object, you can (or must) run validations.

When an object is invalid, you can ask the object which 'parts' are invalid.

```Ruby
gquery = ETSource::Gquery.find(:co2_emissions_total)
gquery.unit = nil

gquery.save!
=> false

gquery.errors.message
=> {:unit=>["can't be blank"]}
```

Validation will be run at three levels:

1. We can have the Continuous Integration server checking whether all objects
   are valid, thus to prevent malicious data.
2. You will not be able to save the File through the ORM when the object is not
   valid.
3. ETengine import of ETSource date will only proceed when all objects are
   valid.

## Easier import for data from ETSource

With the ORM it will be easier to import all the objects needed for instance
for ETEngine.
{'fraction_non_alphanumeric': 0.07670338637290902, 'fraction_numerical': 0.006527947776417789, 'mean_word_length': 4.205944798301486, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 9, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4830314', 'n_tokens_mistral': 757, 'n_tokens_neox': 736, 'n_words': 350}