<commit_before># Importers

<!-- vim-markdown-toc GFM -->

* [YSO - General Finnish ontology **(Required)**](#yso---general-finnish-ontology-required)
  * [What is it?](#what-is-it)
  * [What depends on it?](#what-depends-on-it)
  * [How to use it?](#how-to-use-it)
* [tprek - Location registry **(Required)**](#tprek---location-registry-required)
  * [What is it?](#what-is-it-1)
  * [What depends on it?](#what-depends-on-it-1)
  * [How to use it?](#how-to-use-it-1)
* [osoite - Location registry](#osoite---location-registry)
  * [What is it?](#what-is-it-2)
  * [What depends on it?](#what-depends-on-it-2)
  * [How to use it?](#how-to-use-it-2)
* [helmet - Helsinki Metropolitan Area Libraries](#helmet---helsinki-metropolitan-area-libraries)
  * [What is it?](#what-is-it-3)
  * [What depends on it?](#what-depends-on-it-3)
  * [How to use it?](#how-to-use-it-3)
* [espoo - Espoo city](#espoo---espoo-city)
  * [What is it?](#what-is-it-4)
  * [What depends on it?](#what-depends-on-it-4)
  * [How to use it?](#how-to-use-it-4)
* [add_helsinki_audience](#add_helsinki_audience)
  * [What is it?](#what-is-it-5)
  * [What depends on it?](#what-depends-on-it-5)
  * [How to use it?](#how-to-use-it-5)
* [add_helfi_topics](#add_helfi_topics)
  * [What is it?](#what-is-it-6)
  * [What depends on it?](#what-depends-on-it-6)
  * [How to use it?](#how-to-use-it-6)
* [import_organizations](#import_organizations)
  * [What is it?](#what-is-it-7)
  * [What depends on it?](#what-depends-on-it-7)
  * [How to use it?](#how-to-use-it-7)
* [install_templates](#install_templates)
  * [What is it?](#what-is-it-8)
  * [What depends on it?](#what-depends-on-it-8)
  * [How to use it?](#how-to-use-it-8)
* [geo_import](#geo_import)
  * [What is it?](#what-is-it-9)
  * [What depends on it?](#what-depends-on-it-9)
  * [How to use it?](#how-to-use-it-9)

<!-- vim-markdown-toc -->

## YSO - General Finnish ontology **(Required)**

### What is it?
First, some terminology is in order. Starting with *ontology*:

> In computer science and information science, an ontology encompasses a representation,
> formal naming and definition of the categories, properties and relations between the
> concepts, data and entities that substantiate one, many or all domains of discourse.
>
> From [Wikipedia of Ontology (information science)](https://en.wikipedia.org/wiki/Ontology_(information_science))

YSO, short for *Yleinen suomalainen ontologia*, is basically a +30,000 collection of **concepts** in Finnish culture, translated in 3 languages.

> General Finnish Ontology YSO is a trilingual ontology consisting mainly of general concepts.
> YSO has been founded on the basis of concepts in Finnish cultural sphere.
>
> From https://finto.fi/yso/en/. You can browse the concepts there if you wish

Basic example would be, let's say a guitar player coming to Helsinki to perform in a concert and use sauna afterwards.

We might categorize it with "concert", "popular music", and "sauna" *concepts*.

These *concepts* come from YSO.

### What depends on it?
Linked Events application has *events*. Events must be described by *concepts*.
Therefore, all events depend on YSO.

[add_helsinki_audience](#what-depends-on-it-5) also depends on this importer.

[add_helfi_topics](#what-depends-on-it-6) also depends on this importer.

### How to use it?
  ```bash
  python manage.py event_import yso --keywords --all
  ```

Imports all keywords in YSO into the database, without maintaining the existing hierarchy between the concepts.

## tprek - Location registry **(Required)**

### What is it?

*tprek*, short for *Toimipisterekisteri*, is a location registry service giving access to addresses in Helsinki through a REST API.

Basic example would be the address of a concert in a certain theater hall.

### What depends on it?
Linked Events application has *events*. Events must have addresses.

By default, all event addresses are from *tprek* location registry.

Therefore, all events depend on *tprek* addresses to be present.

### How to use it?
  ```bash
  python manage.py event_import tprek --places
  ```

Imports all addresses in *tprek* location registry into the database.

## osoite - Location registry

### What is it?

*osoite*, short for *Pääkaupunkiseudun osoiteluettelo*, is a Metropolitan address list importer.

Basic example would be the address of a concert in a very new building whose address is not available in *tprek*.

### What depends on it?
If you want to give the users of your application the ability to add events with addresses which are not present in *tprek*, you need *osoite*.

In other words, *osoite* is fall-back for *tprek*.

### How to use it?
  ```bash
  python manage.py event_import osoite --places
  ```

Imports all addresses in *osoite* location registry into the database.

## helmet - Helsinki Metropolitan Area Libraries

### What is it?

*helmet* is an importer for Helsinki Metropolitan Area Libraries.

Basic example would be an event organized by Helsinki city center library for children, which you can import with this importer.
<commit_msg>upate the terminology in importers.md to refer to locations as locatiosn not addresses as location is the object which encompasses address among other things<commit_after># Importers

<!-- vim-markdown-toc GFM -->

* [YSO - General Finnish ontology **(Required)**](#yso---general-finnish-ontology-required)
  * [What is it?](#what-is-it)
  * [What depends on it?](#what-depends-on-it)
  * [How to use it?](#how-to-use-it)
* [tprek - Location registry **(Required)**](#tprek---location-registry-required)
  * [What is it?](#what-is-it-1)
  * [What depends on it?](#what-depends-on-it-1)
  * [How to use it?](#how-to-use-it-1)
* [osoite - Location registry](#osoite---location-registry)
  * [What is it?](#what-is-it-2)
  * [What depends on it?](#what-depends-on-it-2)
  * [How to use it?](#how-to-use-it-2)
* [helmet - Helsinki Metropolitan Area Libraries](#helmet---helsinki-metropolitan-area-libraries)
  * [What is it?](#what-is-it-3)
  * [What depends on it?](#what-depends-on-it-3)
  * [How to use it?](#how-to-use-it-3)
* [espoo - Espoo city](#espoo---espoo-city)
  * [What is it?](#what-is-it-4)
  * [What depends on it?](#what-depends-on-it-4)
  * [How to use it?](#how-to-use-it-4)
* [add_helsinki_audience](#add_helsinki_audience)
  * [What is it?](#what-is-it-5)
  * [What depends on it?](#what-depends-on-it-5)
  * [How to use it?](#how-to-use-it-5)
* [add_helfi_topics](#add_helfi_topics)
  * [What is it?](#what-is-it-6)
  * [What depends on it?](#what-depends-on-it-6)
  * [How to use it?](#how-to-use-it-6)
* [import_organizations](#import_organizations)
  * [What is it?](#what-is-it-7)
  * [What depends on it?](#what-depends-on-it-7)
  * [How to use it?](#how-to-use-it-7)
* [install_templates](#install_templates)
  * [What is it?](#what-is-it-8)
  * [What depends on it?](#what-depends-on-it-8)
  * [How to use it?](#how-to-use-it-8)
* [geo_import](#geo_import)
  * [What is it?](#what-is-it-9)
  * [What depends on it?](#what-depends-on-it-9)
  * [How to use it?](#how-to-use-it-9)

<!-- vim-markdown-toc -->

## YSO - General Finnish ontology **(Required)**

### What is it?
First, some terminology is in order. Starting with *ontology*:

> In computer science and information science, an ontology encompasses a representation,
> formal naming and definition of the categories, properties and relations between the
> concepts, data and entities that substantiate one, many or all domains of discourse.
>
> From [Wikipedia of Ontology (information science)](https://en.wikipedia.org/wiki/Ontology_(information_science))

YSO, short for *Yleinen suomalainen ontologia*, is basically a +30,000 collection of **concepts** in Finnish culture, translated in 3 languages.

> General Finnish Ontology YSO is a trilingual ontology consisting mainly of general concepts.
> YSO has been founded on the basis of concepts in Finnish cultural sphere.
>
> From https://finto.fi/yso/en/. You can browse the concepts there if you wish

Basic example would be, let's say a guitar player coming to Helsinki to perform in a concert and use sauna afterwards.

We might categorize it with "concert", "popular music", and "sauna" *concepts*.

These *concepts* come from YSO.

### What depends on it?
Linked Events application has *events*. Events must be described by *concepts*.
Therefore, all events depend on YSO.

[add_helsinki_audience](#what-depends-on-it-5) also depends on this importer.

[add_helfi_topics](#what-depends-on-it-6) also depends on this importer.

### How to use it?
  ```bash
  python manage.py event_import yso --keywords --all
  ```

Imports all keywords in YSO into the database, without maintaining the existing hierarchy between the concepts.

## tprek - Location registry **(Required)**

### What is it?

*tprek*, short for *Toimipisterekisteri*, is a location registry service giving access to locations in Helsinki through a REST API.

Basic example would be the location of a concert in a certain theater hall.

### What depends on it?
Linked Events application has *events*. Events must have locations.

By default, all event locations are from *tprek* location registry.

Therefore, all events depend on *tprek* locations to be present.

### How to use it?
  ```bash
  python manage.py event_import tprek --places
  ```

Imports all locations, which usually have addresses, coordinates and other metadata, from *tprek* location registry into the database.

## osoite - Location registry

### What is it?

*osoite*, short for *Pääkaupunkiseudun osoiteluettelo*, is a Metropolitan location list importer.

Basic example would be the location of a concert in a very new building whose location is not available in *tprek*.

### What depends on it?
If you want to give the users of your application the ability to add events with locations which are not present in *tprek*, you need *osoite*.

In other words, *osoite* is fall-back for *tprek*.

### How to use it?
  ```bash
  python manage.py event_import osoite --places
  ```

Imports all locations from *osoite* location registry into the database.

## helmet - Helsinki Metropolitan Area Libraries

### What is it?

*helmet* is an importer for Helsinki Metropolitan Area Libraries.

Basic example would be an event organized by Helsinki city center library for children, which you can import with this importer.

{'fraction_non_alphanumeric': 0.1337315733693034, 'fraction_numerical': 0.006744387705944696, 'mean_word_length': 4.16674962667994, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 25, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '169713', 'n_tokens_mistral': 3560, 'n_tokens_neox': 3325, 'n_words': 1191}