<commit_before># iDAI.field Web

## Getting started

Prepare the configuration:

    $ cp api/config/dev_prod.exs.template api/config/dev.exs
    $ cp api/config/dev_prod.exs.template api/config/prod.exs
    $ vi api/config/dev.exs # Edit configuration
    $ vi api/config/prod.exs # Edit configuration

Start elasticsearch and the api web service with docker:

    $ docker-compose up

Trigger indexing:

    $ curl -XPOST localhost:4000/api/reindex

Start the web UI:

    $ cd ui
    $ npm i
    $ npm start

Visit `http://localhost:3001`.

## Test Data

To set up test data, and provide them via elasticsearch, do:


    $ docker-compose up elasticsearch
    $ ./put-test-data.sh

Test via `http://localhost:9200/idai-field/_search`.

## Images

In order to be able to see images you can override the images volume by creating
a `docker-compose.override.yml` that contains the the volume definition. This
file gets automatically picked up by `docker-compose` and will not be published
to the repository.

Example:

    version: "3.7"

    services:

    api:
        volumes:
        - "/Volumes/bcloud03/idaiworld-scans/idaifield:/opt/src/api/images"


## API

The api runs on port 4000. Indexed documents can be retrieved via `http://localhost:4000/documents`.

It can be run independently with the following command:

    $ docker-compose up api

or interactively with iex:

    $ docker-compose run --service-ports --entrypoint "iex -S mix" api


### Managing dependencies

In order to add a dependency it has to be added to `mix.exs`. Afterwards, the api docker container
has to be rebuilt explicitly with:

    $ docker-compose build api

After removing a dependency from `mix.exs` the following command has to be run inside api/ to make
sure `mix.lock` reflects the change:

    $ mix deps.clean --unused --unlock

Afterwards, the api docker container has to be rebuilt explicitly with:

    $ docker-compose build api


## UI

The frontend runs on port 3001. It autmatically picks the next available port if 3001 is already in use.

Start the development server with:

    $ cd ui
    $ npm start

To install dependencies use:

    $ npm i

To build for production use:

    $ npm run build


## Elasticsearch

The elasticsearch REST API runs on port 9200. Indexed documents can be retrieved via
`http://localhost:9200/idai-field/_search`.

It can be started independently with:

    $ docker-compose up elasticsearch

## Testing

    $ docker-compose run --service-ports --entrypoint "mix test test/app && mix test --no-start test/unit" api

or
 
    $ docker-compose run --entrypoint "/bin/bash" api
    $ mix test test/app                 # application/subsystem tests
    $ mix test --no-start test/unit     # unit tests

## (Re-)building containers

    $ docker-compose up --build api
<commit_msg>Adjust readme example images path<commit_after># iDAI.field Web

## Getting started

Prepare the configuration:

    $ cp api/config/dev_prod.exs.template api/config/dev.exs
    $ cp api/config/dev_prod.exs.template api/config/prod.exs
    $ vi api/config/dev.exs # Edit configuration
    $ vi api/config/prod.exs # Edit configuration

Start elasticsearch and the api web service with docker:

    $ docker-compose up

Trigger indexing:

    $ curl -XPOST localhost:4000/api/reindex

Start the web UI:

    $ cd ui
    $ npm i
    $ npm start

Visit `http://localhost:3001`.

## Test Data

To set up test data, and provide them via elasticsearch, do:


    $ docker-compose up elasticsearch
    $ ./put-test-data.sh

Test via `http://localhost:9200/idai-field/_search`.

## Images

In order to be able to see images you can override the images volume by creating
a `docker-compose.override.yml` that contains the volume definition. This
file gets automatically picked up by `docker-compose` and will not be published
to the repository.

Example:

    version: "3.7"

    services:

    api:
        volumes:
        - "/host/environment/path/to/images:/opt/src/api/images"


## API

The api runs on port 4000. Indexed documents can be retrieved via `http://localhost:4000/documents`.

It can be run independently with the following command:

    $ docker-compose up api

or interactively with iex:

    $ docker-compose run --service-ports --entrypoint "iex -S mix" api


### Managing dependencies

In order to add a dependency it has to be added to `mix.exs`. Afterwards, the api docker container
has to be rebuilt explicitly with:

    $ docker-compose build api

After removing a dependency from `mix.exs` the following command has to be run inside api/ to make
sure `mix.lock` reflects the change:

    $ mix deps.clean --unused --unlock

Afterwards, the api docker container has to be rebuilt explicitly with:

    $ docker-compose build api


## UI

The frontend runs on port 3001. It autmatically picks the next available port if 3001 is already in use.

Start the development server with:

    $ cd ui
    $ npm start

To install dependencies use:

    $ npm i

To build for production use:

    $ npm run build


## Elasticsearch

The elasticsearch REST API runs on port 9200. Indexed documents can be retrieved via
`http://localhost:9200/idai-field/_search`.

It can be started independently with:

    $ docker-compose up elasticsearch

## Testing

    $ docker-compose run --service-ports --entrypoint "mix test test/app && mix test --no-start test/unit" api

or
 
    $ docker-compose run --entrypoint "/bin/bash" api
    $ mix test test/app                 # application/subsystem tests
    $ mix test --no-start test/unit     # unit tests

## (Re-)building containers

    $ docker-compose up --build api

{'fraction_non_alphanumeric': 0.08376220053238687, 'fraction_numerical': 0.013842058562555456, 'mean_word_length': 3.0314735336194563, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '436356', 'n_tokens_mistral': 1784, 'n_tokens_neox': 1616, 'n_words': 661}