<commit_before># Event Source Atom Data

[![CircleCI](https://circleci.com/gh/xtracdev/es-atom-data.svg?style=svg)](https://circleci.com/gh/xtracdev/es-atom-data)

This project processes events from the [Oracle Event Store](https://github.com/xtracdev/oraeventstore),
writing them into organized storage to allow navigating published events
using the atom protocol.

This project uses two tables: the atom_event table to store the events
associated with the atom feed, and the feed table, which keeps track of the 
feeds. The overall event history can be traversed from the current feed
backwards using the previous entry in the feeds table. The recent items
are those that have not been assigned a feedid.

As events get written to the recent table, once the size threshold for a feed is read,
they are assigned a feed id. The default page size is 100 items; this may be
overridden using the FEED_THRESHOLD environment variable.

## Table Definitions

<pre>
create table atom_event (
    id number generated always as identity,
    feedid varchar2(100),
    event_time timestamp DEFAULT current_timestamp,
    aggregate_id varchar2(60)not null,
    version integer not null,
    typecode varchar2(30) not null,
    payload blob
);

create table feed (
    id  number generated always as identity,
    event_time timestamp DEFAULT current_timestamp,
    feedid varchar2(100) not null,
    previous varchar2(100)
);
</pre>

## Testing

This package has unit tests that may be run using go test, and integration
tests that may be run using gucumber. Note the integration tests requires
a live instance of Oracle.

*Caution* The integration tests delete all records from the event and 
feed table to run in a know good state.
<commit_msg>README.md updates<commit_after># Event Source Atom Data

[![CircleCI](https://circleci.com/gh/xtracdev/es-atom-data.svg?style=svg)](https://circleci.com/gh/xtracdev/es-atom-data)

This project processes events from the [Oracle Event Store](https://github.com/xtracdev/oraeventstore),
writing them into organized storage to allow navigating published events
using the atom protocol.

This project uses two tables: the atom_event table to store the events
associated with the atom feed, and the feed table, which keeps track of the 
feeds. The overall event history can be traversed from the current feed
backwards using the previous entry in the feeds table. The recent items
are those that have not been assigned a feedid.

As events get written to the recent table, once the size threshold for a feed is read,
they are assigned a feed id. The default page size is 100 items; this may be
overridden using the FEED_THRESHOLD environment variable.

## Table Definitions

<pre>
create table atom_event (
    id number generated always as identity,
    feedid varchar2(100),
    event_time timestamp DEFAULT current_timestamp,
    aggregate_id varchar2(60)not null,
    version integer not null,
    typecode varchar2(30) not null,
    payload blob
);

create index atom_event_feedid_ix on atom_event (feedid);

create table feed (
    id  number generated always as identity,
    event_time timestamp DEFAULT current_timestamp,
    feedid varchar2(100) not null,
    previous varchar2(100)
);

create index feed_feedid_ix on feed(feedid);
create index feed_previous_ix on feed(previous);
</pre>

## Testing

This package has unit tests that may be run using go test, and integration
tests that may be run using gucumber. Note the integration tests requires
a live instance of Oracle.

*Caution* The integration tests delete all records from the event and 
feed table to run in a know good state.

{'fraction_non_alphanumeric': 0.05622932745314223, 'fraction_numerical': 0.011576626240352812, 'mean_word_length': 4.408345752608048, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6899195', 'n_tokens_mistral': 1016, 'n_tokens_neox': 967, 'n_words': 477}