<commit_before>## Source

Data sources are the primary feature of the National Data Catalog. They are representations of data sets and APIs. See sources.md for more information.

## Schema

    Field              R  W  Type       Description
    -----              -  -  ----       -----------
    id                 R     String     Unique ID (auto-generated upon creation)
    title                 W  String     Unique name of data source
    slug                     String     URL slug (auto-generated if not provided)
    source_type           W  String     Either "dataset" or "api"
    url                   W  String     Primary URL for this data source (should have metadata) 
    documentation_url        String     Official documentation for data source 
    license                  String     License for the data (e.g. public domain)
    license_url              String     URL describing license
    catalog_name             String     Name of originating catalog (e.g. 'data.gov')
    catalog_url              String     URL of originating catalog   
    released                 Date Hash  When this source was publicly released
    period_start             Date Hash  Beginning date for source's applicability
    period_end               Date Hash  End date for source's applicability
    frequency                String     How often the data source is released
    organization_id          String     Organization that published this source.
    ratings            R     Array      Embedded collection of Rating objects
    rating_stats       R     Hash       Contains total, count, and average.
    created_at         R     Time       Automatic timestamp on creation
    updated_at         R     Time       Automatic timestamp on update

*Notes*:

* R denotes read-only fields
* W denotes required fields when writing (creating or updating)
* "Date Hash" is a Ruby hash intended for use with the Kronos gem.

## Permissions

	Permission   Who
	----------   ---
	List         (add information here)
	Create       (add information here)<commit_msg>add frequency documentation<commit_after>## Source

Data sources are the primary feature of the National Data Catalog. They are representations of data sets and APIs. See sources.md for more information.

## Schema

    Field              R  W  Type       Description
    -----              -  -  ----       -----------
    id                 R     String     Unique ID (auto-generated upon creation)
    title                 W  String     Unique name of data source
    slug                     String     URL slug (auto-generated if not provided)
    source_type           W  String     Either "dataset" or "api"
    url                   W  String     Primary URL for this data source (should have metadata) 
    documentation_url        String     Official documentation for data source 
    license                  String     License for the data (e.g. public domain)
    license_url              String     URL describing license
    catalog_name             String     Name of originating catalog (e.g. 'data.gov')
    catalog_url              String     URL of originating catalog   
    released                 Date Hash  When this source was publicly released
    period_start             Date Hash  Beginning date for source's applicability
    period_end               Date Hash  End date for source's applicability
    frequency                String     How often the data source is released
    organization_id          String     Organization that published this source.
    ratings            R     Array      Embedded collection of Rating objects
    rating_stats       R     Hash       Contains total, count, and average.
    created_at         R     Time       Automatic timestamp on creation
    updated_at         R     Time       Automatic timestamp on update

*Notes*:

* R denotes read-only fields
* W denotes required fields when writing (creating or updating)
* "Date Hash" is a Ruby hash intended for use with the Kronos gem.

### frequency

Valid values for frequency include:

    each (second / minute / hour / day / week / month / quarter / year)
    hourly / daily / weekly / monthly / quarterly / annually / yearly
    one-time / other / unknown

A complete, up-to-date list can be found in [frequency.rb in the frequency gem](http://github.com/djsun/frequency/blob/master/lib/frequency.rb).

## Permissions

	Permission   Who
	----------   ---
	List         (add information here)
	Create       (add information here)
{'fraction_non_alphanumeric': 0.050397877984084884, 'fraction_numerical': 0.0, 'mean_word_length': 1.727546714888487, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7007056', 'n_tokens_mistral': 1037, 'n_tokens_neox': 992, 'n_words': 477}