<commit_before>## API Documentation

### General Request Details

#### Maximum URL Length

The Maximum URL Length supported by CMR is indirectly controlled by the Request Header Size setting in Jetty which is set to 1MB. This translates to roughly 500k characters. Clients using the Search API with query parameters should be careful not to exceed this limit or they will get an HTTP response of 413 FULL HEAD. If a client expects that the query url could be extra long so that it exceeds 500k characters, they should use the POST API for searching.

#### CORS Header support

The CORS headers are supported on search endpoints. Check [CORS Documentation](https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS) for an explanation of CORS headers.

#### Query Parameters

 * `page_size` - number of results per page - default is 10, max is 2000
 * `page_num` - The page number to return
 * `sort_key` - Indicates one or more fields to sort on. Described below
 * `pretty` - return formatted results if set to true
 * `token` - specifies a user/guest token from ECHO to use to authenticate yourself. This can also be specified as the header Echo-Token
 * `echo_compatible` - When set to true results will be returned in an ECHO compatible format. This mostly removes fields and features specific to the CMR such as revision id, granule counts and facets in collection results. Metadata format style results will also use ECHO style names for concept ids such as `echo_granule_id` and `echo_dataset_id`.

#### Parameter Options

The behavior of search with respect to each parameter can be modified using the `options` parameter. The `options` parameter takes the following form:

  `options[parameter][option_key]=value`

where parameter is the URL parameter whose behavior is to be affected, value is either `true` or `false`, and `option_key` is one of the following:

 * `ignore_case` - if set to true, the search will be case insensitive and if set to false, the search will be case sensitive. Defaults to true.
 * `pattern` - if set to true, the search will treat the value provided for the parameter as a pattern with wild-cards, in which '*' matches zero or more characters and '?' matches any single character. For example, `platform[]=AB?D*&options[platform][pattern]=true` would match 'ABAD123', 'ABCD12', 'ABeD', etc. Defaults to false.
 * `and` - if set to true and if multiple values are listed for the param, the concepts must have ALL of these values in order to match. The default is `false` which means concepts with ANY of the values match. This option only applies to fields which may be multivalued; these are documented here.
 * `or` - this option only applies to granule attributes or science-keywords searches. If set to true, attribute searches will find granules that match any of the attributes. The default is false.

##### Collection Query Parameters

These are query parameters specific to collections

  * `include_has_granules` - If this parameter is set to "true" this will include a flag indicating true or false if the collection has any granules at all. Supported in all response formats except opendata.
  * `include_granule_counts` - If this parameter is set to "true" this will include a count of the granules in each collection that would match the spatial and temporal conditions from the collection query. Supported in all response formats except opendata.
  * `include_facets` - If this parameter is set to "true" facets will be included in the collection results (not applicable to opendata results). Facets are described in detail below.
  * `hierarchical_facets` - If this parameter is set to "true" and the parameter `include_facets` is set to "true" the facets that are returned will be hierarchical. Hierarchical facets are described in the facets section below.

#### Headers

  * Accept - specifies the MimeType to return search results in. Default is "application/xml".
    * `curl -H "Accept: application/xml" -i "%CMR-ENDPOINT%/collections"`
  * `Echo-Token` - specifies an ECHO token to use to authenticate yourself.
  * `Client-Id` - Indicates a name for the client using the CMR API. Specifying this helps Operations monitor query performance per client. It can also make it easier for them to identify your requests if you contact them for assistance.

  * The response headers include CMR-Hits and CMR-Took which indicate the number of result hits
     and the time to build and execute the query, respectively.

#### Extensions

Besides MimeTypes, client can also use extensions to specify the format for search results. Default is xml.

  * `curl -i "%CMR-ENDPOINT%/collections"`
  * `curl -i "%CMR-ENDPOINT%/collections.json"`
  * `curl -i "%CMR-ENDPOINT%/collections.echo10"`
  * `curl -i "%CMR-ENDPOINT%/collections.iso_mends"`

Here is a list of supported extensions and their corresponding MimeTypes:

  * `json`      "application/json"
  * `xml`       "application/xml"
  * `echo10`    "application/echo10+xml"
  * `iso`       "application/iso19115+xml"
  * `iso19115`  "application/iso19115+xml"
  * `dif`       "application/dif+xml"
  * `dif10`     "application/dif10+xml"
  * `csv`       "text/csv"
  * `atom`      "application/atom+xml"
  * `opendata`  "application/opendata+json" (only supported for collections)
  * `kml`       "application/vnd.google-earth.kml+xml"
  * `native`    "application/metadata+xml" (Returns search results in their individual native formats)

### Supported Result Formats

#### Atom

See the [Atom specification](http://tools.ietf.org/html/rfc4287) for a full description of Atom.

The CMR Atom format provides search results in an XML file representing a feed of entries. The feed has the following fields:

|                         Atom Feed Level Feeds                         ||
|   Field    |                        Description                        |
| ---------- | --------------------------------------------------------- |
| id         | the URL linking to this feed                               |
| title      | Either 'ECHO dataset metadata' or 'ECHO granule metadata' |
| updated    | The date/time the search was executed                     |

 Each entry represents one search result, consisting of the following Atom standard fields and additional CMR specific fields:


|                                                           Atom Standard Fields                                                          ||
|            Field            |                                                Description                                                 |
| --------------------------- | ---------------------------------------------------------------------------------------------------------- |
| id                          | the CMR identifier for the result                                                                          |
| title                       | the UMM Entry Title                                                                                        |
| summary  (collections only) | the summary of intentions with which this collection was developed. - corresponds to the UMM summary field |
| updated                     | date/time of the last update to the assocated metadata                                                     |

The following fields are specific to the CMR output and correspond to ECHO10 fields of the same name:

|                                                                      CMR Specific Fields                                                                      ||
|                  Field                  |                                                     Description                                                      |
| --------------------------------------- | -------------------------------------------------------------------------------------------------------------------- |
| echo:datasetId                          | description of the dataset                                                                                           |
| echo:shortName (collections only)       | UMM Entry Title                                                                                                      |
| echo:versionId  (collections only)      | provider defined version id of the metadata                                                                          |
| echo:originalFormat                     | original metadata format                                                                                             |
| echo:dataCenter                         | datacenter providing the metadata                                                                                    |
| echo:orbitParameters (collections only) | fields releated to the satellite orbit (startCircularLatitude, numberOfOrbits, inclinationAngle, period, swathWidth) |
| echo:onlineAccessFlag                   | true if the data is available online                                                                                 |
| echo:browseFlag                         | true if the data contains browse imagery                                                                             |

__Example__

```xml
<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns:os="http://a9.com/-/spec/opensearch/1.1/"
      xmlns:georss="http://www.georss.org/georss/10"
      xmlns="http://www.w3.org/2005/Atom"
      xmlns:dc="http://purl.org/dc/terms/"
      xmlns:echo="http://www.echo.nasa.gov/esip"
      xmlns:esipdiscovery="http://commons.esipfed.org/ns/discovery/1.2/"
      xmlns:gml="http://www.opengis.net/gml"
      xmlns:time="http://a9.com/-/opensearch/extensions/time/1.0/"
      esipdiscovery:version="1.2">
   <updated>2015-06-05T20:10:39.166Z</updated>
   <id>%CMR-ENDPOINT%/collections.atom?pretty=true</id>
   <title type="text">ECHO dataset metadata</title>
   <entry>
      <id>C1200000000-PROV1</id>
      <title type="text">dataset-id</title>
      <summary type="text">The AMSR-E/Aqua Level-3 5-day snow water equivalent (SWE) product includes global 5-day maximum SWE on Northern and Southern Hemisphere 25 km EASE-Grids, generated by the GSFC algorithm using Level-2A TBs.</summary>
      <updated>2010-10-06T11:45:39.530Z</updated>
      <echo:datasetId>dataset-id</echo:datasetId>
      <echo:shortName>short</echo:shortName>
      <echo:versionId>v1</echo:versionId>
      <echo:originalFormat>ECHO10</echo:originalFormat>
      <echo:dataCenter>PROV1</echo:dataCenter>
      <echo:orbitParameters/>
      <echo:onlineAccessFlag>false</echo:onlineAccessFlag>
      <echo:browseFlag>false</echo:browseFlag>
   </entry>
</feed>
```

#### CSV

The comma separated value (CSV) format is only supported for granules.

__Example__

```csv
Granule UR,Producer Granule ID,Start Time,End Time,Online Access URLs,Browse URLs,Cloud Cover,Day/Night,Size
SC:SPL1AA.001:12345,SMAP_L1C_S0_HIRES_00016_A_20150530T160100_R03001_001.h5,,,,,,,
```

#### XML Formats (DIF, DIF 10, ECHO 10, ISO-SMAP, ISO-MENDS)

All of the XML formats (except the XML used in returning references only) have the same structure, differing only in the way each result is returned. These formats return a single XML document with a `<results>` tag containing the following fields as sub-tags:

|         Field         |                    Description                     |
| --------------------- | -------------------------------------------------- |
| hits                  | the number of results matching the search query    |
| took                  | time in milliseconds it took to perform the search |
| result (zero or more) | a single search result - documented below          |
<commit_msg>Making small fix to search api documentation.<commit_after>## API Documentation

### General Request Details

#### Maximum URL Length

The Maximum URL Length supported by CMR is indirectly controlled by the Request Header Size setting in Jetty which is set to 1MB. This translates to roughly 500k characters. Clients using the Search API with query parameters should be careful not to exceed this limit or they will get an HTTP response of 413 FULL HEAD. If a client expects that the query url could be extra long so that it exceeds 500k characters, they should use the POST API for searching.

#### CORS Header support

The CORS headers are supported on search endpoints. Check [CORS Documentation](https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS) for an explanation of CORS headers.

#### Query Parameters

 * `page_size` - number of results per page - default is 10, max is 2000
 * `page_num` - The page number to return
 * `sort_key` - Indicates one or more fields to sort on. Described below
 * `pretty` - return formatted results if set to true
 * `token` - specifies a user/guest token from ECHO to use to authenticate yourself. This can also be specified as the header Echo-Token
 * `echo_compatible` - When set to true results will be returned in an ECHO compatible format. This mostly removes fields and features specific to the CMR such as revision id, granule counts and facets in collection results. Metadata format style results will also use ECHO style names for concept ids such as `echo_granule_id` and `echo_dataset_id`.

#### Parameter Options

The behavior of search with respect to each parameter can be modified using the `options` parameter. The `options` parameter takes the following form:

  `options[parameter][option_key]=value`

where parameter is the URL parameter whose behavior is to be affected, value is either `true` or `false`, and `option_key` is one of the following:

 * `ignore_case` - if set to true, the search will be case insensitive and if set to false, the search will be case sensitive. Defaults to true.
 * `pattern` - if set to true, the search will treat the value provided for the parameter as a pattern with wild-cards, in which '*' matches zero or more characters and '?' matches any single character. For example, `platform[]=AB?D*&options[platform][pattern]=true` would match 'ABAD123', 'ABCD12', 'ABeD', etc. Defaults to false.
 * `and` - if set to true and if multiple values are listed for the param, the concepts must have ALL of these values in order to match. The default is `false` which means concepts with ANY of the values match. This option only applies to fields which may be multivalued; these are documented here.
 * `or` - this option only applies to granule attributes or science-keywords searches. If set to true, attribute searches will find granules that match any of the attributes. The default is false.

##### Collection Query Parameters

These are query parameters specific to collections

  * `include_has_granules` - If this parameter is set to "true" this will include a flag indicating true or false if the collection has any granules at all. Supported in all response formats except opendata.
  * `include_granule_counts` - If this parameter is set to "true" this will include a count of the granules in each collection that would match the spatial and temporal conditions from the collection query. Supported in all response formats except opendata.
  * `include_facets` - If this parameter is set to "true" facets will be included in the collection results (not applicable to opendata results). Facets are described in detail below.
  * `hierarchical_facets` - If this parameter is set to "true" and the parameter `include_facets` is set to "true" the facets that are returned will be hierarchical. Hierarchical facets are described in the facets section below.

#### Headers

  * Accept - specifies the MimeType to return search results in. Default is "application/xml".
    * `curl -H "Accept: application/xml" -i "%CMR-ENDPOINT%/collections"`
  * `Echo-Token` - specifies an ECHO token to use to authenticate yourself.
  * `Client-Id` - Indicates a name for the client using the CMR API. Specifying this helps Operations monitor query performance per client. It can also make it easier for them to identify your requests if you contact them for assistance.

  * The response headers include CMR-Hits and CMR-Took which indicate the number of result hits
     and the time to build and execute the query, respectively.

#### Extensions

Besides MimeTypes, client can also use extensions to specify the format for search results. Default is xml.

  * `curl -i "%CMR-ENDPOINT%/collections"`
  * `curl -i "%CMR-ENDPOINT%/collections.json"`
  * `curl -i "%CMR-ENDPOINT%/collections.echo10"`
  * `curl -i "%CMR-ENDPOINT%/collections.iso_mends"`

Here is a list of supported extensions and their corresponding MimeTypes:

  * `json`      "application/json"
  * `xml`       "application/xml"
  * `echo10`    "application/echo10+xml"
  * `iso`       "application/iso19115+xml"
  * `iso19115`  "application/iso19115+xml"
  * `dif`       "application/dif+xml"
  * `dif10`     "application/dif10+xml"
  * `csv`       "text/csv"
  * `atom`      "application/atom+xml"
  * `opendata`  "application/opendata+json" (only supported for collections)
  * `kml`       "application/vnd.google-earth.kml+xml"
  * `native`    "application/metadata+xml" (Returns search results in their individual native formats)

### Supported Result Formats

#### Atom

See the [Atom specification](http://tools.ietf.org/html/rfc4287) for a full description of Atom.

The CMR Atom format provides search results in an XML file representing a feed of entries. The feed has the following fields:

|                         Atom Feed Level Feeds                         ||
|   Field    |                        Description                        |
| ---------- | --------------------------------------------------------- |
| id         | the URL linking to this feed                               |
| title      | Either 'ECHO dataset metadata' or 'ECHO granule metadata' |
| updated    | The date/time the search was executed                     |

 Each entry represents one search result, consisting of the following Atom standard fields and additional CMR specific fields:


|                                                           Atom Standard Fields                                                          ||
|            Field            |                                                Description                                                 |
| --------------------------- | ---------------------------------------------------------------------------------------------------------- |
| id                          | the CMR identifier for the result                                                                          |
| title                       | the UMM Entry Title                                                                                        |
| summary  (collections only) | the summary of intentions with which this collection was developed. - corresponds to the UMM summary field |
| updated                     | date/time of the last update to the assocated metadata                                                     |

The following fields are specific to the CMR output and correspond to ECHO10 fields of the same name:

|                                                                      CMR Specific Fields                                                                      ||
|                  Field                  |                                                     Description                                                      |
| --------------------------------------- | -------------------------------------------------------------------------------------------------------------------- |
| echo:datasetId                          | description of the dataset                                                                                           |
| echo:shortName (collections only)       | UMM Entry Title                                                                                                      |
| echo:versionId  (collections only)      | provider defined version id of the metadata                                                                          |
| echo:originalFormat                     | original metadata format                                                                                             |
| echo:dataCenter                         | datacenter providing the metadata                                                                                    |
| echo:orbitParameters (collections only) | fields releated to the satellite orbit (startCircularLatitude, numberOfOrbits, inclinationAngle, period, swathWidth) |
| echo:onlineAccessFlag                   | true if the data is available online                                                                                 |
| echo:browseFlag                         | true if the data contains browse imagery                                                                             |

__Example__

```xml
<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns:os="http://a9.com/-/spec/opensearch/1.1/"
      xmlns:georss="http://www.georss.org/georss/10"
      xmlns="http://www.w3.org/2005/Atom"
      xmlns:dc="http://purl.org/dc/terms/"
      xmlns:echo="http://www.echo.nasa.gov/esip"
      xmlns:esipdiscovery="http://commons.esipfed.org/ns/discovery/1.2/"
      xmlns:gml="http://www.opengis.net/gml"
      xmlns:time="http://a9.com/-/opensearch/extensions/time/1.0/"
      esipdiscovery:version="1.2">
   <updated>2015-06-05T20:10:39.166Z</updated>
   <id>%CMR-ENDPOINT%/collections.atom?pretty=true</id>
   <title type="text">ECHO dataset metadata</title>
   <entry>
      <id>C1200000000-PROV1</id>
      <title type="text">dataset-id</title>
      <summary type="text">The AMSR-E/Aqua Level-3 5-day snow water equivalent (SWE) product includes global 5-day maximum SWE on Northern and Southern Hemisphere 25 km EASE-Grids, generated by the GSFC algorithm using Level-2A TBs.</summary>
      <updated>2010-10-06T11:45:39.530Z</updated>
      <echo:datasetId>dataset-id</echo:datasetId>
      <echo:shortName>short</echo:shortName>
      <echo:versionId>v1</echo:versionId>
      <echo:originalFormat>ECHO10</echo:originalFormat>
      <echo:dataCenter>PROV1</echo:dataCenter>
      <echo:orbitParameters/>
      <echo:onlineAccessFlag>false</echo:onlineAccessFlag>
      <echo:browseFlag>false</echo:browseFlag>
   </entry>
</feed>
```

#### CSV

The comma separated value (CSV) format is only supported for granules.

__Example__

```csv
Granule UR,Producer Granule ID,Start Time,End Time,Online Access URLs,Browse URLs,Cloud Cover,Day/Night,Size
SC:SPL1AA.001:12345,SMAP_L1C_S0_HIRES_00016_A_20150530T160100_R03001_001.h5,,,,,,,
```

#### Metadata Responses (DIF, DIF 10, ECHO 10, ISO-SMAP, ISO-MENDS)

All of the XML Metadata formats (except the XML used in returning references only) have the same structure, differing only in the way each result is returned. These formats return a single XML document with a `<results>` tag containing the following fields as sub-tags:

|         Field         |                    Description                     |
| --------------------- | -------------------------------------------------- |
| hits                  | the number of results matching the search query    |
| took                  | time in milliseconds it took to perform the search |
| result (zero or more) | a single search result - documented below          |

{'fraction_non_alphanumeric': 0.10987179487179487, 'fraction_numerical': 0.01452991452991453, 'mean_word_length': 1.910934195795497, 'pattern_counts': {'":': 0, '<': 73, '<?xml version=': 2, '>': 73, 'https://': 2, 'lorem ipsum': 0, 'www.': 8, 'xml': 46}, 'pii_count': 0, 'substrings_counts': 2, 'word_list_counts': {'cursed_substrings.json': 107, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4588930', 'n_tokens_mistral': 5840, 'n_tokens_neox': 5387, 'n_words': 2251}