<commit_before>## GBIF-Portal

Java Web Application to exposes the functionality that mostly is produced consuming the GBIF REST services, such
functionality includes:
  * Dataset: faceted search and detail dataset view.
  * Species: faceted search and detail view.
  * Occurrence: filtered search and detail view.
  * Publisher: search and detail view.
  * Network/Node: simple search and detail view.
  * Netwoork: detail view.
  * IPT: display information about releases and statistics.
  * Developer: documentation about the REST API services.
  * Infrastructure: documentation about the occurrence data indexing, processing and tools.
  
This portal was developed using the following technologies: [Strust2](http://struts.apache.org/), [Freemarker](http://freemarker.org/), 
[Less](http://lesscss.org/), [Leaflet](http://leafletjs.com/) and  [JQuery](https://jquery.com/). 

## How to built the project
This project requires a Maven with the following settings defined in the properties section:

```
<profile>
  <id>gbif-portal</id>
  <properties>
        <cargo.context>portal</cargo.context>
        <devMode>false</devMode>
        <notDevMode>true</notDevMode>
        <baseurl>http://www.gbif-dev.org</baseurl>
        <api.baseurl>http://api.gbif-dev.org/v1</api.baseurl>
        <servername>http://www.gbif-dev.org</servername>
        <drupal.url>http://www.gbif-dev.org</drupal.url>
        <url.includeContext>false</url.includeContext>        
        <httpTimeout>60000</httpTimeout>
        <maxHttpConnections>100</maxHttpConnections>
        <maxHttpConnectionsPerRoute>100</maxHttpConnectionsPerRoute>
        <annosys.url>https://annosys.bgbm.fu-berlin.de/AnnoSysTest/</annosys.url> 
        <registry.ws.url>http://api.gbif-dev.org/v1/</registry.ws.url>
        <checklistbank.ws.url>http://api.gbif-dev.org/v1/</checklistbank.ws.url>
        <checklistbank.match.ws.url>http://api.gbif-dev.org/v1/species/match/</checklistbank.match.ws.url>
        <occurrence.ws.url>http://api.gbif-dev.org/v1/</occurrence.ws.url>
        <occurrence.ws.url.public>http://api.gbif-dev.org/v1/</occurrence.ws.url.public>
        <tile-server.url>http://api.gbif-dev.org/v1/map</tile-server.url>
        <image-cache.url>http://api.gbif-dev.org/v1/image</image-cache.url>
        <metrics.ws.url>http://api.gbif-dev.org/v1/</metrics.ws.url>   
        <drupal.cookiename>drupal_cookiename</drupal.cookiename>        
        <drupal.db.host>mysqlserver</drupal.db.host>
        <drupal.db.url>jdbc:mysql://mysqlserver:3306/drupal_db?useUnicode=true&amp;characterEncoding=UTF8&amp;characterSetResults=UTF8</drupal.db.url>
        <drupal.db.name>drupal_db_username</drupal.db.name>
        <drupal.db.username>drupal_db_password</drupal.db.username>
        <drupal.db.poolSize>8</drupal.db.poolSize>
        <drupal.db.connectionTimeout>2000</drupal.db.connectionTimeout>
        <portal.application.secret></portal.application.secret>
      </properties>
</profile>
```
Using the profile above, exceute the Maven command:

```
mvn clean jetty:run -Pgbif-portal
```

## Releasing the project
To release this project use the following Maven command:
```
mvn clean package verify -Pgbif-portal,release
```

## Project structure

The initial version of this project was created using this a base design that is documented [here](draftportal.md).

### Apache Struts2 usage

#### URL conventions
All the URLs are exposed following a restful pattern, pretty urls like the following.
The examples are mainly for datasets, but would be similar for species, occurrences, etc

/ (portal home)
/dataset (datasets home)
/dataset/search?q=puma (search datasets for the word puma)
/dataset/{UUID} (detail page for specific dataset)
/dataset/{UUID}/stats (stats page for specific dataset X)
/dataset/{UUID}/activity (activity page for specific dataset X)
/dataset/{UUID}/discussion (discussion page for specific dataset X)

#### Action mapping
we use wildcard mapping with the namedVariable pattern matcher for the default action mapper.
http://struts.apache.org/2.x/docs/wildcard-mappings.html

The conventions plugin (http://struts.apache.org/2.x/docs/convention-plugin.html) looked good, but failed to do the above urls with id params in the middle of the urls.
It also isnt as configurable as the explicit struts.xml definitions, for example to change http headers, content type, serving binary (eg image) streams, etc so it was finally not used.

#### Case sensitivity
URLs should be case insensitive. Struts, like servlets, seems case sensitive by default.
We need to see if we can change a setting or use a rewrite to lower case filter?

#### Internationalisation
We are going to stick to a single resource file. Markus's point of replacing the native struts2 text provider makes sense now.
not done yet as we expect major changes in the html still and its less work to replace the real strings once we reached a considerable stable state. Otherwise we also run into lots of orphaned entries. Use the struts tags for i18n, for example <@s.text name="menu.species"/>
Consider replacing the native struts2 text provider with a much simpler one we use in the IPT that increased page rendering with many getText lookups by more than 100% as the native one does an extensive resource bundle search across various classpaths and other places that we dont need if we stick to a single resource file.

##### Using i18n in JQuery
to make available the properties from Java's resource bundle to javascript, a jquery plugin is used: http://plugins.jquery.com/project/resourcebundle
To make any property available, this process should be followed:
1. On the Action responsible for passing the values to the ftl, create a getResourceBundleProperties() method that calls the superclass's method:  
  Example:
  ```
      MyAction extends BaseAction
         //this will return ALL properties
       public Map<String, String> getResourceBundleProperties() {
          return super.getResourceBundleProperties();
       }
         //this will return just properties starting with "rank."
       public Map<String, String> getResourceBundleProperties() {
          return super.getResourceBundleProperties("rank.");
       }
         //this will return just properties starting with "rank." and "species."
         //you can add as many prefixes as needed
       public Map<String, String> getResourceBundleProperties() {
           return super.getResourceBundleProperties("rank.", "species.");<commit_msg>Note about release and deployment.<commit_after>## GBIF-Portal

Java Web Application to exposes the functionality that mostly is produced consuming the GBIF REST services, such
functionality includes:
  * Dataset: faceted search and detail dataset view.
  * Species: faceted search and detail view.
  * Occurrence: filtered search and detail view.
  * Publisher: search and detail view.
  * Network/Node: simple search and detail view.
  * Netwoork: detail view.
  * IPT: display information about releases and statistics.
  * Developer: documentation about the REST API services.
  * Infrastructure: documentation about the occurrence data indexing, processing and tools.

This portal was developed using the following technologies: [Strust2](http://struts.apache.org/), [Freemarker](http://freemarker.org/),
[Less](http://lesscss.org/), [Leaflet](http://leafletjs.com/) and  [JQuery](https://jquery.com/).

## How to built the project
This project requires a Maven with the following settings defined in the properties section:

```
<profile>
  <id>gbif-portal</id>
  <properties>
        <cargo.context>portal</cargo.context>
        <devMode>false</devMode>
        <notDevMode>true</notDevMode>
        <baseurl>http://www.gbif-dev.org</baseurl>
        <api.baseurl>http://api.gbif-dev.org/v1</api.baseurl>
        <servername>http://www.gbif-dev.org</servername>
        <drupal.url>http://www.gbif-dev.org</drupal.url>
        <url.includeContext>false</url.includeContext>
        <httpTimeout>60000</httpTimeout>
        <maxHttpConnections>100</maxHttpConnections>
        <maxHttpConnectionsPerRoute>100</maxHttpConnectionsPerRoute>
        <annosys.url>https://annosys.bgbm.fu-berlin.de/AnnoSysTest/</annosys.url>
        <registry.ws.url>http://api.gbif-dev.org/v1/</registry.ws.url>
        <checklistbank.ws.url>http://api.gbif-dev.org/v1/</checklistbank.ws.url>
        <checklistbank.match.ws.url>http://api.gbif-dev.org/v1/species/match/</checklistbank.match.ws.url>
        <occurrence.ws.url>http://api.gbif-dev.org/v1/</occurrence.ws.url>
        <occurrence.ws.url.public>http://api.gbif-dev.org/v1/</occurrence.ws.url.public>
        <tile-server.url>http://api.gbif-dev.org/v1/map</tile-server.url>
        <image-cache.url>http://api.gbif-dev.org/v1/image</image-cache.url>
        <metrics.ws.url>http://api.gbif-dev.org/v1/</metrics.ws.url>
        <drupal.cookiename>drupal_cookiename</drupal.cookiename>
        <drupal.db.host>mysqlserver</drupal.db.host>
        <drupal.db.url>jdbc:mysql://mysqlserver:3306/drupal_db?useUnicode=true&amp;characterEncoding=UTF8&amp;characterSetResults=UTF8</drupal.db.url>
        <drupal.db.name>drupal_db_username</drupal.db.name>
        <drupal.db.username>drupal_db_password</drupal.db.username>
        <drupal.db.poolSize>8</drupal.db.poolSize>
        <drupal.db.connectionTimeout>2000</drupal.db.connectionTimeout>
        <portal.application.secret></portal.application.secret>
      </properties>
</profile>
```
Using the profile above, exceute the Maven command:

```
mvn clean jetty:run -Pgbif-portal
```

## Releasing and deploying the project
Releases can be done from `builds.gbif.org`.
```
cd code-release/portal-web
git pull
mvn -s ../settings.xml -Pportal-dev,secrets-dev,release clean release:prepare
mvn -s ../settings.xml -Pportal-dev,secrets-dev,release release:perform
```

Deploys should be done using the scripts in the private `deploy` Git repository, from either `uatapps-vh.gbif.org` or `prodapps-vh.gbif.org`, in `/root/deploy`.

See [README](https://github.com/gbif/deploy/blob/master/portal-web/README.md) in that project.

## Project structure

The initial version of this project was created using this a base design that is documented [here](draftportal.md).

### Apache Struts2 usage

#### URL conventions
All the URLs are exposed following a restful pattern, pretty urls like the following.
The examples are mainly for datasets, but would be similar for species, occurrences, etc

/ (portal home)
/dataset (datasets home)
/dataset/search?q=puma (search datasets for the word puma)
/dataset/{UUID} (detail page for specific dataset)
/dataset/{UUID}/stats (stats page for specific dataset X)
/dataset/{UUID}/activity (activity page for specific dataset X)
/dataset/{UUID}/discussion (discussion page for specific dataset X)

#### Action mapping
we use wildcard mapping with the namedVariable pattern matcher for the default action mapper.
http://struts.apache.org/2.x/docs/wildcard-mappings.html

The conventions plugin (http://struts.apache.org/2.x/docs/convention-plugin.html) looked good, but failed to do the above urls with id params in the middle of the urls.
It also isnt as configurable as the explicit struts.xml definitions, for example to change http headers, content type, serving binary (eg image) streams, etc so it was finally not used.

#### Case sensitivity
URLs should be case insensitive. Struts, like servlets, seems case sensitive by default.
We need to see if we can change a setting or use a rewrite to lower case filter?

#### Internationalisation
We are going to stick to a single resource file. Markus's point of replacing the native struts2 text provider makes sense now.
not done yet as we expect major changes in the html still and its less work to replace the real strings once we reached a considerable stable state. Otherwise we also run into lots of orphaned entries. Use the struts tags for i18n, for example <@s.text name="menu.species"/>
Consider replacing the native struts2 text provider with a much simpler one we use in the IPT that increased page rendering with many getText lookups by more than 100% as the native one does an extensive resource bundle search across various classpaths and other places that we dont need if we stick to a single resource file.

##### Using i18n in JQuery
to make available the properties from Java's resource bundle to javascript, a jquery plugin is used: http://plugins.jquery.com/project/resourcebundle
To make any property available, this process should be followed:
1. On the Action responsible for passing the values to the ftl, create a getResourceBundleProperties() method that calls the superclass's method:
  Example:
  ```
      MyAction extends BaseAction
         //this will return ALL properties
       public Map<String, String> getResourceBundleProperties() {
          return super.getResourceBundleProperties();
       }
         //this will return just properties starting with "rank."
       public Map<String, String> getResourceBundleProperties() {
          return super.getResourceBundleProperties("rank.");
       }
         //this will return just properties starting with "rank." and "species."
         //you can add as many prefixes as needed
       public Map<String, String> getResourceBundleProperties() {
           return super.getResourceBundleProperties("rank.", "species.");
{'fraction_non_alphanumeric': 0.10638456937799043, 'fraction_numerical': 0.0067284688995215315, 'mean_word_length': 3.8625954198473282, 'pattern_counts': {'":': 0, '<': 135, '<?xml version=': 0, '>': 135, 'https://': 5, 'lorem ipsum': 0, 'www.': 6, 'xml': 4}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4680155', 'n_tokens_mistral': 3932, 'n_tokens_neox': 3824, 'n_words': 1235}