<commit_before># poc-analysis-results-metadata

This repository contains programs for proof of concept for generating
tables from Analysis Results &amp; Metadata model
(http://www.phusewiki.org/wiki/index.php?title=Analysis_Results_Model),
a project under the Semantic Technology
(http://www.phusewiki.org/wiki/index.php?title=Semantic_Technology) in
the PhUSE Computational Science (CS) working groups
(http://www.phusewiki.org/wiki/index.php?title=CSS_Working_Groups).

The purpose is to show how tables for the clinical study report can be
generated from analysis results and metadata stored a RDF data cubes.
The CDISC SDTM/ADaM Pilot Project
(http://www.cdisc.org/sdtmadam-pilot-project) serves as input for data
and table specification.

Several tools are used, see below for list of tools/packages.

Previous versions of Apache Jena and Apache Fuseki versions are
choosen as they can be executed with Java 7. Never versions exists,
but they require Java 8, which is not always available.

# Setup

This right now describes a rather convoluted setup.

The recommended setup is to store all tools/packages within one
directory (see below). This simplifies reference between packages, as
relative paths can be used.

Directory structure

    apache-jena-2.13.0
    apache-jena-fuseki-2.0.0
    poc-analysis-results-metadata
    rdf.cdisc.org
	phuse-scripts
    rrdfqbcrnd0
    SAS-SPARQLwrapper

## Tools/packages

The tools/packages are shown below (including the present projects).

### Java runtime (for Apache Jena)

Note SAS provides a private java runtime, which may be used - though
not recommended.  The path belows shows one possible location for a
default SAS 9.4 installation on windows 10.

For windows use:

    path C:\Program Files\SASHome\SASPrivateJavaRuntimeEnvironment\9.4\jre\bin;C:\Program Files\SASHome\SASPrivateJavaRuntimeEnvironment\9.4\bin 

### Web Server (for the application - not needed for generating RDF data cubes)

The current setup uses python 3 with the modulse -m http.server

The use of a java based webserver is investigated, e.g. tomcat or jetty.

#### Using tomcat

This is the very first attempt - and is most likely not the best approach.

install tomcat from (http://tomcat.apache.org/download-80.cgi).
Unpack to same level as the other packages.

Create directory for the application TT05 and copy the contents to that directory;

    mkdir apache-tomcat-8.5.4\webapps\TT05
    copy poc-analysis-results-metadata\application-html\ apache-tomcat-8.5.4\webapps\TT05

Start tomcat:

     set JAVA_HOME=c:\Program Files\Java\jre1.8.0_101
     cd apache-tomcat-8.5.4\bin
     startup.bat
	 
### Apache Jena

Apache Jena - see (https://jena.apache.org/download/), download at (http://archive.apache.org/dist/jena/binaries/)

### Apache Jena Fuseki

Apache Jena Fuseki - version 2.0.0, see below

* Zip (http://archive.apache.org/dist/jena/binaries/apache-jena-fuseki-2.0.0.zip)
* MD5 (http://archive.apache.org/dist/jena/binaries/apache-jena-fuseki-2.0.0.zip.md5)

Apache Jena - version 2.13.0, optional, see below

* Zip (http://archive.apache.org/dist/jena/binaries/apache-jena-2.13.0.zip)
* MD5 (http://archive.apache.org/dist/jena/binaries/apache-jena-2.13.0.zip.md5)

### GitHub

GitHub packages as zip files

    https://github.com/phuse-org/rdf.cdisc.org/archive/master.zip<commit_msg>Added details on tomcat on linux<commit_after># poc-analysis-results-metadata

This repository contains programs for proof of concept for generating
tables from Analysis Results &amp; Metadata model
(http://www.phusewiki.org/wiki/index.php?title=Analysis_Results_Model),
a project under the Semantic Technology
(http://www.phusewiki.org/wiki/index.php?title=Semantic_Technology) in
the PhUSE Computational Science (CS) working groups
(http://www.phusewiki.org/wiki/index.php?title=CSS_Working_Groups).

The purpose is to show how tables for the clinical study report can be
generated from analysis results and metadata stored a RDF data cubes.
The CDISC SDTM/ADaM Pilot Project
(http://www.cdisc.org/sdtmadam-pilot-project) serves as input for data
and table specification.

Several tools are used, see below for list of tools/packages.

Previous versions of Apache Jena and Apache Fuseki versions are
choosen as they can be executed with Java 7. Never versions exists,
but they require Java 8, which is not always available.

# Setup

This right now describes a rather convoluted setup.

The recommended setup is to store all tools/packages within one
directory (see below). This simplifies reference between packages, as
relative paths can be used.

Directory structure

    apache-jena-2.13.0
    apache-jena-fuseki-2.0.0
    poc-analysis-results-metadata
    rdf.cdisc.org
	phuse-scripts
    rrdfqbcrnd0
    SAS-SPARQLwrapper

## Tools/packages

The tools/packages are shown below (including the present projects).

### Java runtime (for Apache Jena)

Note SAS provides a private java runtime, which may be used - though
not recommended.  The path belows shows one possible location for a
default SAS 9.4 installation on windows 10.

For windows use:

    path C:\Program Files\SASHome\SASPrivateJavaRuntimeEnvironment\9.4\jre\bin;C:\Program Files\SASHome\SASPrivateJavaRuntimeEnvironment\9.4\bin 

### Web Server (for the application - not needed for generating RDF data cubes)

The current setup uses python 3 with the modulse -m http.server

The use of a java based webserver is investigated, e.g. tomcat or jetty.

#### Using tomcat

This is the very first attempt - and is most likely not the best approach.

install tomcat from (http://tomcat.apache.org/download-80.cgi).
Unpack to same level as the other packages.

Create directory for the application TT05 and copy the contents to that directory;

    mkdir apache-tomcat-8.5.4\webapps\TT05
    copy poc-analysis-results-metadata\application-html\ apache-tomcat-8.5.4\webapps\TT05

Start tomcat:

     set JAVA_HOME=c:\Program Files\Java\jre1.8.0_101
     cd apache-tomcat-8.5.4\bin
     startup.bat

#### Tomcat on Linux

The approach below is tested on Fedora 24.

Install tomcat

     dnf install tomcat


Copy application files from default tomcat webapps directory.

     webappdir=/usr/share/tomcat/webapps/TT05
     cd application-html
	 # as root, or other with write access to the tomcat webapps directory
     mkdir ${webappdir}
     chgrp -R tomcat ${webappdir}
     chown -R tomcat 
     cp -rv . ${webappdir}


### Apache Jena

Apache Jena - see (https://jena.apache.org/download/), download at (http://archive.apache.org/dist/jena/binaries/)

### Apache Jena Fuseki

Apache Jena Fuseki - version 2.0.0, see below

* Zip (http://archive.apache.org/dist/jena/binaries/apache-jena-fuseki-2.0.0.zip)
* MD5 (http://archive.apache.org/dist/jena/binaries/apache-jena-fuseki-2.0.0.zip.md5)

Apache Jena - version 2.13.0, optional, see below

* Zip (http://archive.apache.org/dist/jena/binaries/apache-jena-2.13.0.zip)
* MD5 (http://archive.apache.org/dist/jena/binaries/apache-jena-2.13.0.zip.md5)

### GitHub

GitHub packages as zip files

    https://github.com/phuse-org/rdf.cdisc.org/archive/master.zip
{'fraction_non_alphanumeric': 0.09226441631504922, 'fraction_numerical': 0.019409282700421943, 'mean_word_length': 3.904137931034483, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 4, 'lorem ipsum': 0, 'www.': 8, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2508240', 'n_tokens_mistral': 2330, 'n_tokens_neox': 2236, 'n_words': 739}