<commit_before>
=============
odML Tutorial
=============

:Author:
	<NAME>;
	based on work by <NAME>
:Release:
	0.1
:License:
	Creative Commons Attribution-ShareAlike 4.0 International 
	`License <http://creativecommons.org/licenses/by-sa/4.0/>`_
-------------------------------------------------------------------------------


odML (open metadata Markup Language)
====================================

odML (open metadata Markup Language) is an XML based file format, 
proposed by [Grewe et al. (2011) Front Neuroinform 5:16], in order 
to provide metadata in an organized, human- and machine-readable way. 
In this tutorial we will illustrate the conceptual design of odML and 
show hands-on how you can generate your own odML metadata collection. 
In addition, we demonstrate the advantages of using odML to screen 
large numbers of data sets according to selection criteria relevant for 
subsequent analyses. Well organized metadata management is a key 
component to guarantee reproducibility of experiments and to track 
provenance of performed analyses.

What are metadata and why are they needed?
	Metadata are data about data. They describe the conditions under which the 
	actual raw-data of an experimental study were acquired. The organization of 
	such metadata and their accessibility may sound like a trivial task, and 
	most laboratories developed their home-made solutions to keep track of 
	their metadata. Most of these solutions, however, break down if data and 
	metadata need to be shared within a collaboration, because implicit 
	knowledge of what is important and how it is organized is often 
	underestimated.

	While maintaining the relation to the actual raw-data, odML can help to 
	collect all metadata which are usually distributed over several files and 
	formats, and to store them unitetly which facilitates sharing data and 
	metadata.

Key features of odML
	- open, XML based language, to collect, store and share metadata
	- Machine- and human-readable
	- Interactive odML-Editor
	- Python-odML library
-------------------------------------------------------------------------------


Structure of this tutorial
==========================

The scientific background of the possible user community of odML varies 
enormously (e.g. physics, informatics, mathematics, biology, medicine,
psychology). Some users will be trained programmers, others probably have never 
learned a programming language. 

To cover the different demands of all users, we first provide a slow 
introduction to odML that allows programming beginners to learn the basic 
concepts. In a next step, we will demonstrate how to generate an odML file via 
the Python-odML library. In later chapters we present more advanced possibilies 
of the Python-odML library (e.g. how to search for certain metadata or how to
integrate existing terminologies or templates). 

Although the structure of an odML is depending on the needs of each individual 
user, we would like to provide at the end of this tutorial a few guidelines.

The code for the example odML files, which we use within this tutorial is part 
of the documentation package (see doc/example_odMLs/). 

A summary of available odML terminologies and templates can be found `here
<http://portal.g-node.org/odml/terminologies/v1.0/terminologies.xml>`_. 


-------------------------------------------------------------------------------


Download and Installation
=========================

The Python-odML library (including the odML-Editor) is available on 
`GitHub <https://github.com/G-Node/python-odml>`_. If you are not familiar with 
the version control system **git**, but still want to use it, have a look at 
the documentaion available on the `git-scm website <https://git-scm.com/>`_. 

Dependencies
------------

The Python-odML library runs under Python 2.7. 

Additionally, the Python-odML library depends on Enum (version 0.4.4).


Installation
------------

To download the Python-odML library please either use git and clone the 
repository from GitHub::

	$ cd /home/usr/toolbox/
	$ git clone https://github.com/G-Node/python-odml.git
	
... or if you don't want to use git download the ZIP file also provided on 
GitHub to your computer (e.g. as above on your home directory under a "toolbox" 
folder).

To install the Python-odML library, enter the corresponding directory and run::

	$ cd /home/usr/toolbox/python-odml/
	$ python setup.py install
	

Bugs & Questions
----------------

Should you find a behaviour that is likely a bug, please file a bug report at 
`the github bug tracker <https://github.com/G-Node/python-odml/issues>`_.

If you have questions regarding the use of the library or the editor, ask<commit_msg>format updates in intro and installation section<commit_after>
=============
odML Tutorial
=============

:Author:
	<NAME>;
	based on work by <NAME>
:Release:
	0.2
:License:
	Creative Commons Attribution-ShareAlike 4.0 International 
	`License <http://creativecommons.org/licenses/by-sa/4.0/>`_
-------------------------------------------------------------------------------


odML (open metadata Markup Language)
====================================

odML (**o**pen meta**d**ata **M**arkup **L**anguage) is a framework, proposed 
by `Grewe et al. (2011) <http://journal.frontiersin.org/article/10.3389/fninf.2011.00016/full>`_, 
to organize and store experimental metadata in a human- and machine-readable, 
XML based format (odml). In this tutorial we will illustrate the conceptual 
design of the odML framework and show hands-on how you can generate your own 
odML metadata file collection. A well organized metadata management of your 
experiment is a key component to guarantee the reproducibility of your research 
and facilitate the provenance tracking of your analysis projects.

What are metadata and why are they needed?
	Metadata are data about data. They describe the conditions under which the 
	actual raw-data of an experimental study were acquired. The organization of 
	such metadata and their accessibility may sound like a trivial task, and 
	most laboratories developed their home-made solutions to keep track of 
	their metadata. Most of these solutions, however, break down if data and 
	metadata need to be shared within a collaboration, because implicit 
	knowledge of what is important and how it is organized is often 
	underestimated.

	While maintaining the relation to the actual raw-data, odML can help to 
	collect all metadata which are usually distributed over several files and 
	formats, and to store them unitetly which facilitates sharing data and 
	metadata.

Key features of odML
	- open, XML based language, to collect, store and share metadata
	- Machine- and human-readable
	- Interactive odML-Editor
	- Python-odML library
-------------------------------------------------------------------------------


Structure of this tutorial
==========================

The scientific background of the possible user community of odML varies 
enormously (e.g. physics, informatics, mathematics, biology, medicine,
psychology). Some users will be trained programmers, others probably have never 
learned a programming language. 

To cover the different demands of all users, we provide a slow introduction to 
the odML framework that even allows programming beginners to learn the basic 
concepts. We will demonstrate how to generate an odML file and present more 
advanced possibilies of the Python-odML library (e.g., how to search for 
certain metadata or how to integrate existing terminologies). 

At the end of this tutorial we will provide a few guidelines that will help you
to create an odML file structure that is optimised for your individual 
experimental project and complements the special needs of your laboratory. 

The code for the example odML files, which we use within this tutorial is part 
of the documentation package (see doc/example_odMLs/). 

A summary of available odML terminologies and templates can be found `here
<http://portal.g-node.org/odml/terminologies/v1.0/terminologies.xml>`_. 


-------------------------------------------------------------------------------


Download and Installation
=========================

The odML framework is an open source project of the **G**erman Neuroinformatics
**Node** (`G-Node <http://www.g-node.org/`_, 
`odML project website <http://www.g-node.org/projects/odml>`_) of the 
**I**nternational **N**euroinformatics **C**oordination **F**acility 
(`INCF <http://www.g-node.org/`_). The source code for the Python library is 
available on `GitHub <https://github.com/>`_ under the project name 
`python-odml <https://github.com/G-Node/python-odml>`_.

Dependencies
------------

The Python-odML library (version 1.3) runs under Python 2.7 or 3.5. 

Additionally, the Python-odML library depends on Enum (version 0.4.4).

When the odML-Python library is installed via pip or the setup.py, these 
packages will be automatically downloaded and installed. Alternatively, they 
can be installed from the OS package manager. 

On Ubuntu, the dependency packages are available as:
- python-enum
- python-lxml

If you prefer the later way of installion, the following packages are required
to build the python-lxml package on Ubuntu 14.04:
- libxml2-dev
- libxslt1-dev
- lib32z1-dev


Installation...
---------------

... via pip:
************

The simplest way to install the Python-odML library is from 
`PyPI <https://pypi.python.org/pypi>`_ using 
`pip <https://pip.pypa.io/en/stable/>`_::

	$ pip install odml

The appropriate Python dependencies (Enum and lxml) will be automatically 
downloaded and installed.

If you are not familiar with PyPI and pip, please have a look at the available
online documentation.

Installation
------------

To download the Python-odML library please either use git and clone the 
repository from GitHub::

	$ cd /home/usr/toolbox/
	$ git clone https://github.com/G-Node/python-odml.git
	
... or if you don't want to use git download the ZIP file also provided on 
GitHub to your computer (e.g. as above on your home directory under a "toolbox" 
folder).

To install the Python-odML library, enter the corresponding directory and run::

	$ cd /home/usr/toolbox/python-odml/
	$ python setup.py install
	

Bugs & Questions
----------------

Should you find a behaviour that is likely a bug, please file a bug report at 
`the github bug tracker <https://github.com/G-Node/python-odml/issues>`_.

If you have questions regarding the use of the library or the editor, ask
{'fraction_non_alphanumeric': 0.13400590082801941, 'fraction_numerical': 0.006186351955838965, 'mean_word_length': 3.2680747359870024, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 21, 'https://': 10, 'lorem ipsum': 0, 'www.': 3, 'xml': 6}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3049049', 'n_tokens_mistral': 2810, 'n_tokens_neox': 2611, 'n_words': 1314}