<commit_before><!--
The contents of this file are subject to the MonetDB Public License
Version 1.1 (the "License"); you may not use this file except in
compliance with the License. You may obtain a copy of the License at
http://monetdb.cwi.nl/Legal/MonetDBLicense-1.1.html

Software distributed under the License is distributed on an "AS IS"
basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the
License for the specific language governing rights and limitations
under the License.

The Original Code is the MonetDB Database System.

The Initial Developer of the Original Code is CWI.
Portions created by CWI are Copyright (C) 1997-2006 CWI.
All Rights Reserved.
-->

<h2> The Gold Demo Database</h2> <p>
MonetDB has played a pivotal role in the development of datamining
applications at DataDistilleries (now SPSS).
During its course of development we used a small mock-up database
to study performance issues.
This database is included in the <b>MonetDB source</b> distribution directory <tt>scripts/gold</tt>
and provides an easy way to explore some of the MonetDB's functionality
when it comes to processing MIL queries.
The steps below should help you understand some of the basics in
using MonetDB.<br />
<b>Please note that you need a source distribution to follow this
tutorial.  Binary installations are currently not (yet) supported.</b>
</p>
<h4>Creating the database:</h4> <p>
We suppose your MonetDB sources are in <tt>$MONETDB_SRC</tt> and
you have configured and successfully installed MonetDB
in a directory <tt>$MONETDB_PREFIX<tt>.
Make sure <tt>$MONETDB_PREFIX/bin</tt> is in your PATH and type:
<pre>
$ mkdir $MONETDB_PREFIX/var/MonetDB/dbfarm/gold
</pre>
So, creating a database is nothing more than creating a directory in the 
'dbfarm' directory (the default dbfarm is <tt>$MONETDB_PREFIX/var/MonetDB/dbfarm</tt>)
This manual step is only meant as an illustration, because when the
environment variables are set, typing the following would produce
the same effects and produces a message that the database has been initialized:
<pre>
$ Mserver --dbname=gold
# Monet Database Server V4.6.0
# Copyright (c) 1993-2006, CWI. All rights reserved.
# Compiled for i686-pc-linux-gnu/32bit; dynamically linked.
# Visit http://monetdb.cwi.nl/ for further information.
MonetDB&gt;
</pre>
Note that the actual output may differ slightly, depending on
the version downloaded and the system it runs on.
A different dbfarm location can be chosen and passed to
Mserver with the <tt>--dbfarm=&lt;dir&gt;</tt> command line switch.
</p>
<h4>Loading the database:</h4> <p>
A small MIL script is provided to quickly populate the database.
It uses a compressed file with tuples to be inserted and a format
file for the database loader.
<pre>
$ cd $MONETDB_SRC/scripts/gold
$ export TSTTRGDIR=`pwd`
$ Mserver --dbname=gold &lt; load.mil
</pre>
</p>
<h4>Create two supporting tables:</h4>
<p>
The database is now loaded. A few more tables are constructed
for subsequent processing. 
Don't disturb the output layout, because this script is
also used every night as part of the intensive testing machinery.
It suffices to note that we now have a skewed insurance<commit_msg>Made documentation aware of monetdb assets location<commit_after><!--
The contents of this file are subject to the MonetDB Public License
Version 1.1 (the "License"); you may not use this file except in
compliance with the License. You may obtain a copy of the License at
http://monetdb.cwi.nl/Legal/MonetDBLicense-1.1.html

Software distributed under the License is distributed on an "AS IS"
basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the
License for the specific language governing rights and limitations
under the License.

The Original Code is the MonetDB Database System.

The Initial Developer of the Original Code is CWI.
Portions created by CWI are Copyright (C) 1997-2006 CWI.
All Rights Reserved.
-->

<h2> The Gold Demo Database</h2> <p>
MonetDB has played a pivotal role in the development of datamining
applications at DataDistilleries (now SPSS).
During its course of development we used a small mock-up database
to study performance issues in datamining algorithms.
This database is included in the <b> MonetDB source</b> distribution 
directory <tt>scripts/gold</tt> and accessible from the MonetDB
website (http://monetdb.cwi.nl/Assets/).
It provides an easy way to explore some of the MonetDB's functionality
when it comes to processing MIL queries.<br/>
The steps below should help you understand some of the basics in
using MonetDB. We assume that you successfully installed a MonetDB
and have a copy of the Gold demo files. Its location is denoted
by the variable $GOLD.<br />
</p>
<h4>Creating the database:</h4> <p>
We assume MonetDB is successfully installed in a directory <tt>$MONETDB_PREFIX<tt>.
Make sure <tt>$MONETDB_PREFIX/bin</tt> is in your PATH and type:
<pre>
$ mkdir $MONETDB_PREFIX/var/MonetDB/dbfarm/gold
</pre>
So, creating a database is nothing more than creating a directory in the 
'dbfarm' directory (the default dbfarm is <tt>$MONETDB_PREFIX/var/MonetDB/dbfarm</tt>)
This manual step is only meant as an illustration, because when the
environment variables are properly set, typing the following would produce
the same effects and produces a message that the database has been initialized:
<pre>
$ Mserver --dbname=gold
# Monet Database Server V4.10.0
# Copyright (c) 1993-2006, CWI. All rights reserved.
# Compiled for i686-pc-linux-gnu/32bit; dynamically linked.
# Visit http://monetdb.cwi.nl/ for further information.
MonetDB&gt;
</pre>
Note that the actual output may differ slightly, depending on
the version downloaded and the system it runs on.
A different dbfarm location can be chosen and passed to
Mserver with the <tt>--dbfarm=&lt;dir&gt;</tt> command line switch.
</p>
<h4>Loading the database:</h4> <p>
A small MIL script is provided to quickly populate the database.
It uses a compressed file with tuples to be inserted and a format
file for the database loader.
<pre>
$ cd $GOLD
$ export TSTTRGDIR=`pwd`
$ Mserver --dbname=gold &lt; load.mil
</pre>
</p>
<h4>Create two supporting tables:</h4>
<p>
The database is now loaded. A few more tables are constructed
for subsequent processing. 
Don't disturb the output layout, because this script is
also used every night as part of the intensive testing machinery.
It suffices to note that we now have a skewed insurance
{'fraction_non_alphanumeric': 0.07349285270354257, 'fraction_numerical': 0.011342448725916719, 'mean_word_length': 4.6713656387665194, 'pattern_counts': {'":': 0, '<': 78, '<?xml version=': 0, '>': 78, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6626210', 'n_tokens_mistral': 1900, 'n_tokens_neox': 1794, 'n_words': 917}