<commit_before>=Versions=
This documentation covers 0.3-beta. The 0.1.x
versions of Webspr come with a README containing full documentation.
0.2.x versions are documented in another page on the wiki.
The
latest version of this document can be found at
http://code.google.com/p/webspr/wiki/Documentation_0.3.
See the end of this document for the changelog.

Note that prior to version 0.3 this software was called "webspr",
not "Ibex".

=Requirements=
    * Python >= 2.3. (Python 3000 not currently supported.)

=Introduction=
Ibex ("Internet Based EXperiments) allows you to run various kinds of psycholinguistic experiment online.
The system
is modular and the range of possible experiments is steadily growing. Ibex
uses !JavaScript and HTML only, and doesn't require the use of plugins such as
Java or Flash.

=Setting up the server=
There are two ways of running the server: either using the stand-alone toy HTTP server,
or as a CGI process. By default, the server runs in stand-alone mode. To start
it, change to the `www` directory and execute server.py:
{{{
    python server.py
}}}
This will start the server on port 3000 by default (so the
experiment will be at http://localhost:3000/experiment.html).

The stand-alone server is limited to serving files relating to Ibex, so it is
unlikely to be useful for real work (unless perhaps you hide it behind a proxy).
However, it is useful for testing experimental designs without setting up a
real HTTP server, and for running experiments offline.

The server can be configured by editing `server_conf.py`. Each of the
options is commented. Change the `SERVER_MODE` variable to determine
whether the server runs in stand-alone mode or as a CGI
application. When `SERVER_MODE` is set to "cgi", `server.py` will work
as a standard CGI program; when it is set to "toy", the server will
run in stand-alone mode. (The value "paste" has the same effect as
"toy", for backwards compatibility with 0.2.x version.) The value of
the `PORT` variable determines the port the server will listen on when
operating in stand-alone mode. As an alternative to editing the
configuration file, the `-m` and `-p` command-line options can also be
used to set the server mode and port respectively. Command-line options
override those set in `server_conf.py`.

*New:* Version 0.3 adds some new ways of configuring the server.
These are unlikely to be of interest to most users, but are documented
in the "New Configuration Methods" section below.

If running the server as a CGI application, make sure that the web
server can serve up `experiment.html`, and the other files in the
`www` dir, as static files. All dynamic requests go through
`server.py`, so as long as the files in `www` can be accessed, and the
server recognizes `server.py` as a CGI application, everything should
work. The "Step-by-step CGI" section below goes through the process
of setting up webspr as a CGI app.

The directories `js_includes`, `data_includes` and `css_includes`
contain !JavaScript and CSS files required for particular kinds of
experiment. When an HTTP request is made for `server.py?include=js`,
`server.py?include=data` or `server.py?include=css`, the server
concatenates all the files in the specified directory and serves up
the result. Your data file(s) will live in `data_includes`.
*Note:* The server mangles CSS files by adding a prefix to each
class/id name based on the name of the CSS file; see the section "CSS
mangling" below for more details.

The server stores some state in the `server_state` directory. This
directory is automatically created by the server if it does not
already exist, and in order to start the server from a fresh state you
may simply delete this directory and/or its contents. Currently, the
directory just contains the counter used to alternate latin square
designs (see "Latin Square Designs" section below). This counter can
be reset on server startup using the `-r` option.

The server logs messages in the file `server.log`. The default is to
store all files in the same directory as `server.py` (i.e. logs,
results, the `server_state` directory, etc.), but the working
directory of the server can be modified by setting the variable
`IBEX_WORKING_DIR` in `server_conf.py`. You can also set the
environment variable of the same name.  The value of this environment
variable overrides the value of the variable in `server_conf.py`.

On Linux/Unix/OS X, the server uses file locking to ensure that the
results file and server state remain consistent. Currently, it does
not do so on Windows, so there is a theoretical possibility of the
server state or the results being corrupted if the server is deployed
in a Windows environment (but unless there is very high traffic, it's
purely theoretical).

The stand-alone server has been tested on Windows and OS X (but will
almost certainly work on any system with Python 2.3-2.6).  The CGI
server has been tested on OS X and Linux using the lighttpd web server
in both cases.

==New Configuration Methods==
Version 0.3 of this software introduces two new methods of setting
configuration variables. The first is simply to include the variable
definitions in `server_conf.py` directly in `server.py`, obviating the need
to set the `SERVER_CONF_PY_FILE` variables in `server.py`. The second
is to specify that `server.py` should issue an HTTP GET request to a given
URL, with the result obtained being interpreted as a JSON dictionary specifying the
values of the configuration variables. This mode is configured by setting
the configuration variables `EXTERNAL_CONFIG_URL`, `EXTERNAL_CONFIG_METHOD`
and `EXTERNAL_CONFIG_PASS_PARAMS`. The first of these variables is self-explanatory;
the second must be set to `"GET"`; the third should be set to a boolean,
specifying whether or not `server.py` should set a `dir` paramater in the
query string giving the directory in which `server.py` resides.
Note that the values of these three variables may be set either in `server_conf.py`,
or directly in `server.py`.

The new configuration methods are used to handle the configuration
of experiments in the Ibex Farm (http://spellout.net/ibexfarm).
They are unlikely to be of interest to other users.

==Step-by-step CGI==
This subsection describes how to set up webspr as a CGI application from
scratch.  Since the particulars of different web servers differ, some parts are
necessarily rather vague. There are many possible ways of organizing the
files/directories; the setup described below is just the simplest.

*Step 1:* Ensure that the `www` directory is somewhere within the root
directory of your HTTP server. You may want to place the entire `webspr`
directory within the root directory, or you may want to copy the files in `www`
to a new location within the root directory. All of these files are ordinary
static files except for `server.py`, which needs to be run as a CGI app.

*Step 2:* Decide on a location for the main webspr directory. Depending on the
details of your setup, you may be able to leave it in place, or you may need to
copy it to a directory which the HTTP server has permission to access (so that
`server.py` may access files in this directory when it is executed).

*Step 3:* Edit `server.py` and change the value of<commit_msg>Manual.<commit_after>=Versions=
This documentation covers 0.3-beta. The 0.1.x
versions of Webspr come with a README containing full documentation.
0.2.x versions are documented in another page on the wiki.
The
latest version of this document can be found at
http://code.google.com/p/webspr/wiki/Documentation_0.3.
See the end of this document for the changelog.

Note that prior to version 0.3 this software was called "webspr",
not "Ibex".

=Requirements=
    * Python >= 2.3. (Python 3000 not currently supported.)

=Introduction=
Ibex ("Internet Based EXperiments) allows you to run various kinds of psycholinguistic experiment online.
The system
is modular and the range of possible experiments is steadily growing. Ibex
uses !JavaScript and HTML only, and doesn't require the use of plugins such as
Java or Flash.

=Setting up the server=
There are two ways of running the server: either using the stand-alone toy HTTP server,
or as a CGI process. By default, the server runs in stand-alone mode. To start
it, change to the `www` directory and execute server.py:
{{{
    python server.py
}}}
This will start the server on port 3000 by default (so the
experiment will be at http://localhost:3000/experiment.html).

The stand-alone server is limited to serving files relating to Ibex, so it is
unlikely to be useful for real work (unless perhaps you hide it behind a proxy).
However, it is useful for testing experimental designs without setting up a
real HTTP server, and for running experiments offline.

The server can be configured by editing `server_conf.py`. Each of the
options is commented. Change the `SERVER_MODE` variable to determine
whether the server runs in stand-alone mode or as a CGI
application. When `SERVER_MODE` is set to "cgi", `server.py` will work
as a standard CGI program; when it is set to "toy", the server will
run in stand-alone mode. (The value "paste" has the same effect as
"toy", for backwards compatibility with 0.2.x version.) The value of
the `PORT` variable determines the port the server will listen on when
operating in stand-alone mode. As an alternative to editing the
configuration file, the `-m` and `-p` command-line options can also be
used to set the server mode and port respectively. Command-line options
override those set in `server_conf.py`.

*New:* Version 0.3 adds some new ways of configuring the server.
These are unlikely to be of interest to most users, but are documented
in the "New Configuration Methods" section below.

If running the server as a CGI application, make sure that the web
server can serve up `experiment.html`, and the other files in the
`www` dir, as static files. All dynamic requests go through
`server.py`, so as long as the files in `www` can be accessed, and the
server recognizes `server.py` as a CGI application, everything should
work. The "Step-by-step CGI" section below goes through the process
of setting up webspr as a CGI app.

The directories `js_includes`, `data_includes` and `css_includes`
contain !JavaScript and CSS files required for particular kinds of
experiment. When an HTTP request is made for `server.py?include=js`,
`server.py?include=data` or `server.py?include=css`, the server
concatenates all the files in the specified directory and serves up
the result. Your data file(s) will live in `data_includes`.
*Note:* The server mangles CSS files by adding a prefix to each
class/id name based on the name of the CSS file; see the section "CSS
mangling" below for more details.

The server stores some state in the `server_state` directory. This
directory is automatically created by the server if it does not
already exist, and in order to start the server from a fresh state you
may simply delete this directory and/or its contents. Currently, the
directory just contains the counter used to alternate latin square
designs (see "Latin Square Designs" section below). This counter can
be reset on server startup using the `-r` option.

The server logs messages in the file `server.log`. The default is to
store all files in the same directory as `server.py` (i.e. logs,
results, the `server_state` directory, etc.), but the working
directory of the server can be modified by setting the variable
`IBEX_WORKING_DIR` in `server_conf.py`. You can also set the
environment variable of the same name.  The value of this environment
variable overrides the value of the variable in `server_conf.py`.

On Linux/Unix/OS X, the server uses file locking to ensure that the
results file and server state remain consistent. Currently, it does
not do so on Windows, so there is a theoretical possibility of the
server state or the results being corrupted if the server is deployed
in a Windows environment (but unless there is very high traffic, it's
purely theoretical).

The stand-alone server has been tested on Windows and OS X (but will
almost certainly work on any system with Python 2.3-2.6).  The CGI
server has been tested on OS X and Linux using the lighttpd web server
in both cases.

==New Configuration Methods==
Version 0.3 of this software introduces two new methods of setting
configuration variables. The first is simply to include the variable
definitions in `server_conf.py` directly in `server.py`, obviating the need
to set the `SERVER_CONF_PY_FILE` variables in `server.py`. The second
is to specify that `server.py` should issue an HTTP GET request to a given
URL, with the result obtained being interpreted as a JSON dictionary specifying the
values of the configuration variables. This mode is configured by setting
the configuration variables `EXTERNAL_CONFIG_URL`, `EXTERNAL_CONFIG_METHOD`
and `EXTERNAL_CONFIG_PASS_PARAMS`. The first of these variables is self-explanatory;
the second must be set to `"GET"`; the third should be set to a boolean,
specifying whether or not `server.py` should pass a `dir` paramater in the
query string giving the directory in which `server.py` resides.
Note that the values of these three variables may be set either in `server_conf.py`,
or directly in `server.py`.

The new configuration methods are used to handle the configuration
of experiments in the Ibex Farm (http://spellout.net/ibexfarm).
They are unlikely to be of interest to other users.

==Step-by-step CGI==
This subsection describes how to set up webspr as a CGI application from
scratch.  Since the particulars of different web servers differ, some parts are
necessarily rather vague. There are many possible ways of organizing the
files/directories; the setup described below is just the simplest.

*Step 1:* Ensure that the `www` directory is somewhere within the root
directory of your HTTP server. You may want to place the entire `webspr`
directory within the root directory, or you may want to copy the files in `www`
to a new location within the root directory. All of these files are ordinary
static files except for `server.py`, which needs to be run as a CGI app.

*Step 2:* Decide on a location for the main webspr directory. Depending on the
details of your setup, you may be able to leave it in place, or you may need to
copy it to a directory which the HTTP server has permission to access (so that
`server.py` may access files in this directory when it is executed).

*Step 3:* Edit `server.py` and change the value of
{'fraction_non_alphanumeric': 0.0544953496383052, 'fraction_numerical': 0.005098174302445746, 'mean_word_length': 4.502653525398029, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 10, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7008613', 'n_tokens_mistral': 3810, 'n_tokens_neox': 3697, 'n_words': 2281}