<commit_before>#rdfstore-js


rdfstore-js is a pure Javascript implementation of a RDF graph store with support for the SPARQL query and data manipulation language.

    var rdfstore = require('rdfstore');
    
    rdfstore.create(function(store) {
      store.execute('LOAD <http://dbpedia.org/resource/Tim_Berners-Lee> INTO GRAPH <http://example.org/people>', function() {

        store.setPrefix('dbp', 'http://dbpedia.org/resource/');
        
        store.node(store.rdf.resolve('dbp:Tim_Berners-Lee'), "http://example.org/people", function(success, graph) {

          var peopleGraph = graph.filter(store.rdf.filters.type(store.rdf.resolve("foaf:Person")));
          
          store.execute('PREFIX rdf:  <http://www.w3.org/1999/02/22-rdf-syntax-ns#>\
                         PREFIX foaf: <http://xmlns.com/foaf/0.1/>\
                         PREFIX : <http://example.org/>\
                         SELECT ?s FROM NAMED :people { GRAPH ?g { ?s rdf:type foaf:Person } }',
                         function(success, results) {

                           console.log(peopleGraph.toArray()[0].subject.valueOf() === results[0].s.value);

                         });

        });

      });
    })

rdfstore-js can be executed in a web browser or can be included as a library in a node.js application. It can also be executed as a stand-alone SPARQL end-point accepting SPARQL RDF Protocol HTTP requests.

The current implementation is far from complete but it already passes all the test cases for the SPARQL 1.0 query language and supports data manipulation operations from the SPARQL 1.1/Update version of the language.

Some other features included in the library are the following:

- SPARQL 1.0 support
- SPARQL 1.1/Update support
- Partial SPARQL 1.1 query support
- JSON-LD parser
- Turtle/N3 parser
- W3C RDF Interfaces API
- RDF graph events API
- Parallel execution where WebWorkers are available
- Persistent storage using HTML5 LocalStorage in the browser version
- Persistent storage using MongoDB in the Node.js version
- Node.js HTTP server implementating the [SPARQL Protocol for RDF](http://www.w3.org/TR/rdf-sparql-protocol/) recommendation

## Documentation

Documentation for the store can be found [here](http://antoniogarrote.github.com/rdfstore-js/doc/index.html).

## SPARQL support

rdfstore-js supports at the moment SPARQL 1.0 and most of SPARQL 1.1/Update.
Only some parts of SPARQL 1.1 query have been implemented yet.

This is a list of the different kind of queries currently implemented:
  
- SELECT queries
- UNION, OPTIONAL clauses
- NAMED GRAPH identifiers
- LIMIT, OFFSET
- ORDER BY clauses
- SPARQL 1.0 filters and builtin functions
- variable aliases
- variable aggregation: MAX, MIN, COUNT, AVG, SUM functions
- GROUP BY clauses
- DISTINCT query modifier
- CONSTRUCT queries
- ASK queries
- INSERT DATA queries
- DELETE DATA queries
- DELETE WHERE queries
- WITH/DELETE/INSERT/WHERE queries
- LOAD queries
- CREATE GRAPH clauses
- DROP DEFAULT/NAMED/ALL/GRAPH clauses
- CLEAR DEFAULT/NAMED/ALL/Graph clauses
- FILTER EXISTS / NOT EXISTS operators

##Installation

To use the library in a node.js application, there is available a [package](http://search.npmjs.org/#/rdfstore) that can be installed using the NPM package manager:

    $npm install rdfstore

The library can be used as a persistent RDF store using MongoDB as the backend. An instance of MongoDB must be running in order to use this version of the store.

It is also possible to use rdfstore-js in a web application being executed in a browser. There is [minimized version](https://raw.github.com/antoniogarrote/rdfstore-js/master/dist/browser/rdf_store_min.js) of the library in a single Javascript file that can be linked from a HTML document. There is also a [minimized and gunzipped version](https://raw.github.com/antoniogarrote/rdfstore-js/master/dist/browser/rdf_store_min.js) available. Both versions have been compiled using Google's Closure Javascript compiler.
The persistent versions can be found [here (min)](https://raw.github.com/antoniogarrote/rdfstore-js/master/dist/browser_persistent/rdf_store_min.js).


##Building

The library can be built for different environments using the included Ruby script and configuration file. The JSON 1.5 Ruby gem is required by the build script.

To build the library for node.js execute the following command from the root directory of the project:

    $./make.rb nodejs

To build the library for the browser configuration, execute the following command:

    $./make.rb browser

To build the library for the browser, including support for persistent
storage execute this command:

    $./make.rb browser_persistent

The output of each configuration will be created in the dist subdirectory at the root path of the project.


## Tests

To execute the whole test suite of the library, including the DAWG test cases for SPARQL 1.0 and the test cases for SPARQL 1.1 implemented at the moment, the build script can be used:

    $./make.rb tests

The tests depend on [nodeunit](http://search.npmjs.org/#/nodeunit). That node.js library must be installed in order to run the tests.

You can also run the tests on the minimized version of the library with the command:

    $./make.rb test_min

Additionally, there are some smoke tests for both browser versions that can be found ithe 'browsertests' directory.
These tests are now also available online at these adresses:

- [non persistent version](http://antoniogarrote.github.com/rdfstore-js/browsertests/non_persistent/index.html)
- [persistent version](http://antoniogarrote.github.com/rdfstore-js/browsertests/persistent/index.html)

## Stand-alone SPARQL end-point

The Node.js version of the store can be used as a stand-alone SPARQL end-point. In the library distribution there is an executable script that can be used in UNIX platforms to invoke the store as an application.

    $npm install rdfstore
    $./node_modules/rdfstore/bin/rdfstorejs webserver --store-name test --store-engine mongodb

The previous shell command starts the execution of an instance of the store that uses a persistent instance of MongoDB as the backend and can accept HTTP SPARQL protocol requests.

The *rdfstorejs* script can also be used to some administrative tasks. For example it can be used to load RDF data into a graph in the store:

    $./bin/rdfstorejs load http://dbpedia.org/resource/Tim_Berners-Lee http://test.com/graph1 --store-name test --store-engine mongodb

When dealing with a remote resource, the store will perform automatically content negotiation. When loading data from a local file or standard input, the media type must be passed to the store with the *--media-type* flag.

The previous command loads the graph for a DBPedia article in the store graph passed as second argument. This graph can be retrieved using a HTTP requests according to the SPARQL RDF Protocol:

    $./node_modules/rdfstore/bin/rdfstorejs webserver --store-name test --store-engine mongodb &
    $ curl -v -d "default-graph-uri=http://test.com/graph1" --data-urlencode "query=select * { ?s ?p ?o } limit 3" -H "Accept: application/rdf+xml" http://localhost:8080/sparql

    * About to connect() to localhost port 8080 (#0)
    *   Trying ::1... Connection refused
    *   Trying fe80::1... Connection refused
    *   Trying 127.0.0.1... connected
    * Connected to localhost (127.0.0.1) port 8080 (#0)
    > POST /sparql HTTP/1.1
    > User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
    > Host: localhost:8080
    > Accept: application/rdf+xml
    > Content-Length: 104
    > Content-Type: application/x-www-form-urlencoded
    > 
    < HTTP/1.1 200 OK
    < Content-Type: application/sparql-results+xml
    < Access-Control-Allow-Origin: *
    < Access-Control-Allow-Methods: POST, GET, OPTIONS
    < Access-Control-Allow-Headers: Content-Type, Depth, User-Agent, X-File-Size, X-Requested-With, If-Modified-Since, X-File-Name, Cache-Control
    < Connection: keep-alive
    < Transfer-Encoding: chunked
    < 
    * Connection #0 to host localhost left intact
    * Closing connection #0
    <?xml version="1.0" encoding="UTF-8"?><sparql xmlns="http://www.w3.org/2005/sparql-results#"><head><variable name="s"/>...</sparql>

The store supports these formats in the response of *CONSTRUCT* SPARQL queries: rdf/xml, turtle, json-ld. When responding to *SELECT* and *ASK* queries results can be retrieved in the normative rdf/xml serialization, but they can also be retrieved as JSON passing an application/json media type in the HTTP Accept header.

Data can be removed from an instance of the store using a persistent backend with the *clear* command:

    $./bin/rdfstorejs load --store-name test --store-engine mongodb

Several aspects of the server execution can be configured passing arguments to the *rdfstorejs* script. A list of these flags, as well as a list of the available commands can be obtained invoking the script without arguments:

    $./bin/rdfstorejs

    Usage: rdfstorejs Command [Args] [Options]

    Commands:
    * webserver: starts the HTTP frontend for the store
    * load URI|stdin [dstGraphURI]: load the graph pointed by the URI argument into the store. The graph will be loaded in the 'dstGraphURI' graph or the default graph if none specified
    * clear: removes all data from the store<commit_msg>typo in README<commit_after>#rdfstore-js


rdfstore-js is a pure Javascript implementation of a RDF graph store with support for the SPARQL query and data manipulation language.

    var rdfstore = require('rdfstore');
    
    rdfstore.create(function(store) {
      store.execute('LOAD <http://dbpedia.org/resource/Tim_Berners-Lee> INTO GRAPH <http://example.org/people>', function() {

        store.setPrefix('dbp', 'http://dbpedia.org/resource/');
        
        store.node(store.rdf.resolve('dbp:Tim_Berners-Lee'), "http://example.org/people", function(success, graph) {

          var peopleGraph = graph.filter(store.rdf.filters.type(store.rdf.resolve("foaf:Person")));
          
          store.execute('PREFIX rdf:  <http://www.w3.org/1999/02/22-rdf-syntax-ns#>\
                         PREFIX foaf: <http://xmlns.com/foaf/0.1/>\
                         PREFIX : <http://example.org/>\
                         SELECT ?s FROM NAMED :people { GRAPH ?g { ?s rdf:type foaf:Person } }',
                         function(success, results) {

                           console.log(peopleGraph.toArray()[0].subject.valueOf() === results[0].s.value);

                         });

        });

      });
    })

rdfstore-js can be executed in a web browser or can be included as a library in a node.js application. It can also be executed as a stand-alone SPARQL end-point accepting SPARQL RDF Protocol HTTP requests.

The current implementation is far from complete but it already passes all the test cases for the SPARQL 1.0 query language and supports data manipulation operations from the SPARQL 1.1/Update version of the language.

Some other features included in the library are the following:

- SPARQL 1.0 support
- SPARQL 1.1/Update support
- Partial SPARQL 1.1 query support
- JSON-LD parser
- Turtle/N3 parser
- W3C RDF Interfaces API
- RDF graph events API
- Parallel execution where WebWorkers are available
- Persistent storage using HTML5 LocalStorage in the browser version
- Persistent storage using MongoDB in the Node.js version
- Node.js HTTP server implementating the [SPARQL Protocol for RDF](http://www.w3.org/TR/rdf-sparql-protocol/) recommendation

## Documentation

Documentation for the store can be found [here](http://antoniogarrote.github.com/rdfstore-js/doc/index.html).

## SPARQL support

rdfstore-js supports at the moment SPARQL 1.0 and most of SPARQL 1.1/Update.
Only some parts of SPARQL 1.1 query have been implemented yet.

This is a list of the different kind of queries currently implemented:
  
- SELECT queries
- UNION, OPTIONAL clauses
- NAMED GRAPH identifiers
- LIMIT, OFFSET
- ORDER BY clauses
- SPARQL 1.0 filters and builtin functions
- variable aliases
- variable aggregation: MAX, MIN, COUNT, AVG, SUM functions
- GROUP BY clauses
- DISTINCT query modifier
- CONSTRUCT queries
- ASK queries
- INSERT DATA queries
- DELETE DATA queries
- DELETE WHERE queries
- WITH/DELETE/INSERT/WHERE queries
- LOAD queries
- CREATE GRAPH clauses
- DROP DEFAULT/NAMED/ALL/GRAPH clauses
- CLEAR DEFAULT/NAMED/ALL/Graph clauses
- FILTER EXISTS / NOT EXISTS operators

##Installation

To use the library in a node.js application, there is available a [package](http://search.npmjs.org/#/rdfstore) that can be installed using the NPM package manager:

    $npm install rdfstore

The library can be used as a persistent RDF store using MongoDB as the backend. An instance of MongoDB must be running in order to use this version of the store.

It is also possible to use rdfstore-js in a web application being executed in a browser. There is [minimized version](https://raw.github.com/antoniogarrote/rdfstore-js/master/dist/browser/rdf_store_min.js) of the library in a single Javascript file that can be linked from a HTML document. There is also a [minimized and gunzipped version](https://raw.github.com/antoniogarrote/rdfstore-js/master/dist/browser/rdf_store_min.js) available. Both versions have been compiled using Google's Closure Javascript compiler.
The persistent versions can be found [here (min)](https://raw.github.com/antoniogarrote/rdfstore-js/master/dist/browser_persistent/rdf_store_min.js).


##Building

The library can be built for different environments using the included Ruby script and configuration file. The JSON 1.5 Ruby gem is required by the build script.

To build the library for node.js execute the following command from the root directory of the project:

    $./make.rb nodejs

To build the library for the browser configuration, execute the following command:

    $./make.rb browser

To build the library for the browser, including support for persistent
storage execute this command:

    $./make.rb browser_persistent

The output of each configuration will be created in the dist subdirectory at the root path of the project.


## Tests

To execute the whole test suite of the library, including the DAWG test cases for SPARQL 1.0 and the test cases for SPARQL 1.1 implemented at the moment, the build script can be used:

    $./make.rb tests

The tests depend on [nodeunit](http://search.npmjs.org/#/nodeunit). That node.js library must be installed in order to run the tests.

You can also run the tests on the minimized version of the library with the command:

    $./make.rb test_min

Additionally, there are some smoke tests for both browser versions that can be found ithe 'browsertests' directory.
These tests are now also available online at these adresses:

- [non persistent version](http://antoniogarrote.github.com/rdfstore-js/browsertests/non_persistent/index.html)
- [persistent version](http://antoniogarrote.github.com/rdfstore-js/browsertests/persistent/index.html)

## Stand-alone SPARQL end-point

The Node.js version of the store can be used as a stand-alone SPARQL end-point. In the library distribution there is an executable script that can be used in UNIX platforms to invoke the store as an application.

    $npm install rdfstore
    $./node_modules/rdfstore/bin/rdfstorejs webserver --store-name test --store-engine mongodb

The previous shell command starts the execution of an instance of the store that uses a persistent instance of MongoDB as the backend and can accept HTTP SPARQL protocol requests.

The *rdfstorejs* script can also be used to some administrative tasks. For example it can be used to load RDF data into a graph in the store:

    $./bin/rdfstorejs load http://dbpedia.org/resource/Tim_Berners-Lee http://test.com/graph1 --store-name test --store-engine mongodb

When dealing with a remote resource, the store will perform automatically content negotiation. When loading data from a local file or standard input, the media type must be passed to the store with the *--media-type* flag.

The previous command loads the graph for a DBPedia article in the store graph passed as second argument. This graph can be retrieved using a HTTP requests according to the SPARQL RDF Protocol:

    $./node_modules/rdfstore/bin/rdfstorejs webserver --store-name test --store-engine mongodb &
    $ curl -v -d "default-graph-uri=http://test.com/graph1" --data-urlencode "query=select * { ?s ?p ?o } limit 3" -H "Accept: application/rdf+xml" http://localhost:8080/sparql

    * About to connect() to localhost port 8080 (#0)
    *   Trying ::1... Connection refused
    *   Trying fe80::1... Connection refused
    *   Trying 127.0.0.1... connected
    * Connected to localhost (127.0.0.1) port 8080 (#0)
    > POST /sparql HTTP/1.1
    > User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
    > Host: localhost:8080
    > Accept: application/rdf+xml
    > Content-Length: 104
    > Content-Type: application/x-www-form-urlencoded
    > 
    < HTTP/1.1 200 OK
    < Content-Type: application/sparql-results+xml
    < Access-Control-Allow-Origin: *
    < Access-Control-Allow-Methods: POST, GET, OPTIONS
    < Access-Control-Allow-Headers: Content-Type, Depth, User-Agent, X-File-Size, X-Requested-With, If-Modified-Since, X-File-Name, Cache-Control
    < Connection: keep-alive
    < Transfer-Encoding: chunked
    < 
    * Connection #0 to host localhost left intact
    * Closing connection #0
    <?xml version="1.0" encoding="UTF-8"?><sparql xmlns="http://www.w3.org/2005/sparql-results#"><head><variable name="s"/>...</sparql>

The store supports these formats in the response of *CONSTRUCT* SPARQL queries: rdf/xml, turtle, json-ld. When responding to *SELECT* and *ASK* queries results can be retrieved in the normative rdf/xml serialization, but they can also be retrieved as JSON passing an application/json media type in the HTTP Accept header.

Data can be removed from an instance of the store using a persistent backend with the *clear* command:

    $./bin/rdfstorejs clear --store-name test --store-engine mongodb

Several aspects of the server execution can be configured passing arguments to the *rdfstorejs* script. A list of these flags, as well as a list of the available commands can be obtained invoking the script without arguments:

    $./bin/rdfstorejs

    Usage: rdfstorejs Command [Args] [Options]

    Commands:
    * webserver: starts the HTTP frontend for the store
    * load URI|stdin [dstGraphURI]: load the graph pointed by the URI argument into the store. The graph will be loaded in the 'dstGraphURI' graph or the default graph if none specified
    * clear: removes all data from the store
{'fraction_non_alphanumeric': 0.08557118499573743, 'fraction_numerical': 0.012254901960784314, 'mean_word_length': 3.519383578136287, 'pattern_counts': {'":': 0, '<': 39, '<?xml version=': 2, '>': 37, 'https://': 6, 'lorem ipsum': 0, 'www.': 8, 'xml': 16}, 'pii_count': 4, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1805695', 'n_tokens_mistral': 5621, 'n_tokens_neox': 5282, 'n_words': 2157}