<commit_before># fcrepo-sample-dataset-tests

This repository contains tests that can be performed on a Fedora4 repository. 

## Import / Export tests

This directory contains tests that will verify and validate that the
fcrepo-import-export client can successfully import data to or export data from a
Fedora4 server.

### Resources: 
* [Fedora4](http://github.com/fcrepo4/fcrepo4)
* [Fedora4 Import Export Client](http://github.com/fcrepo4-labs/fcrepo-import-export)
* [Import Export Datasets](http://github.com/fcrepo4-labs/fcrepo-sample-dataset)

Many of these tests assume that you've either already exported or imported data from/to a
Fedora4 system.  To learn how to export or import data, please visit the 
Fedora4 Import Export Client page, linked to above.


Below are the tests and their specific usage.
---

## Verify

This script has been run in python 2.7.11, but should, in theory, work in python 3.

This tool has the following modes:
* `export` : it will verify everything in the specified export directory is in Fedora4
* `import` : it will verify that everything in Fedora4 is located on the file system in the export directory
* `both` : runs both export and import mode on the sources.  This is the most comprehensive way to check the sources, but it may be slow.


How to use this in testing:
* Load a data set from [Import Export Datasets](http://github.com/fcrepo4-labs/fcrepo-sample-dataset), or use your own.
* Export data from a Fedora4 system using the [Fedora4 Import Export Client](http://github.com/fcrepo4-labs/fcrepo-import-export)
* Configure this verification tool to point to both your Fedora4 server and the disk location where the files are.
* Run this tool and check the 'verification-report.txt' file that was created. It will report information on errors, if any are found.

To run the verify test, first edit the configuration file ```config.ini``` (details below) and
then run:

    $> python ./verify.py

This will create a report file that's at the location specified in the configuration file.
To include more detailed information (including all the records being looked at), change loglevel to INFO:

    $> python ./verify.py --loglevel INFO<commit_msg>Reverting change - n-triples requirement still required.<commit_after># fcrepo-sample-dataset-tests

This repository contains tests that can be performed on a Fedora4 repository. 

## Import / Export tests

This directory contains tests that will verify and validate that the
fcrepo-import-export client can successfully import data to or export data from a
Fedora4 server.

### Resources: 
* [Fedora4](http://github.com/fcrepo4/fcrepo4)
* [Fedora4 Import Export Client](http://github.com/fcrepo4-labs/fcrepo-import-export)
* [Import Export Datasets](http://github.com/fcrepo4-labs/fcrepo-sample-dataset)

Many of these tests assume that you've either already exported or imported data from/to a
Fedora4 system.  To learn how to export or import data, please visit the 
Fedora4 Import Export Client page, linked to above.


Below are the tests and their specific usage.
---

## Verify

This script has been run in python 2.7.11, but should, in theory, work in python 3.

This tool has the following modes:
* `export` : it will verify everything in the specified export directory is in Fedora4
* `import` : it will verify that everything in Fedora4 is located on the file system in the export directory
* `both` : runs both export and import mode on the sources.  This is the most comprehensive way to check the sources, but it may be slow.


How to use this in testing:
* Load a data set from [Import Export Datasets](http://github.com/fcrepo4-labs/fcrepo-sample-dataset), or use your own.
* Export data from a Fedora4 system in **n-triples** format, using the [Fedora4 Import Export Client](http://github.com/fcrepo4-labs/fcrepo-import-export)
* Configure this verification tool to point to both your Fedora4 server and the disk location where the files are.
* Run this tool and check the 'verification-report.txt' file that was created. It will report information on errors, if any are found. 

To run the verify test, first edit the configuration file ```config.ini``` (details below) and
then run:

    $> python ./verify.py

This will create a report file that's at the location specified in the configuration file.
To include more detailed information (including all the records being looked at), change loglevel to INFO:

    $> python ./verify.py --loglevel INFO
{'fraction_non_alphanumeric': 0.07403265488704988, 'fraction_numerical': 0.009841198836949228, 'mean_word_length': 4.254994124559342, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '846951', 'n_tokens_mistral': 1260, 'n_tokens_neox': 1214, 'n_words': 601}