<commit_before># Learning-to-navigate-without-a-map
The goal of the project is to determine how well deep learning is suited for planning under incomplete information.

## Resources for VIN

+ [Code from VIN authors](https://github.com/avivt/VIN)
+ [Code in TensorFlow](https://github.com/TheAbhiKumar/tensorflow-value-iteration-networks): maybe a reference code?
+ [Paper Notes from Karpathy](https://github.com/karpathy/paper-notes/blob/master/vin.md)

## Access data in HDF5 format

The Matlab format data is simply too slow to load.
Therefore I created some APIs to convert the raw data to
the HDF5 format and made them available in Google Drive

### Instructions to use the dataset

Before doing everything, make sure you've installed the package `h5py`,
for Anaconda user, you can simply do:

```
conda install h5py
```

1. Check you have the project resource folder at

```
$HOME/.rlvision
```

2. Copy HDF5 folder to following path

```
$HOME/.rlvision/data/
```

Note that `HDF5` folder should be a sub-folder of `data` folder

3. Check out the script `hdf5_data_test.py` in `tests` folder.
To run the script, simply type

```
make hdf5-data-test
```

You will see some lines of messages printed in the console,
and finally a plot for one grid.

4. In case you want to run the dstar algorithm with the dataset checkout `dstar_test.py` in `tests` folder
First, you will need to build the c++ source code, for that do the following:
* Move the dstar-lite directory
* Create a build directory:
```
mkdir build
```
* Run cmake:
```
cmake ..
```
* Compile:
```
make
```
* Now you can return to the root directory of this repo where the Makefile is.
```
cd ../..
```

Second, to run the dstar example script, simpy type
```
make dstar_test.py
```
<commit_msg>Add installation procedure for zeromq<commit_after># Learning-to-navigate-without-a-map
The goal of the project is to determine how well deep learning is suited for planning under incomplete information.

## Resources for VIN

+ [Code from VIN authors](https://github.com/avivt/VIN)
+ [Code in TensorFlow](https://github.com/TheAbhiKumar/tensorflow-value-iteration-networks): maybe a reference code?
+ [Paper Notes from Karpathy](https://github.com/karpathy/paper-notes/blob/master/vin.md)

## Access data in HDF5 format

The Matlab format data is simply too slow to load.
Therefore I created some APIs to convert the raw data to
the HDF5 format and made them available in Google Drive

### Instructions to use the dataset

Before doing everything, make sure you've installed the package `h5py`,
for Anaconda user, you can simply do:

```
conda install h5py
```

1. Check you have the project resource folder at

```
$HOME/.rlvision
```

2. Copy HDF5 folder to following path

```
$HOME/.rlvision/data/
```

Note that `HDF5` folder should be a sub-folder of `data` folder

3. Check out the script `hdf5_data_test.py` in `tests` folder.
To run the script, simply type

```
make hdf5-data-test
```

You will see some lines of messages printed in the console,
and finally a plot for one grid.

4. In case you want to run the dstar algorithm with the dataset checkout `dstar_test.py` in `tests` folder
First, you will need to build the c++ source code, for that do the following:
* Install zeromq (\ØMQ\:): (this is only for ubuntu users, for others, please refer to (zeromq)[http://zeromq.org/intro:get-the-software])
```
sudo apt-get install libzmq3-dev
```
(**NOTE**: Currently the CMakeLists.txt assumes that libzmq is at /usr/local/lib, if you have issues while linking please refer to google: search for things like "link zmq library" zmq stands for (zeromq)[http://zeromq.org/]
* Move to the dstar-lite directory
* Create a build directory:
```
mkdir build
```
* Run cmake:
```
cmake ..
```
* Compile:
```
make
```
* Now you can return to the root directory of this repo where the Makefile is.
```
cd ../..
```

Second, to run the dstar example script, simpy type
```
make dstar_test.py
```

{'fraction_non_alphanumeric': 0.10465116279069768, 'fraction_numerical': 0.006319514661274014, 'mean_word_length': 4.172549019607843, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4341537', 'n_tokens_mistral': 1269, 'n_tokens_neox': 1200, 'n_words': 533}