<commit_before># deep-soli

Gesture Recognition Using Neural Networks with Google's Project Soli Sensor

## Update

I'm still preparing the dataset and trained model, please come back in one
or two weeks. Thanks! :-)

## Introduction

This is the open source evaluation code base of our paper:

**Interacting with Soli: Exploring Fine-Grained Dynamic Gesture Recognition
in the Radio-Frequency Spectrum** <br />
<NAME>, <NAME>, <NAME>, <NAME>, <NAME> <br />
(link to the [paper](http://bit.ly/2ftSRcn))

This project uses Google's [Project Soli](atap.google.com/soli) sensor.

> Soli is a new sensing technology that uses miniature radar to
> detect touchless gesture interactions.
>
> ![Soli sensor image](http://bit.ly/2fbwLYm)
>
> Soli sensor technology works by emitting electromagnetic waves in a
> broad beam. Objects within the beam scatter this energy, reflecting
> some portion back towards the radar antenna. Properties of the
> reflected signal, such as energy, time delay, and frequency shift
> capture rich information about the object’s characteristics and
> dynamics, including size, shape, orientation, material, distance,
> and velocity.

Our paper uses a light-weight end-to-end trained Convolutional Neural Networks
and Recurrent Neural Networks architecture, recognizes 11 in-air gestures
with 87% per-frame accuracy, and can perform realtime predictions at 140Hz
on commodity hardware. (link to the [paper video](http://bit.ly/2fDd9iJ))

## Pre-request

- Python 2: HDF5, OpenCV 2 interfaces for python.
- C++: HDF5, OpenCV 2, Boost
- Lua JIT and Torch 7.
- Torch 7 packages: `class`, GPU support `cunn` and `cutorch`, Matlab
  support `mattorch`, JSON support `lunajson`, Torch image library `image`
- Please note that `mattorch` is an outdated packages which is no
  longer maintained.

## Quick start

- Preprocessing (HDF5 to images)...
- Preprocessing (generate mean file)...
- Evaluate...

## Dataset

- Project Soli gesture sequence dataset: ...
- The gestures are listed in the table below. Each column represents
  one gesture and we snapshot three important steps for each gestures.
  The gesture label is indicated by the number in the circle above. Please
  notice that the gesture label order is different than the paper, as
  we regroup gestures in the paper.

![gesture](http://bit.ly/2fHcMRX)

## Pre-trained model

- Trained proposed model: ...

## Comments on the code base<commit_msg>Add data link<commit_after># deep-soli

Gesture Recognition Using Neural Networks with Google's Project Soli Sensor

## Update

Dataset and trained model available.

## Introduction

This is the open source evaluation code base of our paper:

**Interacting with Soli: Exploring Fine-Grained Dynamic Gesture Recognition
in the Radio-Frequency Spectrum** <br />
<NAME>, <NAME>, <NAME>, <NAME>, <NAME> <br />
(link to the [paper](http://bit.ly/2ftSRcn))

This project uses Google's [Project Soli](atap.google.com/soli) sensor.

> Soli is a new sensing technology that uses miniature radar to
> detect touchless gesture interactions.
>
> ![Soli sensor image](http://bit.ly/2fbwLYm)
>
> Soli sensor technology works by emitting electromagnetic waves in a
> broad beam. Objects within the beam scatter this energy, reflecting
> some portion back towards the radar antenna. Properties of the
> reflected signal, such as energy, time delay, and frequency shift
> capture rich information about the object’s characteristics and
> dynamics, including size, shape, orientation, material, distance,
> and velocity.

Our paper uses a light-weight end-to-end trained Convolutional Neural Networks
and Recurrent Neural Networks architecture, recognizes 11 in-air gestures
with 87% per-frame accuracy, and can perform realtime predictions at 140Hz
on commodity hardware. (link to the [paper video](http://bit.ly/2fDd9iJ))

## Pre-request

- Python 2: HDF5, OpenCV 2 interfaces for python.
- C++: HDF5, OpenCV 2, Boost
- Lua JIT and Torch 7.
- Torch 7 packages: `class`, GPU support `cunn` and `cutorch`, Matlab
  support `mattorch`, JSON support `lunajson`, Torch image library `image`
- Please note that `mattorch` is an outdated packages which is no
  longer maintained.

## Quick start

- Preprocessing (HDF5 to images)...
- Preprocessing (generate mean file)...
- Evaluate...

## Dataset

- [Download dataset](https://polybox.ethz.ch/index.php/s/wG93iTUdvRU8EaT)
- Project Soli gesture sequence dataset: ...
- The gestures are listed in the table below. Each column represents
  one gesture and we snapshot three important steps for each gestures.
  The gesture label is indicated by the number in the circle above. Please
  notice that the gesture label order is different than the paper, as
  we regroup gestures in the paper.

![gesture](http://bit.ly/2fHcMRX)

## Pre-trained model

- [Download model](https://polybox.ethz.ch/index.php/s/0SEdZqkn433dbEh)
- Trained proposed model: ...

## Comments on the code base
{'fraction_non_alphanumeric': 0.0896551724137931, 'fraction_numerical': 0.009533468559837728, 'mean_word_length': 4.353963083604778, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 41, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '391571', 'n_tokens_mistral': 1480, 'n_tokens_neox': 1376, 'n_words': 627}