<commit_before># Shallow and Deep Convolutional Networks for Saliency Prediction

|  ![CVPR 2016 logo][logo-cvpr] | Paper accepted at [2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)](http://cvpr2016.thecvf.com/)   |
|:-:|---|

[logo-cvpr]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/logos/cvpr2016.jpg "CVPR 2016 logo"

| ![Junting Pan][JuntingPan-photo]  | ![<NAME>][KevinMcGuinness-photo]  | ![<NAME>][ElisaSayrol-photo]  | ![<NAME>][NoelOConnor-photo]  | ![<NAME>][XavierGiro-photo]  |
|:-:|:-:|:-:|:-:|:-:|
| Junting Pan (*)  | [<NAME>](KevinMcGuinness-web) (*)   |  [<NAME>](Elisa Sayrol-web) | [<NAME>](NoelOConnor-web)   | [<NAME>](XavierGiro-web)   |

(*) Equal contribution

[KevinMcGuinness-web]: https://www.insight-centre.org/users/kevin-mcguinness
[ElisaSayrol-web]: https://imatge.upc.edu/web/people/elisa-sayrol
[NoelOConnor-web]: https://www.insight-centre.org/users/noel-oconnor
[XavierGiro-web]: https://imatge.upc.edu/web/people/xavier-giro

[JuntingPan-photo]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/authors/JuntingPan.jpg "Junting Pan"
[KevinMcGuinness-photo]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/authors/KevinMcGuinness.jpg "<NAME>"
[ElisaSayrol-photo]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/authors/ElisaSayrol.jpg "<NAME>"
[NoelOConnor-photo]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/authors/NoelOConnor.jpg "<NAME>"
[XavierGiro-photo]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/authors/XavierGiro.jpg "<NAME>"

A joint collaboration of:

| ![logo-insight] | ![logo-dcu] | ![logo-upc] | ![logo-etsetb] | ![logo-gpi] | 
|:-:|:-:|:-:|:-:|:-:|
| [Insight Centre for Data Analytics](insight-web) | [Dublin City University (DCU)](dcu-web)  |[Universitat Politecnica de Catalunya (UPC)](upc-web)   | [UPC ETSETB TelecomBCN](etsetb-web)  | [UPC Image Processing Group](gpi-web) | 

[insight-web]: https://www.insight-centre.org/ 
[dcu-web]: http://www.dcu.ie/
[upc-web]: http://www.upc.edu/?set_language=en 
[etsetb-web]: https://www.etsetb.upc.edu/en/ 
[gpi-web]: https://imatge.upc.edu/web/ 


[logo-insight]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/logos/insight.jpg "Insight Centre for Data Analytics"
[logo-dcu]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/logos/dcu.png "Dublin City University"
[logo-upc]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/logos/upc.jpg "Universitat Politecnica de Catalunya"
[logo-etsetb]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/logos/etsetb.png "ETSETB TelecomBCN"
[logo-gpi]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/logos/gpi.png "UPC Image Processing Group"

#### Models

The two convnets presented in our work can be downloaded from the links provided below each respective figure:

| Shallow ConvNet (aka JuntingNet)  |  Deep ConvNet (aka SalNet) |
|:-:|:-:|
|  ![shallow-fig] | ![deep-fig]  |
| [[Lasagne Model (2.5 GB)]](shallow-model)  | [[Caffe Model (99 MB)]](deep-model) [[Caffe Prototxt]](deep-prototxt)  |

[shallow-fig]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/figs/shallow.png "Shallow convnet architecture"
[deep-fig]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/figs/deep.png "Deep convnet architecture"

[shallow-model]: https://imatge.upc.edu/web/sites/default/files/resources/1720/saliency/2016-cvpr/shallow_net.pickle
[deep-model]: https://imatge.upc.edu/web/sites/default/files/resources/1720/saliency/2016-cvpr/deep_net_model.caffemodel
[deep-prototxt]: https://imatge.upc.edu/web/sites/default/files/resources/1720/saliency/2016-cvpr/deep_net_deploy.prototxt


Our previous winning shallow models for the [LSUN Saliency Prediction Challenge 2015](http://lsun.cs.princeton.edu/#saliency) are described in [this preprint](https://imatge.upc.edu/web/publications/end-end-convolutional-network-saliency-prediction) and available from [this other site](https://imatge.upc.edu/web/resources/end-end-convolutional-networks-saliency-prediction-software). That work was also part of <NAME>'s bachelor thesis at [UPC TelecomBCN school](https://www.etsetb.upc.edu/en/) in June 2015, which report, slides and video are available [here](https://imatge.upc.edu/web/publications/visual-saliency-prediction-using-deep-learning-techniques).


### Datasets

#### Training
As explained in our paper, our networks were trained on the training and validation data provided by [SALICON](http://salicon.net/).

#### Test
Three different dataset were used for test:
* Test partition of [SALICON](http://salicon.net/) dataset.
* Test partition of [iSUN](http://vision.princeton.edu/projects/2014/iSUN/) dataset.
* [MIT300](http://saliency.mit.edu/datasets.html).

A collection of links to the SALICON and iSUN datasets is available from the [LSUN Challenge site](http://lsun.cs.princeton.edu/#saliency).

### Software frameworks

Our paper presents two different convolutional neural networks trained with different frameworks. For this reason, different instructions and source code folders are provided.

#### Shallow Network on Lasagne

The shallow network is implemented in [Lasagne](https://github.com/Lasagne/Lasagne), which at its time is developed over [Theano](http://deeplearning.net/software/theano/).
To install required version of Lasagne and all the remaining dependencies, you should run this [pip](https://pip.pypa.io/en/stable/) command.

```
pip install -r https://raw.githubusercontent.com/dnouri/kfkd-tutorial/master/requirements.txt
```

This requirements file was provided by [<NAME>](http://danielnouri.org/notes/2014/12/17/using-convolutional-neural-nets-to-detect-facial-keypoints-tutorial/).

#### Deep Network on Caffe

The deep network was developed over [Caffe](http://caffe.berkeleyvision.org/) by [Berkeley Vision and Learning Center (BVLC)](http://bvlc.eecs.berkeley.edu/). You will need to follow [these instructions](http://caffe.berkeleyvision.org/installation.html) to install Caffe.

### Acknowledgements

We would like to especially thank <NAME> and <NAME> from our technical support team at the Image Processing Group at the UPC.

| ![AlbertGil-photo]  | ![JosepPujal-photo]  |
|:-:|:-:|
| [<NAME>](AlbertGil-web)  |  [<NAME>](JosepPujal-web) |<commit_msg>Pip requirements file is linked to our own repo<commit_after># Shallow and Deep Convolutional Networks for Saliency Prediction

|  ![CVPR 2016 logo][logo-cvpr] | Paper accepted at [2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)](http://cvpr2016.thecvf.com/)   |
|:-:|---|

[logo-cvpr]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/logos/cvpr2016.jpg "CVPR 2016 logo"

| ![Junting Pan][JuntingPan-photo]  | ![<NAME>][KevinMcGuinness-photo]  | ![<NAME>][ElisaSayrol-photo]  | ![Noel O'Connor][NoelOConnor-photo]  | ![<NAME>][XavierGiro-photo]  |
|:-:|:-:|:-:|:-:|:-:|
| Junting Pan (*)  | [<NAME>](KevinMcGuinness-web) (*)   |  [<NAME>](Elisa Sayrol-web) | [<NAME>](NoelOConnor-web)   | [<NAME>](XavierGiro-web)   |

(*) Equal contribution

[KevinMcGuinness-web]: https://www.insight-centre.org/users/kevin-mcguinness
[ElisaSayrol-web]: https://imatge.upc.edu/web/people/elisa-sayrol
[NoelOConnor-web]: https://www.insight-centre.org/users/noel-oconnor
[XavierGiro-web]: https://imatge.upc.edu/web/people/xavier-giro

[JuntingPan-photo]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/authors/JuntingPan.jpg "<NAME>"
[KevinMcGuinness-photo]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/authors/KevinMcGuinness.jpg "<NAME>"
[ElisaSayrol-photo]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/authors/ElisaSayrol.jpg "<NAME>"
[NoelOConnor-photo]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/authors/NoelOConnor.jpg "<NAME>"
[XavierGiro-photo]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/authors/XavierGiro.jpg "<NAME>"

A joint collaboration of:

| ![logo-insight] | ![logo-dcu] | ![logo-upc] | ![logo-etsetb] | ![logo-gpi] | 
|:-:|:-:|:-:|:-:|:-:|
| [Insight Centre for Data Analytics](insight-web) | [Dublin City University (DCU)](dcu-web)  |[Universitat Politecnica de Catalunya (UPC)](upc-web)   | [UPC ETSETB TelecomBCN](etsetb-web)  | [UPC Image Processing Group](gpi-web) | 

[insight-web]: https://www.insight-centre.org/ 
[dcu-web]: http://www.dcu.ie/
[upc-web]: http://www.upc.edu/?set_language=en 
[etsetb-web]: https://www.etsetb.upc.edu/en/ 
[gpi-web]: https://imatge.upc.edu/web/ 


[logo-insight]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/logos/insight.jpg "Insight Centre for Data Analytics"
[logo-dcu]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/logos/dcu.png "Dublin City University"
[logo-upc]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/logos/upc.jpg "Universitat Politecnica de Catalunya"
[logo-etsetb]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/logos/etsetb.png "ETSETB TelecomBCN"
[logo-gpi]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/logos/gpi.png "UPC Image Processing Group"

#### Models

The two convnets presented in our work can be downloaded from the links provided below each respective figure:

| Shallow ConvNet (aka JuntingNet)  |  Deep ConvNet (aka SalNet) |
|:-:|:-:|
|  ![shallow-fig] | ![deep-fig]  |
| [[Lasagne Model (2.5 GB)]](shallow-model)  | [[Caffe Model (99 MB)]](deep-model) [[Caffe Prototxt]](deep-prototxt)  |

[shallow-fig]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/figs/shallow.png "Shallow convnet architecture"
[deep-fig]: https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/figs/deep.png "Deep convnet architecture"

[shallow-model]: https://imatge.upc.edu/web/sites/default/files/resources/1720/saliency/2016-cvpr/shallow_net.pickle
[deep-model]: https://imatge.upc.edu/web/sites/default/files/resources/1720/saliency/2016-cvpr/deep_net_model.caffemodel
[deep-prototxt]: https://imatge.upc.edu/web/sites/default/files/resources/1720/saliency/2016-cvpr/deep_net_deploy.prototxt


Our previous winning shallow models for the [LSUN Saliency Prediction Challenge 2015](http://lsun.cs.princeton.edu/#saliency) are described in [this preprint](https://imatge.upc.edu/web/publications/end-end-convolutional-network-saliency-prediction) and available from [this other site](https://imatge.upc.edu/web/resources/end-end-convolutional-networks-saliency-prediction-software). That work was also part of <NAME>'s bachelor thesis at [UPC TelecomBCN school](https://www.etsetb.upc.edu/en/) in June 2015, which report, slides and video are available [here](https://imatge.upc.edu/web/publications/visual-saliency-prediction-using-deep-learning-techniques).


### Datasets

#### Training
As explained in our paper, our networks were trained on the training and validation data provided by [SALICON](http://salicon.net/).

#### Test
Three different dataset were used for test:
* Test partition of [SALICON](http://salicon.net/) dataset.
* Test partition of [iSUN](http://vision.princeton.edu/projects/2014/iSUN/) dataset.
* [MIT300](http://saliency.mit.edu/datasets.html).

A collection of links to the SALICON and iSUN datasets is available from the [LSUN Challenge site](http://lsun.cs.princeton.edu/#saliency).

### Software frameworks

Our paper presents two different convolutional neural networks trained with different frameworks. For this reason, different instructions and source code folders are provided.

#### Shallow Network on Lasagne

The shallow network is implemented in [Lasagne](https://github.com/Lasagne/Lasagne), which at its time is developed over [Theano](http://deeplearning.net/software/theano/).
To install required version of Lasagne and all the remaining dependencies, you should run this [pip](https://pip.pypa.io/en/stable/) command.

```
pip install -r https://github.com/imatge-upc/saliency-2016-cvpr/blob/master/shallow/requirements.txt
```

This requirements file was provided by [<NAME>](http://danielnouri.org/notes/2014/12/17/using-convolutional-neural-nets-to-detect-facial-keypoints-tutorial/).

#### Deep Network on Caffe

The deep network was developed over [Caffe](http://caffe.berkeleyvision.org/) by [Berkeley Vision and Learning Center (BVLC)](http://bvlc.eecs.berkeley.edu/). You will need to follow [these instructions](http://caffe.berkeleyvision.org/installation.html) to install Caffe.

### Acknowledgements

We would like to especially thank <NAME> and <NAME> from our technical support team at the Image Processing Group at the UPC.

| ![AlbertGil-photo]  | ![JosepPujal-photo]  |
|:-:|:-:|
| [<NAME>](AlbertGil-web)  |  [<NAME>](JosepPujal-web) |
{'fraction_non_alphanumeric': 0.1664699866257572, 'fraction_numerical': 0.01966800409094485, 'mean_word_length': 4.9457436856875585, 'pattern_counts': {'":': 0, '<': 39, '<?xml version=': 0, '>': 39, 'https://': 60, 'lorem ipsum': 0, 'www.': 14, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2382686', 'n_tokens_mistral': 4903, 'n_tokens_neox': 4518, 'n_words': 852}