<commit_before>rf-mono-matlab
==============

This repo contains Matlab code to calculate energy efficient receptive fields to encode a dataset of images.

This is my first Git repo so I am experimenting, and more info will appear at some point.

**Coming soon...**

* updates to code
* a python version, in a new repo.


**Instructions, copied from my original blog post [here](http://www.inferencelab.com/energy-efficient-receptive-field-code/)**

This minimal set of MATLAB functions will set up a simple neural network to learn receptive fields. These receptive fields minimise an energy function which involves a) image patch reconstruction error, b) sum of absolute firing rates, c) sum of absolute synaptic strengths. We haven’t done this explicitly, but this can be interpreted within a Bayesian framework where the constraints on synapses and firing rates represent some prior distribution over parameters.

Included is a sample of nearly 50,000 16×16 pixel image patches which were randomly sampled from the van Hateren image database (which doesn’t seem to be available online any more, but are possibly mirrored elsewhere). You can of course make your own set of image samples, and the code will work with whatever size image patches you give it. Although it assumes the image patches are square.

**Instructions**

* Download the MATLAB code and the image patch dataset and place in a folder.
* Set the MATLAB path to that folder
* Run the code in RUNME.m.

**Notes**

* I would recommend starting with costs on EITHER synapses OR firing rates. These parameters amount to a dimensionless constraint, so a value of zero means unconstrained.
* The image patches are 16×16, so there are a total of 256 input pixels. So if you want to examine under-complete codes, then set num_neurons to any value under 256. Likewise for over-complete codes set it to any value above 256.
* From our (me and <NAME>’s) research, if you want something that looks like biological reality, then synaptic costs seem to be relevant in the retinal (under-complete) and firing rates seem to be relevant in V1 (over-complete) codes.
* This code trains the network using gradient descent, which is not the fastest method, however the code is as minimal (and hopefully understandable) as possible. You will have to run for a lot of iterations before you approach the global minimum, and you might have to manually decrease net.lr when the receptive fields look like they have begun to converge. If the receptive fields are changing too much or are unstable, then also decreasenet.lr.

**Cite us**

If you use this code, please cite both these papers. Thanks!

<NAME>, <NAME>, <NAME>, <NAME>, (2005) Is the early visual system optimised to be energy efficient, Network: Computation in Neural Systems, 16(2/3): 175-190.

<NAME> & <NAME>, (2003) Synaptic energy efficiency in retinal processing, Vision Research 43:1283-1290.<commit_msg>embeded images into README.md<commit_after>rf-mono-matlab
==============

This repo contains Matlab code to calculate energy efficient receptive fields to encode a dataset of images.

This is my first Git repo so I am experimenting, and more info will appear at some point.

**Coming soon...**

* updates to code
* a python version, in a new repo.


**Instructions, copied from my original blog post [here](http://www.inferencelab.com/energy-efficient-receptive-field-code/)**

This minimal set of MATLAB functions will set up a simple neural network to learn receptive fields. These receptive fields minimise an energy function which involves a) image patch reconstruction error, b) sum of absolute firing rates, c) sum of absolute synaptic strengths. We haven’t done this explicitly, but this can be interpreted within a Bayesian framework where the constraints on synapses and firing rates represent some prior distribution over parameters.

Included is a sample of nearly 50,000 16×16 pixel image patches which were randomly sampled from the van Hateren image database (which doesn’t seem to be available online any more, but are possibly mirrored elsewhere). You can of course make your own set of image samples, and the code will work with whatever size image patches you give it. Although it assumes the image patches are square.

![](http://www.inferencelab.com/energy-efficient-receptive-field-code/rfplot/)
![](http://www.inferencelab.com/?attachment_id=125)

**Instructions**

* Download the MATLAB code and the image patch dataset and place in a folder.
* Set the MATLAB path to that folder
* Run the code in RUNME.m.

**Notes**

* I would recommend starting with costs on EITHER synapses OR firing rates. These parameters amount to a dimensionless constraint, so a value of zero means unconstrained.
* The image patches are 16×16, so there are a total of 256 input pixels. So if you want to examine under-complete codes, then set num_neurons to any value under 256. Likewise for over-complete codes set it to any value above 256.
* From our (me and <NAME>’s) research, if you want something that looks like biological reality, then synaptic costs seem to be relevant in the retinal (under-complete) and firing rates seem to be relevant in V1 (over-complete) codes.
* This code trains the network using gradient descent, which is not the fastest method, however the code is as minimal (and hopefully understandable) as possible. You will have to run for a lot of iterations before you approach the global minimum, and you might have to manually decrease net.lr when the receptive fields look like they have begun to converge. If the receptive fields are changing too much or are unstable, then also decreasenet.lr.

**Cite us**

If you use this code, please cite both these papers. Thanks!

<NAME>, <NAME>, <NAME>, <NAME>, (2005) Is the early visual system optimised to be energy efficient, Network: Computation in Neural Systems, 16(2/3): 175-190.

<NAME> & <NAME>, (2003) Synaptic energy efficiency in retinal processing, Vision Research 43:1283-1290.
{'fraction_non_alphanumeric': 0.06168505516549649, 'fraction_numerical': 0.017552657973921765, 'mean_word_length': 4.612570356472795, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2288749', 'n_tokens_mistral': 1600, 'n_tokens_neox': 1423, 'n_words': 898}