<commit_before>resources/papers/why-does-the-neocortex-have-layers-and-columns,resources/papers/a-theory-of-how-columns-in-the-neocortex-enable-learning-the-structure-of-the-world
company/events/2017/04/25/earth-pixels,company/events
company/newsletters/year-in-review-numenta-2015.html,company/newsletter/2016/01/07/year-in-review-numenta-2015
company/newsletters/announcing-bami-biological-and-machine-intelligence.html,company/newsletter/2016/05/11/announcing-bami-biological-and-machine-intelligence
company/newsletters/numenta-releases-htm-studio-desktop-application.html,press/2016/06/26/numenta-releases-htm-studio
assets/pdf/apps/de,applications
%23applications,applications
for-developers/education/ProblemsThatFitHTMs.pdf,resources/biological-and-machine-intelligence
machine-intelligence-technology.html,machine-intelligence-technology
about-numenta/numenta-technology.php,machine-intelligence-technology
machine-intelligence-technology.php,machine-intelligence-technology
machine-intelligence-technology-overview,machine-intelligence-technology
about-numenta/technology/pictures-demo.php,machine-intelligence-technology
install/java/osx,https://numenta.com
about-numenta/customers.php,partners
assets/pdf/biological-and-machine-intelligence/0.4/BaMI-Encoders.pdf,resources/biological-and-machine-intelligence
assets/pdf/biological-and-machine-intelligence/0.4/BaMI-HTM-Overview.pdf,resources/biological-and-machine-intelligence
assets/pdf/biological-and-machine-intelligence/0.4/BaMI-Problem-Sets.pdf,resources/biological-and-machine-intelligence
assets/pdf/biological-and-machine-intelligence/0.4/BaMI-SDR.pdf,resources/biological-and-machine-intelligence
assets/pdf/biological-and-machine-intelligence/0.4/BaMI-Temporal-Memory.pdf,resources/biological-and-machine-intelligence/temporal-memory-algorithm
assets%E3%80%80/pdf/biological-and-machineintelligence/BAMI-Complete.pdf,resources/biological-and-machine-intelligence
assets/pdf/biological-and-machine-,resources/biological-and-machine-intelligence
assets/pdf/whitepa-,resources/papers-videos-and-more
founders.php,company/careers-and-team
company/newsletter/index.php,company/newsletter
fordevelopers/education/DileepThesis.pdf,resources/papers-videos-and-more
resources/papers/encoding-data-for-htm-systems.html,https://arxiv.org/abs/1602.05925
resources/papers-videos-and-more/hierarchical-temporal-memory-white-paper.html,resources/papers-videos-and-more/hierarchical-temporal-memory-white-paper
resources/papers-videos-and-more/resources/layers-and-columns,resources/papers/a-theory-of-how-columns-in-the-neocortex-enable-learning-the-structure-of-the-world
resources/papers-videos-and-more/on-intelligence.html,resources/papers-videos-and-more/on-intelligence
resources/papers-videos-and-more/resources/on-intelligence,resources/papers-videos-and-more/on-intelligence
resources/papers-videos-and-more/sparse-distributed-representations.html,resources/papers-videos-and-more/sparse-distributed-representations
resources/papers-videos-and-more/principles-of-hierarchical-temporal-memory.html,resources/papers-videos-and-more/htm-videos-from-jeff-hawkins
resources/papers-videos-and-more/hierarchical-memorywhite-paper.html,resources/papers-videos-and-more/hierarchical-temporal-memory-white-paper
press/2015/05/22/numenta-and-ibm-to-build-biologically-inspired-intelligent-machines,press/releases
press/2015/04/14/numenta-the-bruce-lee-of-ai-and-deep-learning,press/releases
company/events/2017/12/14/boston-university,company/events/2017/12/13/boston-university
company/events/2017/06/18/dac,company/events
about-numenta/people.php,company/careers-and-team
company/careers-and-team/director-of-marketing.html,company/careers-and-team
archives/education/heli_static.swf,https://numenta.com
archives/software/vision/VisionToolkitTutorial.pdf,https://github.com/numenta/nupic.vision
blog/winner-of-htm-contest-at-yale.html,blog/2015/07/01/winner-of-htm-contest-at-yale
blog/and-the-award-goes-to.html,blog/2016/01/14/and-the-award-goes-to
blog/2017/10/19/todays-weak-ai,blog/2017/10/25/todays-weak-ai
blog/an-insiders-look-interview-with-yang-lan-and-jeff-hawkins.html,blog/2016/08/18/an-insiders-look-interview-with-yang-lan-and-jeff-hawkins
blog/can-neuroscientists-understand-the-brain.html,blog/2016/06/16/can-neuroscientists-understand-the-brain
blog/detecting-anomalies-in-stock-volumes.html,blog/2014/07/15/detecting-anomalies-in-stock-volumes
blog/how-is-groks-algorithm-different.html,blog/2012/06/04/how-is-groks-algorithm-different
blog/increasing-research-transparency.html,blog/2014/09/17/increasing-research-transparency
blog/index.rss,blog
blog/introducing-the-numenta-htm-challenge.html,blog/2015/09/29/introducing-the-numenta-htm-challenge
blog/jeff-hawkins-on-skynet-vs-c3po.html,blog/2012/12/13/jeff-hawkins-on-skynet-vs-c3po
blog/learning-through-active-exploration.html,blog/2014/10/07/learning-through-active-exploration
blog/machine-intelligence-machine-learning-deep-learning-artificial-intelligence.html,blog/2016/01/11/machine-intelligence-machine-learning-deep-learning-artificial-intelligence
blog/machine-intelligence-machine-learning-deep-learningartificial-intelligence.html,blog/2016/01/11/machine-intelligence-machine-learning-deep-learning-artificial-intelligence
blog/nab-a-benchmark-for-streaming-anomaly-detection.html,blog/2015/06/02/nab-a-benchmark-for-streaming-anomaly-detection
blog/nnachine-intelligence-nnachine-learnine-deep-learnine-,blog/2016/01/11/machine-intelligence-machine-learning-deep-learning-artificial-intelligence
blog/not-your-fathers-neural-network.html,blog/2013/02/19/not-your-fathers-neural-network
blog/numenta-anomaly-benchmark-contest-ieee-wcci-2016.html,blog/2016/02/11/numenta-anomaly-benchmark-contest-ieee-wcci-2016
blog/numenta-anomaly-benchmark-nab-competition-2016-winners.html,blog/2016/08/10/numenta-anomaly-benchmark-nab-competition-2016-winners
blog/numenta-at-computational-and-systems-neuroscience-conference.html,blog/2016/03/30/numenta-at-computational-and-systems-neuroscience-conference
blog/numenta-licensing-update-agpl.html,blog/2015/08/15/numenta-licensing-update-agpl
blog/progress-at-numenta.html,blog/2014/08/18/progress-at-numenta
blog/revisiting-a-1986-essay-on-brain-theory.html,blog/2016/08/04/revisiting-a-1986-essay-on-brain-theory
blog/summer-internships-and-extensibility-of-htm.html,blog/2014/08/11/summer-internships-and-extensibility-of-htm
blog/tag/Company.html,blog
blog/revisiting-a-1986-essay-on-brain-theory.html,blog/2016/08/04/revisiting-a-1986-essay-on-brain-theory
blog/the-neuroscience-behind-grok.html,blog/2013/01/03/the-neuroscience-behind-grok
blog/wait-the-brain-is-a-bloom-filter.html,blog/2012/10/07/wait-the-brain-is-a-bloom-filter
blog/winner-of-htm-contest-at-yale.html,blog/2015/07/01/winner-of-htm-contest-at-yale
blog/machine-intelligence-machine-learning-deep-learning-artificialintelligence.html,blog/2016/01/11/machine-intelligence-machine-learning-deep-learning-artificial-intelligence
blog/science-of-anomaly-detection.html,blog/2014/03/20/science-of-anomaly-detection
blog/real-time-insights-from-a-random-walk-htm-for-stocks-hits-the-iphone.html,blog/2016/02/18/real-time-insights-from-a-random-walk-htm-for-stocks-hits-the-iphone
blog/introducing-sparse-football-pool-ii-super-bowl-xlix.html,blog/2013/01/27/introducing-the-first-ever-sparse-football-pool
blog/big-datas-digital-nervous-system.html,blog/2012/09/06/big-datas-digital-nervous-system
blog/2016/01/11/machine-intelligence-machine-learning-deep-learning-artificial-intelligence/f78,blog/2016/01/11/machine-intelligence-machine-learning-deep-learning-artificial-intelligence
blog/brain-activity-map-project-arrives.html,blog/2013/03/04/brain-activity-map-project-arrives
blog/grok-for-managing-aws-cost-anomalies.html,blog/2014/08/29/grok-for-managing-aws-cost-anomalies
blog/2/,blog
blog/jeff-hawkins-speaks-at-berkeley.html,blog/2012/10/01/jeff-hawkins-speaks-at-berkeley
blog/anomaly-of-the-week.html,blog/2014/04/04/anomaly-of-the-week
blog/htm-challenge-2015-results.html,blog/2015/12/01/htm-challenge-2015-results
blog/introducing-the-first-ever-sparse-football-pool.html,blog/2013/01/27/introducing-the-first-ever-sparse-football-pool
blog/jeff-hawkins-at-strange-loop.html,blog/2012/10/22/jeff-hawkins-at-strange-loop
blog/numenta-and-complex-event-processing.html,blog/2013/04/02/numenta-and-complex-event-processing
blog/sparse-football-pool-winner.html,blog/2013/01/29/sparse-football-pool-winner
blog/super-bowl-neuroscience-sparse-pool-ii-2015-results.html,blog/2015/02/04/super-bowl-neuroscience-sparse-pool-ii-2015-results
blog/the-neuroscience-behind-grok-part-2.html,blog/2013/05/01/the-neuroscience-behind-grok-part-2
blog/the-online-learning-advantage.html,blog/2013/04/11/the-online-learning-advantage
blog/the-problem-of-representation.html,blog/2012/08/27/the-problem-of-representation
blog/4/index.html,blog
blog/machine-intelligence-machine-learning-deep-learning-artificial-,blog/2016/01/11/machine-intelligence-machine-learning-deep-learning-artificial-intelligence
blog/2016/01/11/machine-intelligence-machine-learning-deep-learning-artificial-intelligencezEX,blog/2016/01/11/machine-intelligence-machine-learning-deep-learning-artificial-intelligence
/blog/2016/01/11/machine-intelligence-machine-learning-deep-learning-artificial-intelligence,blog/2016/01/11/machine-intelligence-machine-learning-deep-learning-artificial-intelligence
/blog/2014/09/17/increasing-research-transparency,blog/2014/09/17/increasing-research-transparency<commit_msg>Fixed 2 more redirects<commit_after>install/java/osx,https://numenta.com
blog/2/,blog
resources/papers/why-does-the-neocortex-have-layers-and-columns,resources/papers/a-theory-of-how-columns-in-the-neocortex-enable-learning-the-structure-of-the-world
company/events/2017/04/25/earth-pixels,company/events
company/newsletters/year-in-review-numenta-2015.html,company/newsletter/2016/01/07/year-in-review-numenta-2015
company/newsletters/announcing-bami-biological-and-machine-intelligence.html,company/newsletter/2016/05/11/announcing-bami-biological-and-machine-intelligence
company/newsletters/numenta-releases-htm-studio-desktop-application.html,press/2016/06/26/numenta-releases-htm-studio
assets/pdf/apps/de,applications
%23applications,applications
for-developers/education/ProblemsThatFitHTMs.pdf,resources/biological-and-machine-intelligence
machine-intelligence-technology.html,machine-intelligence-technology
about-numenta/numenta-technology.php,machine-intelligence-technology
machine-intelligence-technology.php,machine-intelligence-technology
machine-intelligence-technology-overview,machine-intelligence-technology
about-numenta/technology/pictures-demo.php,machine-intelligence-technology
about-numenta/customers.php,partners
assets/pdf/biological-and-machine-intelligence/0.4/BaMI-Encoders.pdf,resources/biological-and-machine-intelligence
assets/pdf/biological-and-machine-intelligence/0.4/BaMI-HTM-Overview.pdf,resources/biological-and-machine-intelligence
assets/pdf/biological-and-machine-intelligence/0.4/BaMI-Problem-Sets.pdf,resources/biological-and-machine-intelligence
assets/pdf/biological-and-machine-intelligence/0.4/BaMI-SDR.pdf,resources/biological-and-machine-intelligence
assets/pdf/biological-and-machine-intelligence/0.4/BaMI-Temporal-Memory.pdf,resources/biological-and-machine-intelligence/temporal-memory-algorithm
assets%E3%80%80/pdf/biological-and-machineintelligence/BAMI-Complete.pdf,resources/biological-and-machine-intelligence
assets/pdf/biological-and-machine-,resources/biological-and-machine-intelligence
assets/pdf/whitepa-,resources/papers-videos-and-more
founders.php,company/careers-and-team
company/newsletter/index.php,company/newsletter
fordevelopers/education/DileepThesis.pdf,resources/papers-videos-and-more
resources/papers/encoding-data-for-htm-systems.html,https://arxiv.org/abs/1602.05925
resources/papers-videos-and-more/hierarchical-temporal-memory-white-paper.html,resources/papers-videos-and-more/hierarchical-temporal-memory-white-paper
resources/papers-videos-and-more/resources/layers-and-columns,resources/papers/a-theory-of-how-columns-in-the-neocortex-enable-learning-the-structure-of-the-world
resources/papers-videos-and-more/on-intelligence.html,resources/papers-videos-and-more/on-intelligence
resources/papers-videos-and-more/resources/on-intelligence,resources/papers-videos-and-more/on-intelligence
resources/papers-videos-and-more/sparse-distributed-representations.html,resources/papers-videos-and-more/sparse-distributed-representations
resources/papers-videos-and-more/principles-of-hierarchical-temporal-memory.html,resources/papers-videos-and-more/htm-videos-from-jeff-hawkins
resources/papers-videos-and-more/hierarchical-memorywhite-paper.html,resources/papers-videos-and-more/hierarchical-temporal-memory-white-paper
press/2015/05/22/numenta-and-ibm-to-build-biologically-inspired-intelligent-machines,press/releases
press/2015/04/14/numenta-the-bruce-lee-of-ai-and-deep-learning,press/releases
company/events/2017/12/14/boston-university,company/events/2017/12/13/boston-university
company/events/2017/06/18/dac,company/events
about-numenta/people.php,company/careers-and-team
company/careers-and-team/director-of-marketing.html,company/careers-and-team
archives/education/heli_static.swf,https://numenta.com
archives/software/vision/VisionToolkitTutorial.pdf,https://github.com/numenta/nupic.vision
blog/winner-of-htm-contest-at-yale.html,blog/2015/07/01/winner-of-htm-contest-at-yale
blog/and-the-award-goes-to.html,blog/2016/01/14/and-the-award-goes-to
blog/2017/10/19/todays-weak-ai,blog/2017/10/25/todays-weak-ai
blog/an-insiders-look-interview-with-yang-lan-and-jeff-hawkins.html,blog/2016/08/18/an-insiders-look-interview-with-yang-lan-and-jeff-hawkins
blog/can-neuroscientists-understand-the-brain.html,blog/2016/06/16/can-neuroscientists-understand-the-brain
blog/detecting-anomalies-in-stock-volumes.html,blog/2014/07/15/detecting-anomalies-in-stock-volumes
blog/how-is-groks-algorithm-different.html,blog/2012/06/04/how-is-groks-algorithm-different
blog/increasing-research-transparency.html,blog/2014/09/17/increasing-research-transparency
blog/index.rss,blog
blog/introducing-the-numenta-htm-challenge.html,blog/2015/09/29/introducing-the-numenta-htm-challenge
blog/jeff-hawkins-on-skynet-vs-c3po.html,blog/2012/12/13/jeff-hawkins-on-skynet-vs-c3po
blog/learning-through-active-exploration.html,blog/2014/10/07/learning-through-active-exploration
blog/machine-intelligence-machine-learning-deep-learning-artificial-intelligence.html,blog/2016/01/11/machine-intelligence-machine-learning-deep-learning-artificial-intelligence
blog/machine-intelligence-machine-learning-deep-learningartificial-intelligence.html,blog/2016/01/11/machine-intelligence-machine-learning-deep-learning-artificial-intelligence
blog/nab-a-benchmark-for-streaming-anomaly-detection.html,blog/2015/06/02/nab-a-benchmark-for-streaming-anomaly-detection
blog/nnachine-intelligence-nnachine-learnine-deep-learnine-,blog/2016/01/11/machine-intelligence-machine-learning-deep-learning-artificial-intelligence
blog/not-your-fathers-neural-network.html,blog/2013/02/19/not-your-fathers-neural-network
blog/numenta-anomaly-benchmark-contest-ieee-wcci-2016.html,blog/2016/02/11/numenta-anomaly-benchmark-contest-ieee-wcci-2016
blog/numenta-anomaly-benchmark-nab-competition-2016-winners.html,blog/2016/08/10/numenta-anomaly-benchmark-nab-competition-2016-winners
blog/numenta-at-computational-and-systems-neuroscience-conference.html,blog/2016/03/30/numenta-at-computational-and-systems-neuroscience-conference
blog/numenta-licensing-update-agpl.html,blog/2015/08/15/numenta-licensing-update-agpl
blog/progress-at-numenta.html,blog/2014/08/18/progress-at-numenta
blog/revisiting-a-1986-essay-on-brain-theory.html,blog/2016/08/04/revisiting-a-1986-essay-on-brain-theory
blog/summer-internships-and-extensibility-of-htm.html,blog/2014/08/11/summer-internships-and-extensibility-of-htm
blog/tag/Company.html,blog
blog/revisiting-a-1986-essay-on-brain-theory.html,blog/2016/08/04/revisiting-a-1986-essay-on-brain-theory
blog/the-neuroscience-behind-grok.html,blog/2013/01/03/the-neuroscience-behind-grok
blog/wait-the-brain-is-a-bloom-filter.html,blog/2012/10/07/wait-the-brain-is-a-bloom-filter
blog/winner-of-htm-contest-at-yale.html,blog/2015/07/01/winner-of-htm-contest-at-yale
blog/machine-intelligence-machine-learning-deep-learning-artificialintelligence.html,blog/2016/01/11/machine-intelligence-machine-learning-deep-learning-artificial-intelligence
blog/science-of-anomaly-detection.html,blog/2014/03/20/science-of-anomaly-detection
blog/real-time-insights-from-a-random-walk-htm-for-stocks-hits-the-iphone.html,blog/2016/02/18/real-time-insights-from-a-random-walk-htm-for-stocks-hits-the-iphone
blog/introducing-sparse-football-pool-ii-super-bowl-xlix.html,blog/2013/01/27/introducing-the-first-ever-sparse-football-pool
blog/big-datas-digital-nervous-system.html,blog/2012/09/06/big-datas-digital-nervous-system
blog/2016/01/11/machine-intelligence-machine-learning-deep-learning-artificial-intelligence/f78,blog/2016/01/11/machine-intelligence-machine-learning-deep-learning-artificial-intelligence
blog/brain-activity-map-project-arrives.html,blog/2013/03/04/brain-activity-map-project-arrives
blog/grok-for-managing-aws-cost-anomalies.html,blog/2014/08/29/grok-for-managing-aws-cost-anomalies
blog/jeff-hawkins-speaks-at-berkeley.html,blog/2012/10/01/jeff-hawkins-speaks-at-berkeley
blog/anomaly-of-the-week.html,blog/2014/04/04/anomaly-of-the-week
blog/htm-challenge-2015-results.html,blog/2015/12/01/htm-challenge-2015-results
blog/introducing-the-first-ever-sparse-football-pool.html,blog/2013/01/27/introducing-the-first-ever-sparse-football-pool
blog/jeff-hawkins-at-strange-loop.html,blog/2012/10/22/jeff-hawkins-at-strange-loop
blog/numenta-and-complex-event-processing.html,blog/2013/04/02/numenta-and-complex-event-processing
blog/sparse-football-pool-winner.html,blog/2013/01/29/sparse-football-pool-winner
blog/super-bowl-neuroscience-sparse-pool-ii-2015-results.html,blog/2015/02/04/super-bowl-neuroscience-sparse-pool-ii-2015-results
blog/the-neuroscience-behind-grok-part-2.html,blog/2013/05/01/the-neuroscience-behind-grok-part-2
blog/the-online-learning-advantage.html,blog/2013/04/11/the-online-learning-advantage
blog/the-problem-of-representation.html,blog/2012/08/27/the-problem-of-representation
blog/4/index.html,blog
blog/machine-intelligence-machine-learning-deep-learning-artificial-,blog/2016/01/11/machine-intelligence-machine-learning-deep-learning-artificial-intelligence
blog/2016/01/11/machine-intelligence-machine-learning-deep-learning-artificial-intelligencezEX,blog/2016/01/11/machine-intelligence-machine-learning-deep-learning-artificial-intelligence
/blog/2016/01/11/machine-intelligence-machine-learning-deep-learning-artificial-intelligence,blog/2016/01/11/machine-intelligence-machine-learning-deep-learning-artificial-intelligence
/blog/2014/09/17/increasing-research-transparency,blog/2014/09/17/increasing-research-transparency
{'fraction_non_alphanumeric': 0.1441073206518891, 'fraction_numerical': 0.06314520777655505, 'mean_word_length': 5.9904761904761905, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 8, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '947769', 'n_tokens_mistral': 7980, 'n_tokens_neox': 6688, 'n_words': 194}