<commit_before>---
layout: single
title: Current Research - Project SAHDE
permalink: /sds/research/densityEstimation/sahde/
author_profile: true
header:
  overlay_color: "#5e616c"
  overlay_image: /sds/research/densityEstimation/sahde/notes/MDE_20161010_141708_01.jpg
  caption: 
excerpt: 'for Scalable Mathematical Statistical Experiments.<br /><br /><br />'
---

## Project SAHDE:<br /> Scalable Adaptive Histogram Density Estimation 

This programme is partly supported by:

* [databricks academic partners program](https://academics.cloud.databricks.com) (for distributed cloud computing
* research time for this project was party due to:
	* 2009-2015 by industrial consulting revenues of <NAME>
	* 2015, 2016 by the project [CORCON: Correctness by Construction](http://corcon.net/about/), Seventh Framework Programme of the European Union, Marie Curie Actions-People, International Research Staff Exchange Scheme with counter-part funding by The Royal Society of New Zealand 

Project SAHDE is an effort to create a scalable version of the adaptive histogram density estimators implemented in:
* [MRS 2.0, a C++ class library for statistical set processing and computer-aided proofs in statistics](https://github.com/raazesh-sainudiin/mrs2).

based on mathematical statistical notions in:
* Data-adaptive histograms through statistical regular pavings, <NAME>, <NAME>, <NAME> and <NAME>, 2016 ([PDF](http://lamastex.org/preprints/20161121optMAPMDE.pdf) 1.8MB)


## LICENSE

Copyright 2016 <NAME> and <NAME>

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.


### Blackboard discussion notes at LaMaStEx on 2016-10-08.
 <commit_msg>research for histograms priming for 16TB of slowRAM in Spark Driver<commit_after>---
layout: single
title: Current Research - Project SAHDE
permalink: /sds/research/densityEstimation/sahde/
author_profile: true
header:
  overlay_color: "#5e616c"
  overlay_image: /sds/research/densityEstimation/sahde/notes/MDE_20161010_141708_01.jpg
  caption: 
excerpt: 'for Scalable Mathematical Statistical Experiments.<br /><br /><br />'
---

## Project SAHDE:<br /> Scalable Adaptive Histogram Density Estimation 

This programme is partly supported by:

* [databricks academic partners program](https://academics.cloud.databricks.com) for distributed cloud computing
* research time for this project was party due to:
	* 2009-2015 by industrial consulting revenues of <NAME>
	* 2015, 2016 by the project [CORCON: Correctness by Construction](http://corcon.net/about/), Seventh Framework Programme of the European Union, Marie Curie Actions-People, International Research Staff Exchange Scheme with counter-part funding by The Royal Society of New Zealand 

Project SAHDE is an effort to create a scalable version of the adaptive histogram density estimators implemented in:
* [MRS 2.0, a C++ class library for statistical set processing and computer-aided proofs in statistics](https://github.com/raazesh-sainudiin/mrs2).

based on mathematical statistical notions in:
* Data-adaptive histograms through statistical regular pavings, <NAME>, <NAME>, <NAME> and <NAME>, 2016 ([PDF](http://lamastex.org/preprints/20161121optMAPMDE.pdf) 1.8MB)


## LICENSE

Copyright 2016 <NAME> and <NAME>

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.


### Blackboard discussion notes at LaMaStEx on 2016-10-08.
 
{'fraction_non_alphanumeric': 0.08070175438596491, 'fraction_numerical': 0.03274853801169591, 'mean_word_length': 4.98041958041958, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2220528', 'n_tokens_mistral': 1328, 'n_tokens_neox': 1129, 'n_words': 492}