<commit_before>"""
******************
IAMB STRUCTURE
LEARNING ALGORITHM
******************

"""

import numpy as np
from pyBN.utils.independence_tests import are_independent
from pyBN.utils.orient_edges import orient_edges_MB

def iamb(data, alpha=0.05, feature_selection=None):
	"""
	IAMB Algorithm for learning the structure of a
	Discrete Bayesian Network from data.

	Arguments
	---------<commit_msg>structure learning<commit_after>"""
******************
IAMB STRUCTURE
LEARNING ALGORITHM
******************

The IAMB algorithm - Incremental Association Markov Blanket - proceeds
similar to the Grow-Shrink algorithm, where the growing phase adds
random variables to a given node's markov blanket, and the shrinking
phase performs a second pass of conditional independence tests to
eliminate any variables from the markov blanket for which the given
node is conditionally independent given any other node in the markov
blanket (hence doesn't belong in the markov blanket).

It significantly reduces the complexity compared to PC/GS, and is
normally used in conjuction with the mutual information test for
independence (I do so here) [2].

Lastly, it can also be used as a feature selection algorithm for
classification or general dimensionality reduction - simply return
the markov blanket of a given node, which has been shown by <NAME> to be theoretically optimal for prediction [3]. 

IAMB for Feature Selection (from [1]):
	"A principled solution to the feature selection problem is
	to determine a subset of attributes that can "shield" (render
	independent) the attribute of interest from the effect of
	the remaining attributes in the domain. Koller and Sahami
	[4] first showed that the Markov blanket of a given target attribute
	is the theoretically optimal set of attributes to predict
	its value...
	
	Because the Markov blanket of a target attribute T renders
	it statistically independent from all the remaining attributes
	(see the Markov blanket definition below), all information
	that may influence its value is stored in the values
	of the attributes of its Markov blanket. Any attribute
	from the feature set outside its Markov blanket can be effectively
	ignored from the feature set without adversely affecting
	the performance of any classifier that predicts the
	value of T"

References
----------
[1] Yaramakala and Maragritis, "Speculative Markov Blanket 
Discovery for Optimal Feature Selection"

[2] Tsarmardinos, et al. "Algorithms for Large Scale 
Markov Blanket Discovery" 

[3] Koller "Toward Optimal Feature Selection."

"""

import numpy as np
from pyBN.utils.independence_tests import are_independent
from pyBN.utils.orient_edges import orient_edges_MB

def iamb(data, alpha=0.05, feature_selection=None):
	"""
	IAMB Algorithm for learning the structure of a
	Discrete Bayesian Network from data.

	Arguments
	---------
{'fraction_non_alphanumeric': 0.0741777466759972, 'fraction_numerical': 0.004548635409377187, 'mean_word_length': 4.822810590631365, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1780364', 'n_tokens_mistral': 766, 'n_tokens_neox': 710, 'n_words': 376}