An overview of reconstructability analysis <s> Model utilization <s> To achieve reduced training time and improved generalization with artificial neural networks (ANN, or NN), it is important to use a reduced complexity NN structure. A "problem" is defined by constraints among the variables describing it. If knowledge about these constraints could be obtained a priori, this could be used to reduce the complexity of the ANN before training it. Systems theory literature contains methods for determining and representing structural aspects of constrained data (these methods are herein called GSM, general systems method). The suggestion here is to use the GSM model of the given data as a pattern for modularizing a NN prior to training it. The present work assumes the GSM model for the given problem context has been determined (represented here in the form of Boolean functions of known decompositions). This means that certain information is available about constraint among the system variables, and is used to develop a modularized NN. The modularized NN and an equivalent general NN (full interconnect, feed-forward NN) are both trained on the same data. Various predictions are offered: 1) The general NN and the modularized NN will both learn the task, but the modularized NN will learn it faster. 2) If trained on an (appropriate) subset of possible inputs, the modularized NN will perform better generalization than the general NN. 3) If trained on a non-decomposable function of the same variables, the general NN will learn the task, but the modularized NN will not. All of these predictions are verified experimentally. Future work will explore more decomposition types and more general data types. <s> BIB001 </s> An overview of reconstructability analysis <s> Model utilization <s> This is a continuation of work reported by Lendaris at el. (1994) whose objective has been to develop a method that uses certain a priori information about a problem domain to pre-structure artificial neural networks (ANNs) into modules before training. The method is based on a general systems theory methodology, based on information-theoretic ideas, that generates structural information of the problem domain by analyzing I/O pairs from that domain. The notion of performance subset of an ANN structure is described. Extensive experiments on 5-input/1-output and 7-input/1-output Boolean mappings show that significantly improved generalization follows from successful pre-structuring. As the previous work already showed, such pre-structuring also yields improved training speed. <s> BIB002 </s> An overview of reconstructability analysis <s> Model utilization <s> We consider the problem of matching domain-specific statistical structure to neural-network (NN) architecture. In past work we have considered this problem in the function approximation context; here we consider the pattern classification context. General Systems Methodology tools for finding problem-domain structure suffer exponential scaling of computation with respect to the number of variables considered. Therefore we introduce the use of Extended Dependency Analysis (EDA), which scales only polynomially in the number of variables, for the desired analysis. Based on EDA, we demonstrate a number of NN pre-structuring techniques applicable for building neural classifiers. An example is provided in which EDA results in significant dimension reduction of the input space, as well as capability for direct design of an NN classifier. <s> BIB003 </s> An overview of reconstructability analysis <s> Model utilization <s> The building block hypothesis implies that genetic algorithm efficiency will be improved if sets of genes that improve fitness through epistatic interaction are near to one another on the chromosome. We demonstrate this effect with a simple problem, and show that information‐theoretic reconstructability analysis can be used to decide on optimal gene ordering. <s> BIB004
The 11-component model indicated above can be used in at least three ways. First, each predicting component, e.g., BHNRX or LORUX, represents a high ordinality interaction effect involving several inputs and the output (health status, X). If the inputs were originally quantitative variables, these effects are likely also to be nonlinear. Ideally, each of these components should be given a substantive interpretation of how the input variables combine to affect X. This requires extensive subject-specific knowledge, and has not yet been attempted for this study. Second, it is possible to asess the multiple components relative to one another, to identify which are the most important. For the 7 inputs used, this 11-component model was maximally predictive (under the requirement that the model would have been statistically significant if it had been proposed for confirmatory test), but a simpler though less predictive model might be preferred. Further analysis of the model might allow such simplification. Third, the eleven-component model yields a calculated BHLNORUX distribution, which can be converted to p(X | B,H,L,N,O,R,U), a distribution on X conditioned on the input variables. Thus, given the values of all the 7 input variables, RA yields predicted probabilities for all the different possible output states. These can be compared to p(X), the probabilities of the output without any knowledge of the inputs. Finally, it should be reiterated that exploratory searches are precisely that: exploratory. Resulting models should ideally be tested in a confirmatory mode with new data. The selected inputs and/or the specific multi-component model can be used in conjunction with methods which retain the quantitative character of variables which have been discretized. With SPC RA modeling, feature selection can be done from a large set of variables, as noted above, and the selected features fed into a neural net (NN). The NN part of such a RA-NN strategy may sometimes not be necessary. In the pattern-recognition study of BIB003 , RA on discretized variables essentially solved the problem by itself. Multi-predicting-component RA modeling can also be used to prestructure neural nets, so that all-to-all connectivity is not required BIB001   BIB002 ; this speeds NN training times and improves generalization capacity. RA may also have promise for the prestructuring of genetic algorithms, more specifically for determining the optimal order of the variables on the GA genome BIB004 .