Paradigm versus praxis Why psychology "absolute identification" experiments do not reveal sensory processes Lance Nizami Independent Research Scholar, Palo Alto, California, USA Abstract Purpose – A key cybernetics concept, information transmitted in a system, was quantified by Shannon. It quickly gained prominence, inspiring a version by Harvard psychologists Garner and Hake for "absolute identification" experiments. There, human subjects "categorize" sensory stimuli, affording "information transmitted" in perception. The Garner-Hake formulation has been in continuous use for 62 years, exerting enormous influence. But some experienced theorists and reviewers have criticized it as uninformative. They could not explain why, and were ignored. Here, the "why" is answered. The paper aims to discuss these issues. Design/methodology/approach – A key Shannon data-organizing tool is the confusion matrix. Its columns and rows are, respectively, labeled by "symbol sent" (event) and "symbol received" (outcome), such that matrix entries represent how often outcomes actually corresponded to events. Garner and Hake made their own version of the matrix, which deserves scrutiny, and is minutely examined here. Findings – The Garner-Hake confusion-matrix columns represent "stimulus categories", ranges of some physical stimulus attribute (usually intensity), and its rows represent "response categories" of the subject's identification of the attribute. The matrix entries thus show how often an identification empirically corresponds to an intensity, such that "outcomes" and "events" differ in kind (unlike Shannon's). Obtaining a true "information transmitted" therefore requires stimulus categorizations to be converted to hypothetical evoking stimuli, achievable (in principle) by relating categorization to sensation to intensity. But those relations are actually unknown, perhaps unknowable. Originality/value – The author achieves an important understanding: why "absolute identification" experiments do not illuminate sensory processes. Keywords Psychology, Information theory, Sensory, Shannon Paper type Research paper 1. Introduction Shannon (1948) derived the calculation of "information transmitted". It was reformulated by Garner and Hake (1951) for sensory psychology "absolute identification" experiments. What emerged was the "channel capacity", sensationalized as a fundamental cognitive capability in "The magical number seven, plus or minus two: some limits on our capacity for processing information" (Miller, 1956). Garner and Hake (1951) and Miller (1956) were remarkably influential (Nizami, 2010), and "channel capacity" calculations continue to this day (Lee et al., 2012). But theorists (Luce, 2003; Laming, cited therein) and reviewers (Gregory, 1980; Collins, 2007) have declared the effort fruitless, although lacking confidence regarding why. The present paper answers "why". It does not, however, aim to add clarity to those earlier criticisms; they had missed the points to be revealed here, as any interested reader can confirm, and their details are beyond present scope. The current issue and full text archive of this journal is available at www.emeraldinsight.com/0368-492X.htm The Editors, Drs Ranulph Glanville and Dai Griffiths, and the two anonymous reviewers contributed many insights, as did Prof. Claire S. Barnes PhD (VA Palo Alto HCS). Received 30 September 2012 Revised 18 February 2013 Accepted 7 October 2013 Kybernetes Vol. 42 No. 9/10, 2013 pp. 1447-1456 q Emerald Group Publishing Limited 0368-492X DOI 10.1108/K-09-2012-0059 Paradigm versus praxis 1447 2. The "general communication system", its information transmitted, and the "confusion matrix" Shannon (1948) proposed a model "general communication system". Figure 1 shows that system, which, to Shannon, consists of: . "an information source which produces a message or sequence of messages to be communicated to the receiving terminal"; . "a transmitter which operates on the message in some way to produce a signal suitable for transmission over the channel"; . "the channel is merely the medium used to transmit the signal from transmitter to receiver"; . "the receiver ordinarily performs the inverse operation of that done by the transmitter, reconstructing the message from the signal"; and . "the destination is the person (or thing) for whom the message is intended". To Shannon, n "events" are possible. The one that occurs is the "outcome", which is uncertain when n . 1. But if each event's probability of occurrence, pi (i 1⁄4 1, . . . , n) is known, then the "source information", called IS, is: source information ðsource uncertaintyÞ 1⁄4 2K Xn i1⁄41 pi log pi; K . 0; ð1Þ where Xn i1⁄41 pi 1⁄4 1: The base of the logarithms is a positive integer; 2 is used most often. Shannon set K 1⁄4 1. When symbols "k" are the events: IS 1⁄4 2 k X pðkÞ log pðkÞ: ð2Þ A symbol received is presumably from the set of symbols sent. But not all symbols are received as sent; unintended changes occur, i.e. "noisiness". Information transmitted, denoted It, can be calculated knowing: Figure 1. Shannon's "general communication system" Information flow Destination Information Source Received message Noise Received signal (outcome) Transmitter ReceiverImperfect channel Signal (event) Message + K 42,9/10 1448 . what symbols "k" were sent (events); . what symbols "j" were received (outcomes); and . for each "k" and "j", how often the two corresponded. The latter frequencies, denoted Njk, are arrayed in the "confusion matrix". Figure 2 shows the matrix. Note that p(k) 1⁄4 N.k/N 1⁄4 the probability that k was sent, and pj(k) 1⁄4 Njk/Nj. 1⁄4 the probability that k was sent if j was received. From those: ES 1⁄4 2 j X k X pjðkÞ log pjðkÞ is the stimulus uncertainty; H ; ð3Þ information transmitted I t 1⁄4 I S 2 ES 1⁄4 2 k X pðkÞ log pðkÞ þ j X k X pjðkÞ log pjðkÞ: ð4Þ Note that IS $ I t $ ES $ 0. When transmission is perfect, It 1⁄4 IS, the maximum It. 3. Sensory psychology: the confusion matrix of Garner and Hake (1951) In sensory psychology, "absolute identification" experiments involving human subjects preceded Garner and Hake (1951). The latter's unique contribution was to reformulate "absolute identification" so as to quantify "information transmitted" in perception, as follows. A set of sensory stimuli are varied in one attribute, typically intensity. Shannon "events" and "outcomes" were re-imagined as ranges called "stimulus categories" and "response categories". Stimulus intensity categories are represented by individual intensities, although "Which events (or how many) are represented by which particular discrete stimulus is an arbitrary matter" (Garner and Hake, p. 452). Stimuli evoke Figure 2. Shannon's "confusion matrix" -------N1 .N1n-N1k-N12N111 Sum = NN. n-N. k-N. 2N . 1Column totals Nn .Nnn-Nnk-Nn2Nn1n Nj .Njn-Njk-Nj2Nj1j -------N2 .N2n -N2k-N22N212 Symbol received (outcome) Row totalsn-k-21 Symbol sent (event) Paradigm versus praxis 1449 sensations, that rise with intensity. Response categories correspond to non-overlapping sensation ranges, named or numbered from low to high as "1-10" or "very mild to very strong", for example. In experiments, stimuli from a set representing the entire desired intensity range are presented, in randomized order, to the subject. The subject attends to the sensations, stating the response category that they believe is appropriate for each stimulus. How often each stimulus is identified with a particular response category is recorded in the confusion matrix. Figure 3 shows the matrix. To Garner and Hake (1951, p. 452), It expresses "the amount of information about the event continuum which a particular range of stimulus values can transmit". For logarithms of base X in equation (4), the number of identifiable stimulus categories is X It . 4. Problems with the Garner-Hake confusion matrix 4.1 "Noise" can be manipulated To Shannon, "outcomes" were from the set of "events", therefore being similar entities. But in absolute identification, stimulus intensities are the "events", and the responses to them are the "outcomes" – verbal, written, or electronically indicated. Such "outcomes" lack units, whereas stimulus intensity has units, involving mass, length and time (rare exceptions to this exist, which deserve acknowledgment but do not invalidate the present arguments – see Sakitt (1980), and her successors like Georgopoulos and Massey (1988), where "outcomes" were an actual subset of "events"). This difference in units, perhaps, motivated a classification that distracts from differences in units – the Garner-Hake replacement of Shannon's "events" and "outcomes" by "categories". Indeed, the numbers of stimulus and response categories were usually made identical. The appropriate number was deemed to be that which made the stimuli equally discriminable (independent of their (fixed) presentation probabilities p(k)), because it minimized the magnitudes of off-diagonal confusion-matrix entries and thereby improved It, whose highest empirical value thus became the only Figure 3. The sensory psychology "confusion matrix" -------N1 .N1n-N1k-N12N111 Sum = NN. n-N. k-N. 2N . 1Column totals Nn .Nnn-Nnk-Nn2Nn1n Nj .Njn-Njk-Nj2Nj1j -------N2 .N2n -N2k-N22N212 Response category Row totalsn-k-21 Stimulus category K 42,9/10 1450 meaningful one. Equal discriminability could be found by changing the numbers (and hence widths) of the stimulus and response categories, through an elaborate and time-consuming experiment mentioned in Garner and Hake (1951), which was not always employed. Such manipulation, according to equation (4), implies altering the pj(k), which in turn implies altering the system "noise". But noise, to Shannon, is independent of "events" (Figure 1). Thus, there are two significant problems here, namely, that It is meaningless in this context unless maximized, and that the maximization itself consists of adjusting the noise – which, to Shannon, is not adjustable. What kind of "noise", then, involves listener behavior? 4.2 "Noise" is memory variability, obscuring sensation variability Discriminability depends upon comparison – which depends upon memories of the stimuli being compared. Indeed, the "channel capacity" of absolute identification experiments is undoubtedly short-term memory capacity (Nizami, 2010). During each category judgment in the training sessions, the stimulus to be judged is compared to all of the stimuli; but afterwards, only the training memories of the gamut, not immediate memories, are available for comparison to the sensation evoked by a test stimulus. Memory is imperfect; category judgments are therefore distributed for each magnitude of the studied stimulus attribute, e.g. intensity (Nizami, 2011a). This point is crucial, but is undiscernable in channel capacity "reviews" (Broadbent, 1975; Baddeley, 1994; Shiffrin and Nosofsky, 1994), which have been narcissistic. Some trivial algebra now intrudes. Repeating a stimulus of intensity I empirically produces a distribution of the firing rate of voltage spikes in any of the first population of the chain of responding sensory neurons. Firing eventually leads to sensation, which, likewise, must be distributed, with a mean value, call it S(I), and a variance (no symbol needed). Sensation variability is true sensory noisiness. But any one sensation presumably evokes a distribution of category judgments, such that the variability of categorization exceeds (and hence obscures) that of sensation (Nizami, 2010). Figures 4 and 5 show the respective distributions. Information transmitted (equation (4)) depends upon outcomes and events having the same units – so that calculating It for absolute identifications depends upon finding the outcome-as-event (i.e. the intensity) which hypothetically would have evoked the sensation that would have evoked the recorded category judgment. However, the I in question is unknowable precisely because categorizations and sensations are distributed. 4.3 Transformation of outcome to mean event is impossible Can we at least infer the "average I" corresponding to an empirical category judgment? The first sensory neurons' firing rates are non-linear with stimulus intensity, i.e. more complicated than merely y 1⁄4 ax þ b (as any contemporary review paper or textbook will note). Therefore, S(I ) is inevitably non-linear. Let us imagine a continua of infinitely thin categories. On average, the subject assigns a mean category, call it f, to the mean sensation S(I ), through f(S ) – most likely non-linear, so that (on average) altogether category ("outcome") relates to stimulus intensity ("event") through a non-linearity f(S(I )) 1⁄4 F(I ). Presumably S(I ), f(S ), and hence F(I ) are continuous, monotonic, and smooth, hence meaningful as functions. Figure 6 shows two examples of the transformation of I to S(I ) to f(S(I )) 1⁄4 F(I ). Paradigm versus praxis 1451 Figure 4. A stimulus of intensity I0, when repeated, evokes a distribution of sensations, here assumed Gaussian with mean value S0, which has probability density p(S0), where probability density should be thought of as extending perpendicularly from the page 0.1 0 Probability density, p (S) Intensity, I Se ns at io n, S ( I) I0 S0 0.2 p (S0) Figure 5. The sensation S0 (Figure 4), when repeated, evokes a distribution of declared categories, here assumed Gaussian 0.1 0 Probability density, p (f(S)) Sensation, S C at eg or iz at io n, f ( S) S0 f (S0) 0.2 p (f(S0)) Note: Probability density should be thought of as extending perpendicularly from the page K 42,9/10 1452 F(I ) is typically unknown. Even if sufficient data were available to infer F(I) through curve-fitting, it could generate the wrong I. S(I ) and f(S ) are thus indispensable. Can we know them? S.S. Stevens of Harvard University successfully proselytized memory-dependent self-quantifications of sensations, called magnitude estimates. Stevens deemed them "reflexes" (Stevens, 1959), i.e. f(S ) 1⁄4 S. S(I) was allegedly a power function. Stevens' claims provoked hundreds of publications over the last half-century (worthy of separate review) which cumulatively expose magnitude estimation as a delusion. Altogether, S(I) and f(S) are unknown – and presently unknowable. Importantly, Figure 6 implies that with fMAX and 0, respectively, as hard upper and lower limits on mean category judgments, the category-judgment distributions will increasingly skew towards higher and lower categorizations, respectively, as categorization approaches those limits. Sensation distributions will skew upwards if SMAX is taken as a hard upper limit on mean sensation; they will skew downwards given zero as its hard lower limit. Nizami (2011a) predicted such skewing for category judgments, but neglected it for sensations. Skewing is seen empirically for categorizations (Nizami, 2011a). 5. Summary: how Garner and Hake (1951) fail Garner and Hake (1951) reformulated information theory (Shannon, 1948) for "absolute identification" experiments in sensory psychology. Their approach became an obsession, Figure 6. Two examples (upper dashes, and lower dashes) of stimulus intensity categorization Intensity, I 0 00 Category, fSensation, S S (I) SMAX fMAX f (S) Note: Two intensities (dots on the I axis) are (on average) transformed into sensations S(I) (dots on the S axis), which are transformed (on average) into categorizations f(S) (dots on the f axis); the non-linearity of S(I) and f(S) is indicated by the differences in slope between the respective pairs of dashed lines linking I to S, and S to f; presumably the highest and lowest mean sensations, respectively, SMAX and S(I = 0) = 0 (stimulus absence results in sensation absence), correspond to the highest and lowest mean category judgments, respectively, fMAX and f(S = 0) = 0 (sensation absence produces no nameable category, here "category 0") Paradigm versus praxis 1453 hence begging of scrutiny. Information theory itself provides "information transmitted" from data arrayed in the "confusion matrix", its columns labeled by symbol sent ("event") and its rows labeled by symbol received ("outcome"), "outcomes" being from among the "events". In contrast, Garner and Hake (1951) redefined "events" and "outcomes" as "stimulus categories" and "response categories", arbitrarily partitioned from respective stimulus continua and response ( 1⁄4 stimulus categorization) continua. To meaningfully compute information transmitted, then, categorizations must be converted to hypothetical stimulus intensities. But the "information transmitted" in absolute identification is redundant, in the first place, unless maximized by making the stimuli equally discriminable. Discriminability, regardless, always depends upon comparison – which depends upon the memories of the sensations of the stimuli being compared. Sensations are distributed, due to neurophysiology. The category judgment from any stimulus-evoked sensation is distributed too, because memory is imperfect. The variability of categorization exceeds (and hence obscures) that of sensation. Altogether, the stimulus intensity evoking any given categorization is unknowable. Even the transformations of mean categorization to mean sensation, and mean sensation to intensity, are unknown, and presently unknowable. In sum: absolute identifications do not lead to true "information transmitted". 6. Insight: why Garner and Hake (1951) fail The present paper is the fourth in a series, and should be placed in context. The background algebras here, and some of the relevant illustrations, have been repeated from paper to paper in order that each paper can be read independently. Each paper proves a different point. The first of the series, Nizami (2010), showed that absolute judgment involves numerous idiosyncracies that are not explainable within Shannon Information Theory, but which altogether suggest that the Garner-Hake measure of "information transmitted" and its alleged asymptote, the channel capacity, must perforce be measures of memory capacity. However, the mechanism whereby memory intrudes into absolute judgments was not identified. Many estimates of memory capacity were already available from memory experiments, so it was recommended that the Garner-Hake measure be abandoned. But entrenched methods are hard to remove. Further agitation was deemed necessary. Nizami (2011a) emphasized that Garner and Hake (1951) gave inconsistent descriptions of how absolute judgments represent information transmission, and that "channel capacity" was an artifact of sampling bias and wishful thinking. Nizami (2011a) then qualitatively (but not quantitatively) integrated memory into absolute judgments through a new, math-free model that predicted how absolute judgments change with the number of judged stimuli. In particular, the model predicted that the distributions of subjects' absolute judgments would become systematically skewed, confirmed by contemporary data. Altogether, then, Nizami (2010, 2011a) presented mounting evidence that the Garner-Hake measure was merely a needlessly convoluted memory measure. Nonetheless, that measure continues to be used (Lee et al., 2012). Such may have been encouraged by ongoing attempts to legitimize the Garner-Hake approach by incorporating its ideas into larger theories of perception. A sterling example is the "entropy theory of perception" of K.H. Norwich and co-authors, dating in print from 1975. In response, the discussion section of Nizami (2010) provided a review of earlier work altogether revealing the entropy theory to be deeply flawed. Nonetheless, it continues to be proselytized (Norwich, 2013). In a dedicated effort to finally unmask this false model, K 42,9/10 1454 the third paper in the series was written, Nizami (2011b). 35 years worth of entropy theory publications were painstakingly scrutinized for how they had interpreted the basis of Shannon's information theory, namely, Shannon's "general communication system". The entropy theory's interpretation proved to be utterly ambiguous. Importantly, Nizami (2011b) identified a core failure of Norwich et al.'s entropy theory: that Shannon's "general communication system" does not compute its own information. That is, an external observer is required, to faultlessly note the "events", their occurrence probabilities, and the "outcomes". But psychologists examining Shannon's system, shown in his "Figure 1", would find no observer (hence none in the present Figure 1). The observer did eventually appear in Shannon (1948, Figure 8), as part of a loop providing "correction data" within a "correction system"; nonetheless, Shannon showed no realization that an observer was already required. In the Garner-Hake measure, the perceivers are asked to quantify their own sensations, under constraint of memory, i.e. to be self-observing. But in perception "The observer, who is (presumed to be) the perceiver, is changed by their interaction with the stimulus. The stimulus is 'a difference that makes a difference'" (Nizami, 2011b, p. 1111). In contrast, Shannon's (hidden) observer is not changed, and only computes a statistic, bereft of any meaning absorbed by the perceiver. Glanville (2007) had already noted that Shannon had omitted the observer, who is necessarily "in the system and is taking part" (Glanville, p. 388). Glanville also noted hidden Shannon assumptions, namely, that the meaning of a message "is perfectly mapped onto" the message (Glanville, p. 377), and that "the meaning at one end is the same as that at the other" (Glanville, p. 377), i.e. that the recipient (at the "destination") must have the same understanding of the possible concepts in the message as the sender. Here we come to the "difference that makes a difference" between the present paper and its predecessors: neither Nizami (2010) nor Nizami (2011a) had explained precisely how the Garner-Hake measure becomes memory-limited. The present paper fills that gap, and also answers the crucial underlying question of why: namely, that Garner and Hake (1951), like Norwich et al.'s entropy theory, did not meaningfully identify the elements of the Shannon "general communication system". In particular, the "noise" of the Shannon system is not the "noise" of absolute identifications, an inconsistency that ultimately arises from a larger error, namely, a failure to realize the observer's presence and possible actions. This brings us to a "vanishing point", in that any point to using Shannon information theory in sensory psychology has vanished. Correcting the misuse of absolute identification experiments should lead to higher credibility for the academic community in society, and may lead to increased influence over public policy. References Baddeley, A. (1994), "The magical number seven: still magic after all these years?", Psychological Review, Vol. 101 No. 2, pp. 353-356. Broadbent, D.E. (1975), "The magic number seven after fifteen years", in Kennedy, A. and Wilkes, A. (Eds), Studies in Long-Term Memory, Wiley, New York, NY, pp. 3-18. Collins, A. (2007), "From H1⁄4log s n to conceptual framework: a short history of information", History of Psychology, Vol. 10 No. 1, pp. 44-72. Garner, W.R. and Hake, H.W. (1951), "The amount of information in absolute judgments", Psychological Review, Vol. 58 No. 6, pp. 445-459. Paradigm versus praxis 1455 Georgopoulos, A.P. and Massey, J.T. (1988), "Cognitive spatial-motor processes. 2: information transmitted by the direction of two-dimensional arm movements and by neuronal populations in primate motor cortex and area 5", Experimental Brain Research, Vol. 69 No. 2, pp. 315-326. Glanville, R. (2007), "Grounding difference", in Müller, A. and Müller, K.H. (Eds), An Unfinished Revolution?, Edition Echoraum, Vienna, pp. 361-406. Gregory, R.L. (1980), "Whatever happened to information theory?", Perception, Vol. 9 No. 5, pp. 489-491. Lee, K.-Y., Adams, R., Lee, H.-J. and Waddington, G. (2012), "Comparisons of indices of movement discrimination: psychometric function, information theory, and signal detection analysis", in Leth-Steensen, C. and Schoenherr, J.R. (Eds), Fechner Day 2012: Proceedings of the 28th Annual Meeting of the International Society for Psychophysics, Ottawa, ON, International Society for Psychophysics, pp. 24-29. Luce, R.D. (2003), "Whatever happened to information theory in psychology?", Reviews of General Psychology, Vol. 7 No. 2, pp. 183-188. Miller, G.A. (1956), "The magical number seven, plus or minus two: some limits on our capacity for processing information", Psychological Review, Vol. 63 No. 2, pp. 81-97. Nizami, L. (2010), "Interpretation of absolute judgments using information theory: channel capacity or memory capacity?", Cybernetics and Human Knowing, Vol. 17 Nos 1/2, pp. 111-155. Nizami, L. (2011a), "Memory model of information transmitted in absolute judgment", Kybernetes, Vol. 40 Nos 1/2, pp. 80-109. Nizami, L. (2011b), "Norwich's entropy theory: how not to go from abstract to actual", Kybernetes, Vol. 40 Nos 7/8, pp. 1102-1118. Norwich, K.H. (2013), "Sensation as reduction in uncertainty: a review", Cognitive Systems Research, Vol. 22-23, pp. 78-88. Sakitt, B. (1980), "Visual-motor efficiency (VME) and the information transmission in visual-motor tasks", Bulletin of the Psychonomic Society, Vol. 16 No. 5, pp. 329-332. Shannon, C.E. (1948), "A mathematical theory of communication", Bell System Technical Journal, Vol. 27 No. 1, pp. 623-656. Shiffrin, R.M. and Nosofsky, R.M. (1994), "Seven plus or minus two: a commentary on capacity limitations", Psychological Review, Vol. 101 No. 2, pp. 357-361. Stevens, S.S. (1959), "The quantification of sensation", Daedalus, Vol. 88 No. 4, pp. 606-621. Corresponding author Lance Nizami can be contacted at: nizamii2@att.net K 42,9/10 1456 To purchase reprints of this article please e-mail: reprints@emeraldinsight.com Or visit our web site for further details: www.emeraldinsight.com/reprints