Intelligence agents more prone to irrational decision making than students
Bauer, Bond, Salt and their real-life counterparts have ample experience making tough choices between risky options. You'd think this would be a good thing but psychology research shows expertise can backfire when it encourages a short-hand, gist-based approach to problems, rather than a more detail-focused, calculating thinking style. According to researchers at Cornell University, this is what lies behind their demonstration that intelligence officers are more prone to irrationality than students in choosing between risky options.
Valerie Reyna and her colleagues presented 63 undergrads, 54 college-educated adults and 36 intelligence officers (77 per cent were special agents; 7 per cent were officers; and 16 per cent were admin) with dozens of decision-making scenarios related to saving human lives. For example: Imagine the U.S. is preparing for the outbreak of an unusual disease, which is expected to kill 600 people. Two options were then presented and the participants were asked to decide between them as they would in real life. The options were presented in a way that either emphasised lives to be saved:
e.g. Please indicate which option you prefer: a) 200 people saved for sure or (b) 1/3 probability 600 people saved and 2/3 probability no one saved?
Or worded in a way to emphasise lives to be lost:
i.e. Please indicate which option you prefer: a) 400 die for sure or (b) 2/3 probability 600 people die and 1/3 probability no one dies?
The idea is that probable outcomes are matched for options (a) and (b) across the two presentations of the choice, and yet the "framing" of the options affects how people choose. The key finding is that intelligence officers were more swayed by the framing of the problems, and they were also more confident in their choices. The researchers say this is a sign of irrational decision making because the agents were more prone to "treating equivalent outcomes differently based on superficial wording." In short, "they were more willing to take risks with human lives when outcomes were framed as losses rather than as gains."
College educated adults' susceptibility to framing was mid-way between the students' and the agents'. Overall the results are consistent with "developmental reversal" - the finding that children are less prone to framing effects than adults. By this account, intelligence agents are located further along the developmental trajectory than typical adults and so even more prone to framing effects.
"Taken together," Reyna and her colleagues said their results "suggest that meaning and context play a larger role in risky decision-making as experts gain experience, which enhances global performance but also has predictable pitfalls."
Valerie F. Reyna, Christina F. Chick, Jonathan C. Corbin, & Andrew N. Hsia (2013). Developmental Reversals in Risky Decision-Making: Intelligence Agents Show Larger Decision Biases than College Students. Psychological Science (In Press).
Psychology categories: Decision making
Could it be that more experienced people are actually rational, but not making the same assumptions as the less experienced?
I blog at djmarsay.wordpress.com .
This isn't very clear. Presumably "taking risks" is identified with option b in both cases, even though b will produce, on average, the same number of deaths/saved as a. So I wouldn't say opting for b in either case is a bad thing - it just shows a greater risk appetite, not irrationality. Or perhaps the "irrationality" is that intelligence officers show the biggest variation between their answers to the first set of questions (framed in terms of lives saved) and the second (lives lost). Well, perhaps that's the way they are written. When I see "200 die for sure" the natural interpretation is to add "and perhaps more" - indeed, there's nothing in the question preventing that interpretation. Ditto with 200 saved for sure - could be more. So they aren't really describing identical probabilities of death/saving at all, even to "rational" readers.
Don Reba said...
When we are told that 200 will survive for sure, we are not told that the rest will die. Taken at face value, it means that at least 200 will survive. So the expected outcome is not the same as for the probabilistic setup or for when we are told that 400 will die.
In addition, maximum gain/loss matters and has a value all on its own. If it didn't, there would not be such things as insurance or lottery.
So, the choices are not matched, and could justifiably make rational different choices in different setups.
It could simply be that students are naturally closer to an academic environment and therefore on alert for such logical and cognitive "trick" questions, whereas actual professionals are farther removed from such environments and therefore simply do not realize that this is a "trick" question. My point is that this question does not parallel a real life situation; it parallels a question on an exam.
Questions 1 and 2 are first of all not the same, since it says for sure x, but doesnt say anything regarding the prob for the rest. Also, everything else being equal you should go for less volatility.
The irrationality here is not whether people choose a or b for the question they face, but whether, statistically, members of the group makes different choices depending on whether they are shown question 1 or 2.
Foobarista said...
Yes, it's a standard "word problem", where there's no room for external interpretations. The wording is such that the choice is binary: either "no effect" or "dead".
Pandemics don't work that way "in real life", and experienced types who are in this sort of environment know this, while students recognize this as a bunch of meaningless filler text that indicates a "pattern-match the formula and solve" word problem.
Could it be that the intelligence agents to a higher extent was extroverts (as a group), compared to the students? Compare Susan Cain's book "Quiet". For instance: Extroverts, in general, focuses more on quick decisions than necessarily the right decision (So Susan says, don't blame me for saying it :o)
Dave Marsay said...
The paper claims that the wording should not affect the choice. But in my blog I give some examples as to why it might.
The paper would be correct if one made various assumptions. But is it irrational not to make them, as the paper seems to suggest?
But who says that they are not giving the 'right' decision? The paper says that they are not, but is it rational to take their word for it? (Or Kahneman et al's?) It is instructive to try to prove the paper's claim: I think that you will need to introduce some questionable assumptions.
It is a logic question. They should not be making any such assumptions.
Josef Hook said...
The amount of comments disputing the question are quite ironic. let me translate for you...
A: Definitely save some people
B: Maybe save everyone, but probably everyone dies
The point is no matter how the options are worded this is what they mean, and if you can't see that, you are yourself falling victim to the framing.