This was a fantastic overview! Extremely helpful for those coming to these debates for the first time (or even the second and third time). I agree wholeheartedly with the goal of "bridging" and finding points of commonality and integrating, rather than the "overwriting" Kahneman self-deprecatingly alluded to. I must say that I have always been puzzled by this kind of zero-sum thinking between research programs/theoretical paradigms, here and elsewhere. I mean, who really thinks one school has all the answers; why throw out insights from rival schools just because you think yours does better justice? This neo-Popperian falsification idea whereby we just throw out the old and bring in the new seems unrealistic. In my opinion there is a lot of historical amnesia (Russell Jacoby) and reinventing of the wheel - even Freud got a lot right no matter how much he got wrong.
I'm curious about the difference between metacognition, and "macrocognition" as you discuss it here. Is it simply metacognition in the NDM context? BTW, your anecdote about the firefighter and his "spidey sense" recalls a lot of crime thrillers where someone is able to sense something's off and gets spooked.
Kahneman definitely came to his own during a Popperian zeitgeist, but I wonder how Popperian he was. In the podcast I linked, he almost seems relativist. It's a very strange interview. He seemed even more pessimistic than his normal self, and it was his last interview before he passed, I believe.
Meanwhile, the modern zeitgeist seems to prioritize synthesis. Perhaps it's not surprising that we see bridge building as the goal rather than falsification.
Macrocognition is very different than metacognition. Macrocognition is about how cognitive processes chain together to create more complex cognitive processes. For example, a given decision can't be boiled down to a single heuristic, but has to be understood in relation to all the other cognitive processes. That's how you get something like Recognition Primed Decision Making (RPD) which can be understood as multiple heuristics working together.
When studying real world decision-making, understanding macrocognitive processes is generally much more important, and microcognition is more just something academics are interested in.
Also, if you are interested in the crime side, the book The Gift Of Fear is really great and talks about intuition in the context of recognizing threats.
Is it not obvious that someone choosing an investment fund will use a different model to a student choosing a college course, and both will use a different model to a pilot handling an emergency?
I agree. Likely all three will use similar heuristics to evaluate the situation and make a decision. But with many heuristics (like the availability and representativeness heuristics) experience matters a lot, and so despite using the same decision-making approach, we wouldn't expect all three to make equal quality of decisions.
Maybe this is a bit specific, but its been at the back of my mind long enough that its one of the major associations for HB for me: Every attempt to explain gambling addiction is *literally the opposite* of a Kahneman&Tversky result. Anti-risk-aversion, anti-loss-aversion, anti-scope-insensitivity. The people proposing these explanations never comment on the contrast. You seem like someone who might know more about this.
Its really unfortunate that the field summarizes Prospect Theory as being about "Loss Aversion." In reality, Kahneman and Tversky identify something called the "four fold pattern of risk" which predicts that people will be (relative to EUT) risk seeking when there's small probabilities of gain or large probabilities of loss, and risk averse when there small probabilities of loss or large probability of gain.
Most of the work pushing back against Loss Aversion is not questioning the central finding that people are relatively loss averse in certain contexts, but questioning when that is, and the cause of that loss aversion. For example, a recent paper (representing the cognitive turn in HB) that caused quite a stir argued that the four fold pattern is better explained by a general attenuation when people are overwhelmed by the complexity of the decision (as opposed to the Prospect Theory explanation that it is about preferences for/against types of risks).
It obviously depends on context. Heuristics are cheap and dirty tools that are amazing in situations of short timescales. When you have more time it can often make sense to apply full blown cost benefit analysis. By careful cost benefit analysis I have decided to decide most things just based on vibes. You could call this strategy 'metarationality' if you define rationality to mean the formal logical stuff. If you define rationality more sanely you'd find no contradiction between the different schools of thought. They all have different roles in decision making
Agreed. You have to know which tool to use, and at different times you will use all of the tools. But knowing when to use which tool (meta-rationality) is still a pretty controversial question. Also it's not always clear which standard to use. If you use a heuristic and end up choosing something very different than what you would have gone with had you done proper cost and benefit analysis, was it still the right decision to use the heuristic
Great overview. I'll come back to this as it's such a good balance between summary and detail. It strikes me that a key piece would be whether or how NDM might 'scale' into areas where experience might not exist or where fast feedback isnt realistic/ possible.
Also, a question: where do concepts like belief-based utility tie into these various models? During my masters, I grew quite fond of the idea that our beliefs are similar to possessions, and we struggle to simply discard them in decision making. It seems like a fusion across #1 and #2.
NDM scales into those area in that it might provide a better explanation of the failures that people make when they are not experienced, and provides a better theory on how to increase expertise. I have used NDM in this way to help an e-commerce app design their interface. But in another sense it doesn't scale because you can't develop expertise in every area, and so sometimes you should rely on classical decision models.
I'm not familiar with Belief Based Utility, but it sounds very much like something someone in HB might propose. (CDM wouldn't propose it because it is more focused on developing prescriptive theories than explaining psychology.)
I need to have a good few reads again of this. I am really interested in researching courage and risk (rather than fear)…… there’s so much in here to dig in to. Thank you.
You might be interested in the book "The Gift of Fear: And Other Survival Signals That Protect Us from Violence" by Gavin de Becker. It makes some interesting points about fear being a signal, but not something that needs to dominate your life. The author is not a researcher, but I was very impressed with how aligned he was with my understanding of the principles of NDM. He is also just a very engaging author, and I have considered sending a copy to all the females in my life. In part because all the females in my life are interested in true crime, but also in part because I think it is really useful information for how to avoid threats.
this was so incredibly interesting. it summed up and organized so many insights i had seen floating around in diff essays. i had no idea they were all part of diff schools of thoughts. and the nerd in me loved the list of resources at the end. the next phase of the great rationality debate is my new personal netflix show 🤣
i also wonder what heuristics / biases / cognitive processes there are that relate to emotional stickiness w beliefs. beliefs that we hold, in spite of conflicting evidence, bc it would be too destabilizing for us to let go of them. i imagine these beliefs would influence more long term decisions eg. life partners, careers, etc.
am i understanding correctly that this great debate seems centered around rapid fire decisions?
Disagreement about reflection and rapid fire decisions is, I think, a manifestation of a deeper disagreement about how orderly vs complex the world is.
If the world is orderly, then we can learn that order, quantify it, and proceduralize it. Both CDM and HB see the world as fairly orderly in this way. There is a belief we can abstract out rules and formalize decision-making in the same way Newton formalized the motion of planets.
Conversely, NDM and FF would argue that the world is complex and so those abstract rules will fail as soon as they are brought in contact with the real world. As a result, we need to instead rely on decision strategies that are context sensitive, and proved over evolutionary times scales to work well.
CDM and HB are both very rational and progressive, whereas FF and NDM are both very empirical and conservative. (I use those terms in their original philosophical meanings, not as a comment on their methodologies or politics)
Of course, this is a simplification. But I think this distinction has helped me to understand why people seem naturally drawn to one school vs. another, and to understand how its the continuation of debates that have been happening for centuries.
Dreyfus and Dreyfus (2005) capture this distinction, too. They argue against the rational and abstract world view, and instead argue for something more context sensitive.
"We must be prepared to abandon the traditional view that runs from Plato to Piaget and Chomsky that a beginner starts with specific cases and, as he or she becomes more proficient, abstracts and interiorizes more and more sophisticated rules. It might turn out that skill acquisition moves in just the opposite direction: from abstract rules to particular cases."
In my language I would summarize this as "We should abandon the view that good decision-making is about learning rational abstract rules that cover all cases, and instead recognize that good decision-making is about learning increasingly refined rules for increasingly specific situations."
ah okay, i read this response a few hours ago and stepped away to let my brain work on it in the background.
coming back to it now, this is my understanding of what ur saying:
- CDM and HB say u can go from specific situations to abstract rules (world is orderly)
- NDM and FF say u should go from abstract rules to specific situations (world is complex)
it follows to me now that NDM would like to analyze how individuals like firefighters make decisions based off their subconscious pattern recognition / intuition. seems very aligned w the idea that decision making is at its best when it’s adapting to specific contexts.
if i were to extrapolate further, are u suggesting that individuals who subscribe to the CDM and HB paradigm are more likely to struggle w emotional stickiness in their beliefs? because releasing a belief is often related to larger frameworks rather than a micro level decision. (aka abstraction vs specific context narrowing).
i imagine w this school of thought, there’s more ego investment in being right from the start bc they assume their beliefs will generalize well across time and space.
not sure if that’s what u were implying, if not, how would u clarify ur view on beliefs that get emotionally stuck and why that happens / which school of thought would best explain it?
Sorry for the confusion. I wasnt trying to imply anything about emotional stickiness. And I'm not sure exactly how to think about emotional stickiness.
I suppose I would guess NDM and FF supporters would be more likely to have some sort of emotional stickiness because they rely more on intuition and feeling, whereas HB and CDM supporters prefer to use abstract models. Or at least, that is how HB and CDM advocates would see themselves - too rational to get stuck like that. Whether that is actually true, I have no idea.
This is a very insightful overview of the respective lines of cognition research, thank you. It took me many years to begin to find a way to understand the overlaps between macrocognition, expertise, heuristics, ndm, prospect theory, and frugal tricks, and Baron’s active open-mindedness and start to see a role for reflective judgment as well (Keith Stanovich was very helpful there). I’m thrilled when I find others interested in this important quest especially when they’ve come to see it from different angles than mine. For my part I think the reflective dimension is particularly important and dispositions qualities like intellectual virtues and especially curiosity play a huge role in shaping our use of the heuristics and ndm. Looking forward to reading more of your work.
I am also fan of thinking in terms of intellectual virtues. But your particular view of decision-making impacts what you consider a virtue. For example "open-mindedness" is often considered an intellectual virtue, but sometimes this is taught as "don't jump to conclusions" which can be bad advice depending on how you interpret it.
Experts regularly jump to conclusions because without a solid theory of what is happening, you don't know what to look for. Conclusions guide your attention and actions. Jumping to a conclusion is a necessary step to figuring out what is going on. Even when your conclusion is wrong, you still need to have one so that you can test and probe the environment. Without some sort of conclusion, you can only sit there inert and passive.
This is a fabulous essay which I could feel changing the way I think about thinking (and particularly K&T). Thank you, Jared. And thanks for the 'further reading' list.
While spending time slowly digesting your terrific overview of the "rationality" debate, I kept thinking about two books that you didn't list (for good reasons as they their connection to this debate is little "slant" to use Emily Dickenson's term). These are Jonsen & Toulmin's The Abuse of Casuistry: A History of Moral Reasoning which is a terrific examination of the nature and need for case-based reasoning as well as its abuse. And Ziliak & McCloskey's The Cult of Statistical Significance: How the Standard Error Costs Us Jobs, Justice, and Lives
This was a fantastic overview! Extremely helpful for those coming to these debates for the first time (or even the second and third time). I agree wholeheartedly with the goal of "bridging" and finding points of commonality and integrating, rather than the "overwriting" Kahneman self-deprecatingly alluded to. I must say that I have always been puzzled by this kind of zero-sum thinking between research programs/theoretical paradigms, here and elsewhere. I mean, who really thinks one school has all the answers; why throw out insights from rival schools just because you think yours does better justice? This neo-Popperian falsification idea whereby we just throw out the old and bring in the new seems unrealistic. In my opinion there is a lot of historical amnesia (Russell Jacoby) and reinventing of the wheel - even Freud got a lot right no matter how much he got wrong.
I'm curious about the difference between metacognition, and "macrocognition" as you discuss it here. Is it simply metacognition in the NDM context? BTW, your anecdote about the firefighter and his "spidey sense" recalls a lot of crime thrillers where someone is able to sense something's off and gets spooked.
Kahneman definitely came to his own during a Popperian zeitgeist, but I wonder how Popperian he was. In the podcast I linked, he almost seems relativist. It's a very strange interview. He seemed even more pessimistic than his normal self, and it was his last interview before he passed, I believe.
Meanwhile, the modern zeitgeist seems to prioritize synthesis. Perhaps it's not surprising that we see bridge building as the goal rather than falsification.
Macrocognition is very different than metacognition. Macrocognition is about how cognitive processes chain together to create more complex cognitive processes. For example, a given decision can't be boiled down to a single heuristic, but has to be understood in relation to all the other cognitive processes. That's how you get something like Recognition Primed Decision Making (RPD) which can be understood as multiple heuristics working together.
When studying real world decision-making, understanding macrocognitive processes is generally much more important, and microcognition is more just something academics are interested in.
Also, if you are interested in the crime side, the book The Gift Of Fear is really great and talks about intuition in the context of recognizing threats.
Is it not obvious that someone choosing an investment fund will use a different model to a student choosing a college course, and both will use a different model to a pilot handling an emergency?
I agree. Likely all three will use similar heuristics to evaluate the situation and make a decision. But with many heuristics (like the availability and representativeness heuristics) experience matters a lot, and so despite using the same decision-making approach, we wouldn't expect all three to make equal quality of decisions.
Maybe this is a bit specific, but its been at the back of my mind long enough that its one of the major associations for HB for me: Every attempt to explain gambling addiction is *literally the opposite* of a Kahneman&Tversky result. Anti-risk-aversion, anti-loss-aversion, anti-scope-insensitivity. The people proposing these explanations never comment on the contrast. You seem like someone who might know more about this.
Its really unfortunate that the field summarizes Prospect Theory as being about "Loss Aversion." In reality, Kahneman and Tversky identify something called the "four fold pattern of risk" which predicts that people will be (relative to EUT) risk seeking when there's small probabilities of gain or large probabilities of loss, and risk averse when there small probabilities of loss or large probability of gain.
Most of the work pushing back against Loss Aversion is not questioning the central finding that people are relatively loss averse in certain contexts, but questioning when that is, and the cause of that loss aversion. For example, a recent paper (representing the cognitive turn in HB) that caused quite a stir argued that the four fold pattern is better explained by a general attenuation when people are overwhelmed by the complexity of the decision (as opposed to the Prospect Theory explanation that it is about preferences for/against types of risks).
Not commenting on the overall conversation—there was a recent paper critiquing opera (2024) which suggests it’s findings are due to methodological errors: https://scholar.google.com/scholar?start=10&hl=en&as_sdt=5,44&sciodt=0,44&cites=17756024044617881972&scipsc=#d=gs_qabs&t=1740899579434&u=%23p%3DxNM0Y9g5tAQJ
Also, I don’t think prospect theory really comments on addiction. It’s just about average behavior.
Thanks for sharing. I hadn't seen this follow-up
It obviously depends on context. Heuristics are cheap and dirty tools that are amazing in situations of short timescales. When you have more time it can often make sense to apply full blown cost benefit analysis. By careful cost benefit analysis I have decided to decide most things just based on vibes. You could call this strategy 'metarationality' if you define rationality to mean the formal logical stuff. If you define rationality more sanely you'd find no contradiction between the different schools of thought. They all have different roles in decision making
Agreed. You have to know which tool to use, and at different times you will use all of the tools. But knowing when to use which tool (meta-rationality) is still a pretty controversial question. Also it's not always clear which standard to use. If you use a heuristic and end up choosing something very different than what you would have gone with had you done proper cost and benefit analysis, was it still the right decision to use the heuristic
Yeah all of those decision theories are in a sense true
Great overview. I'll come back to this as it's such a good balance between summary and detail. It strikes me that a key piece would be whether or how NDM might 'scale' into areas where experience might not exist or where fast feedback isnt realistic/ possible.
Also, a question: where do concepts like belief-based utility tie into these various models? During my masters, I grew quite fond of the idea that our beliefs are similar to possessions, and we struggle to simply discard them in decision making. It seems like a fusion across #1 and #2.
NDM scales into those area in that it might provide a better explanation of the failures that people make when they are not experienced, and provides a better theory on how to increase expertise. I have used NDM in this way to help an e-commerce app design their interface. But in another sense it doesn't scale because you can't develop expertise in every area, and so sometimes you should rely on classical decision models.
I'm not familiar with Belief Based Utility, but it sounds very much like something someone in HB might propose. (CDM wouldn't propose it because it is more focused on developing prescriptive theories than explaining psychology.)
I need to have a good few reads again of this. I am really interested in researching courage and risk (rather than fear)…… there’s so much in here to dig in to. Thank you.
You might be interested in the book "The Gift of Fear: And Other Survival Signals That Protect Us from Violence" by Gavin de Becker. It makes some interesting points about fear being a signal, but not something that needs to dominate your life. The author is not a researcher, but I was very impressed with how aligned he was with my understanding of the principles of NDM. He is also just a very engaging author, and I have considered sending a copy to all the females in my life. In part because all the females in my life are interested in true crime, but also in part because I think it is really useful information for how to avoid threats.
I have heard of it but not read it. Courage and women’s safety is a research space all of its own.
this was so incredibly interesting. it summed up and organized so many insights i had seen floating around in diff essays. i had no idea they were all part of diff schools of thoughts. and the nerd in me loved the list of resources at the end. the next phase of the great rationality debate is my new personal netflix show 🤣
i also wonder what heuristics / biases / cognitive processes there are that relate to emotional stickiness w beliefs. beliefs that we hold, in spite of conflicting evidence, bc it would be too destabilizing for us to let go of them. i imagine these beliefs would influence more long term decisions eg. life partners, careers, etc.
am i understanding correctly that this great debate seems centered around rapid fire decisions?
Disagreement about reflection and rapid fire decisions is, I think, a manifestation of a deeper disagreement about how orderly vs complex the world is.
If the world is orderly, then we can learn that order, quantify it, and proceduralize it. Both CDM and HB see the world as fairly orderly in this way. There is a belief we can abstract out rules and formalize decision-making in the same way Newton formalized the motion of planets.
Conversely, NDM and FF would argue that the world is complex and so those abstract rules will fail as soon as they are brought in contact with the real world. As a result, we need to instead rely on decision strategies that are context sensitive, and proved over evolutionary times scales to work well.
CDM and HB are both very rational and progressive, whereas FF and NDM are both very empirical and conservative. (I use those terms in their original philosophical meanings, not as a comment on their methodologies or politics)
Of course, this is a simplification. But I think this distinction has helped me to understand why people seem naturally drawn to one school vs. another, and to understand how its the continuation of debates that have been happening for centuries.
Dreyfus and Dreyfus (2005) capture this distinction, too. They argue against the rational and abstract world view, and instead argue for something more context sensitive.
"We must be prepared to abandon the traditional view that runs from Plato to Piaget and Chomsky that a beginner starts with specific cases and, as he or she becomes more proficient, abstracts and interiorizes more and more sophisticated rules. It might turn out that skill acquisition moves in just the opposite direction: from abstract rules to particular cases."
In my language I would summarize this as "We should abandon the view that good decision-making is about learning rational abstract rules that cover all cases, and instead recognize that good decision-making is about learning increasingly refined rules for increasingly specific situations."
(Sorry for the length)
ah okay, i read this response a few hours ago and stepped away to let my brain work on it in the background.
coming back to it now, this is my understanding of what ur saying:
- CDM and HB say u can go from specific situations to abstract rules (world is orderly)
- NDM and FF say u should go from abstract rules to specific situations (world is complex)
it follows to me now that NDM would like to analyze how individuals like firefighters make decisions based off their subconscious pattern recognition / intuition. seems very aligned w the idea that decision making is at its best when it’s adapting to specific contexts.
if i were to extrapolate further, are u suggesting that individuals who subscribe to the CDM and HB paradigm are more likely to struggle w emotional stickiness in their beliefs? because releasing a belief is often related to larger frameworks rather than a micro level decision. (aka abstraction vs specific context narrowing).
i imagine w this school of thought, there’s more ego investment in being right from the start bc they assume their beliefs will generalize well across time and space.
not sure if that’s what u were implying, if not, how would u clarify ur view on beliefs that get emotionally stuck and why that happens / which school of thought would best explain it?
Sorry for the confusion. I wasnt trying to imply anything about emotional stickiness. And I'm not sure exactly how to think about emotional stickiness.
I suppose I would guess NDM and FF supporters would be more likely to have some sort of emotional stickiness because they rely more on intuition and feeling, whereas HB and CDM supporters prefer to use abstract models. Or at least, that is how HB and CDM advocates would see themselves - too rational to get stuck like that. Whether that is actually true, I have no idea.
This is a very insightful overview of the respective lines of cognition research, thank you. It took me many years to begin to find a way to understand the overlaps between macrocognition, expertise, heuristics, ndm, prospect theory, and frugal tricks, and Baron’s active open-mindedness and start to see a role for reflective judgment as well (Keith Stanovich was very helpful there). I’m thrilled when I find others interested in this important quest especially when they’ve come to see it from different angles than mine. For my part I think the reflective dimension is particularly important and dispositions qualities like intellectual virtues and especially curiosity play a huge role in shaping our use of the heuristics and ndm. Looking forward to reading more of your work.
I am also fan of thinking in terms of intellectual virtues. But your particular view of decision-making impacts what you consider a virtue. For example "open-mindedness" is often considered an intellectual virtue, but sometimes this is taught as "don't jump to conclusions" which can be bad advice depending on how you interpret it.
Experts regularly jump to conclusions because without a solid theory of what is happening, you don't know what to look for. Conclusions guide your attention and actions. Jumping to a conclusion is a necessary step to figuring out what is going on. Even when your conclusion is wrong, you still need to have one so that you can test and probe the environment. Without some sort of conclusion, you can only sit there inert and passive.
This is a fabulous essay which I could feel changing the way I think about thinking (and particularly K&T). Thank you, Jared. And thanks for the 'further reading' list.
Thanks! Glad you enjoyed it. There is enough content here for an entire book, and I may do a follow-up post at some point.
And I hope you write your book as well … though some “further reflections” on aspects of the ‘arguments’ would likely be more than adequate!
While spending time slowly digesting your terrific overview of the "rationality" debate, I kept thinking about two books that you didn't list (for good reasons as they their connection to this debate is little "slant" to use Emily Dickenson's term). These are Jonsen & Toulmin's The Abuse of Casuistry: A History of Moral Reasoning which is a terrific examination of the nature and need for case-based reasoning as well as its abuse. And Ziliak & McCloskey's The Cult of Statistical Significance: How the Standard Error Costs Us Jobs, Justice, and Lives
Sound interesting! I'll add them to the list. Though given my backlog, I may not get to them for a long time