I am doing this series because I would love to hear your thoughts as I develop what will eventually become a full book review of Mathematica (or my style of book review which is less review and more like a building upon). Please share your ideas and reactions, half-baked or not, in the comments

“One night in bed, after having turned off the light and closed my eyes, I realized that with a bit of effort I could imagine seeing my favorite cartoon.”

Aphantasia comes in degrees, and perhaps Bessis has hyperphantasia. For him it seems simple to conjure images, but maybe it’s not so simple for others and I am not sure how trainable this is.

I can imagine a cartoon, but it’s not strong. I see a series of still frame pictures with most of the details missing. The dialogue is truncated, with meaning coming through but not necessarily full sentences. For some reason, if I try to zoom in on a character’s face, I have a hard time imagining their upper lip - it’s like I have a localized facial blindness.

Books are even worse. I heavily depend on things I’ve seen from elsewhere that have the right vibes, but not any of the right details.

Here are two silly examples.

When first reading Harry Potter (before the movies) I imagined Ron Weasley (red head with freckles) as Gerald from Hey Arnold (black with a tall afro). Details are completely wrong, but they’re both best friends with a sense of humor. The vibes match.

When I read Way of Kings, I imagined Dalinar as General Horton from Courage the Cowardly Dog. Completely different in details, but both are grumpy military leaders.

These two have stood out to me because at some point I realized how absurd they were, and I wondered whether others have a clearer vision of what they are supposed to see.

Bessis doesn’t address aphantasia anywhere in this book, but I wonder what role it plays. There are mathematicians who have aphantasia and still manage to do high level mathematics. Here’s a math overflow on it.

But degrees of aphantasia matter, and can change what’s possible. How much of it is training? How much of it is inherent? I don’t think you can train away complete aphantasia, but perhaps you can train to be better so long as you have some minimal level of mental imagery?

Read more

I am doing this series because I would love to hear your thoughts as I develop what will eventually become a full book review of Mathematica (or my style of book review which is less review and more like a building upon). Please share your ideas and reactions, half-baked or not, in the comments

“Instead of letting yourself be lulled by words, force yourself to think that the sum is physically present in front of you. Force your-self to imagine the whole numbers from 1 to 100 in physical form, made manifest in the real world, carefully lined up in front of you. If you manage to see them and you take the time to carefully examine the scene, you’ll find a way to calculate their sum. To give you a chance to find it for yourself, I recommend you take a short break before continuing.”

This paragraph caused me to put aside the book for a week. In the times when I would normally read, I would instead play around with math problems. This is slightly odd for me because I have always hated math, but I did end up having fun.

At first I started playing around with variations of the Gauss Summation Problem. Summing up 1-101; 2-100; all the odd numbers 1-99 then the even numbers 2-100; 100-200; 101-200; -1-101. I started to get a pretty good intuition for summation problems.

Once that lost its novelty, I went to Claude and started asking for other problems. It took me a bit, but I figured out the first one it gave me.

But then it then gave me other problems, none of which I could figure out without hints.

Then we got to the birthday problem. “How many people do you need before there’s a greater than 50% chance that two of them share a birthday?”

I’ve heard it before, but I didn’t recall the answer. I had enough intuition to know it was between 15-30, but wasn’t sure the exact number.

I came up with a super clever trick for this. One that relied on Gauss’ Summation trick even! I was very proud of my answer: 20.

But my answer was wrong. And for the life of me, I cannot figure out why it is wrong. Or rather, rationally I know exactly why it’s wrong, but my intuition screams to me that it is right. I asked Claude to explain, and it didn’t help. I pushed back on Claude’s reasoning and it apologized, told me that it misspoke and then came up with a different reasoning which still didn’t make sense. At that point I lost faith in Claude and gave up on the effort.

Despite my frustrations with Claude, this little exercise taught me a lot. It helped me to better understand what it is like to have an intuition for when something is right, and what it is like to have an intuition that something is right even though it isn’t. Both useful.

But I still don’t feel like I am quite where I want to be. I want to be able to have that visual experience, and I am not quite there. When solving Gauss’ summation problem and its variants, I never came up with the triangle trick Bessis describes. I never saw the numbers stacked on top of each other as blocks. Instead, I always imagined digits. It’s not that I am incapable of that visual experience, I just don’t have enough familiarly with it for it to come easily.

This has made me reflect on Simone Weil who made a big deal about the difference between geometry and algebra. In my crude understanding, she saw geometry as something embodied - you can work with shapes which have a tangibleness to them. Algebra on the other hand is too rationalist, and too focused on pure calculation. I am stuck with the latter, wanting to move more towards the former.

With the birthday problem I did a bit better; I could see the people as a network of connections, and count up the connections using Gauss trick. But alas, it wasn’t enough. The mental move I know I need to make feels like thinking in negative space, and I can’t quite get there.

I will continue to meditate on this.

Read more

I am doing this series because I would love to hear your thoughts as I develop what will eventually become a full book review of Mathematica (or my style of book review which is less review and more like a building upon). Please share your ideas and reactions, half-baked or not, in the comments

“Grothendieck’s recommendation is to act like the 2-year-old. When he wants to understand something, he goes straight at it, without hesitations, as a child would. He doesn’t wait to understand before launching into it. He acts without thinking, a bit haphazardly”

lol like this series. Or really my entire personality.

“I began to listen to the dissonance between my intuition and logic.”

Perhaps I have been approaching “biases” in the wrong way. I’ve traditionally argued they don’t matter because the underlying rational models don’t matter. But maybe the better argument is that the Heuristics and Biases community is insufficiently curious about our biases. When mathematicians find out their intuition is wrong, they get intensely curious and dig into it - they don’t dismiss it as a universal human heuristic that is impossible to overcome.

For some reason I’ve struggled to make this argument before. I’ve argued against heuristics and biases, and I’ve argued for training intuition. But the world feels so vast, and the diversity of domains in which decisions must be made too diverse, that it can sometimes feel like a cop out to say, “Train your intuition.” As if I am saying, “Become an expert in every domain.”

But concretely, this is what people do when we care about something.

When I get a hint for a Sudoku puzzle, I don’t just use it, but try to deeply understand it so that I can use it in the future.

When I make a mistake in pickleball, I sometimes repeat the move to see if I was just too slow in getting into position, or if the position wouldn’t have worked in the first place. Sometimes I find that I’m holding the paddle too loose, or that my wrist couldn’t turn far enough and I should have switched to a backhand, and I’ll practice the new movement. In fact, it’s exciting when I find a “big glaring error” because there’s a chance that once I fix it, things will click into place and I’ll be a much better player.

I do this in psychology too. When something is unintuitive (e.g., Embodied Cognition), I spend copious amounts of time reading about it until it finally clicks.

Un-intuitiveness is not a property of the world or of our cognition, but a property of how we represent ideas in our head. And that can change. This feels very obvious to me, and Kahneman’s pessimism just seems so silly in retrospect. Like he’s telling us that surfing is not humanely possible because everyone falls the first time they try, and therefore we must have a Falling Down Bias that is universal to all humans that cannot be de-biased away.

Read more

I am doing this series because I would love to hear your thoughts as I develop what will eventually become a full book review of Mathematica (or my style of book review which is less review and more like a building upon). Please share your ideas and reactions, half-baked or not, in the comments

Roman numerals

Bessis talks about how Roman numerals don’t make it intuitive to know a billion minus 1, and it is only intuitive to us because we now use Arabic numerals. Herbert Simon talks about this in Sciences of the Artificial, applying it to problem solving broadly and design thinking more specifically. (page 144 of this PDF)

That representation makes a difference is a long-familiar point. We all believe that arithmetic has become easier since Arabic numerals and place notation replaced Roman numerals, although I know of no theoretic treatment that explains why.

That representation makes a difference is evident for a different reason. All mathematics exhibits in its conclusions only what is already implicit in its premises, as I mentioned in a previous chapter. Hence all mathematical derivation can be viewed simply as change in representation, making evident what was previously true but obscure.

This view can be extended to all of problem solving. Solving a problem simply means representing it so as to make the solution transparent. If the problem solving could actually be organized in these terms, the issue of representation would indeed become central. But even if it cannot if this is too exaggerated a view a deeper understanding of how representations are created and how they contribute to the solution of problems will become an essential component in the future theory of design.

Or as my boss John Schmitt has said, “ problem formulation and problem solving are not merely linked cognitive processes. They are the same cognitive process.”

I’m obsessed with this idea, and talk about it all the time. I’ve heard people use the term “psychotechnologies” to refer to things like currency, writing, and math which so fundamentally change what we are cognitively capable of, and extend our cognition to new domains. Such technologies are, fundamentally, just a change in how information is represented, nothing more. Same with the switch from Roman to Arabic numerals.

Arguably the biggest psychotechnologies created in the last 50 years was the spreadsheet, with word docs and slide decks coming close behind. As Joel Spolsky notes

Spreadsheets are not just tools for doing “what-if” analysis. They provide a specific data structure: a table. Most Excel users never enter a formula. They use Excel when they need a table. The gridlines are the most important feature of Excel, not recalc.

Word processors are not just tools for writing books, reports, and letters. They provide a specific data structure: lines of text which automatically wrap and split into pages.

PowerPoint is not just a tool for making boring meetings. It provides a specific data structure: an array of full-screen images.

It’s funny. When I was a teenager, I stored all my notes in word docs because that is what I was most familiar with. When I did data science, I kept everything in spreadsheets because the table format was natural for me. Then when I became a consultant, I kept track of ideas in slide decks. My profession actually shaped what kind of data structures I wanted to work with because it changed how I thought.

Read more

I am doing this series because I would love to hear your thoughts as I develop what will eventually become a full book review of Mathematica (or my style of book review which is less review and more like a building upon). Please share your ideas and reactions, half-baked or not, in the comments

Kahneman’s unfounded pessimism

Reading Mathematica has made me realize just how absurd Kahneman’s pessimistic view of debiasing is. How did anyone ever believe it? I’ve been “de-converted” from Heuristics and Biases for a while now, but even I hadn’t fully appreciated the absolute insanity of it.

The premise of the Heuristics and Biases methodology is that by studying human errors, you can diagnose the underlying mental activity. For example, if you ask someone the infamous bat-and-ball question and they give the wrong answer, you can diagnose the mental technique they used which would have given that wrong answer (presumably there’s too many ways to come to the right answer, and so a right answer tells you nothing about cognition). Kahneman and Tversky called the mental activity humans use “heuristics,” and the errors they occasionally lead to “biases.” (I’ve written about this here)

Kahneman was so convinced that these heuristics were basic and universal to cognition that he thought de-biasing was impossible. Our brain was permanently stuck using these flawed heuristics, and there was nothing we could do about it.

The problem with that view is that errors like the bat-and-ball example are not caused by basic and universal rules of cognition, but the product of how our intuition is developed in the modern classroom. There’s absolutely no reason to believe we should be stuck with whatever bad heuristic leads to the intuitive but wrong answer.

Read more

So here’s a short term compromise: I’ll share my notes on Mathematica by with paid subscribers. I’ll still publish a full review for everyone, so no one misses the main ideas. But paid subscribers can follow the process and shape how I interpret the book. This article is public; the rest will be behind a paywall.

This “thinking out loud” series is me doing exactly that, and so it will be messy. Expect half-baked ideas, notes to myself, uncertainty in my interpretations, and aimless circling around of ideas as I try to find an angle I like. If you want to understand what I am talking about, it’s probably worth reading the book (which is worth it. Spoiler alert: this is a five star book).

Future perks for paid subscribers are still TBD. I’m considering Q&A posts based on comments and critiques I have received, and possibly more series like this. Suggestions welcome. If money’s tight but you think you will have a lot to say on this series, send me a DM and I’ll gift a free subscription. Good feedback that will help me shape my thoughts is more valuable than a single paid subscription.

Why I am interested in Mathematica: Tacit knowledge, deep expertise, and the joy of a good challenge

I wouldn't ordinarily be all that excited about a book on math. But this is an exception, as it overlaps with my work so much. I study a field called Naturalistic Decision Making which is a particular school of thought in decision science that studies expertise in fields like Law Enforcement, firefighting, medicine and other domains where stakes and time pressure are high, and experience is extremely important.

Those domains may sound quite a bit unlike math. Heck, sometimes I even define what I study by clarifying how academics are not experts in the sense I care about. To differentiate from academic knowledge, we also sometimes say we study deep expertise which is characterized by all the stuff that is hard or impossible to articulate; what we call tacit knowledge.

But the promise of Mathematica is that Bessis is going to teach us “secret math.” The stuff that isn’t articulated in textbooks. The thing all mathematicians know how to do, but which they have difficulty describing, and so isn’t taught in schools. Can you see my interest? This is a book on the deep expertise, that is the tacit knowledge, of math - the discipline that is typically held up as an exemplar of explicit calculation and logic.

Tacit knowledge is often defined negatively; knowledge that can’t be articulated. Daniel Kahneman disliked this definition and once got into a debate against Gary Klein (my boss) and Barb Tversky about the utility of the term tacit knowledge. No surprise, I side with the latter two. And actually, I love the negative definition even more than they do; tacit knowledge is the stuff LLMs are never trained on because it can’t be found on the internet. But more importantly, it is the stuff that allows us to engage deeply with problems, and so seems to me to be the thing that makes us feel the most human, the most alive, and to have the most meaning in life. Using tacit knowledge to solve problems gives meaning to what would otherwise be laborious calculation and drudgery.

However, I’ve never found that experience of meaning and satisfaction in math and Bessis has; what does he know about doing math that I don’t? It seems like mathematicians understand the tacit knowledge and the Agentic Mode of math in a way I have never experienced…

The Agentic Mode of math: “...a human activity of a particular nature”

Agentic Mode is one of my favorite concepts (see here, here). It is one of those that the longer I have dwelt on it, the more central it has become to my understanding of cognition and human well-being.

The term Agentic Mode was coined by philosopher Thi Nguyen to describe how certain problems (especially games) evoke certain ways of engaging with the world, and how it is in the evocation that a game becomes an art form. A game might have other things going for it (maybe beautiful art work, or a good soundtrack), but it is the mode of engagement, the human activity it pulls you into, that gives it beauty. And not just games! There an entire class of activities Nguyen calls Process Arts which evoke beauty through the human activity it invites. For example, painting, cooking, and dancing can be enjoyable independent of the art, food, or dance performance that results.

As I read this chapter, it strikes me that Bessis is trying to describe the Agentic Mode of math. Math itself is a Process Art which evokes a certain human activity that has its own beauty. Bessis has another article where he argues against platonism and nominalism, and instead argues for a constructivist view. But perhaps just as much he is claiming that math is it’s Agentic Mode.

The agentic mode of math simultaneously (1) defines what math is and gives it its joy, (2) is distinct from but related to the Agentic Mode of becoming better at math, and (3) gives mathematicians their edge. (See Bessis three secrets on page 8. Not entirely sure of my description of the second secret)

On point 1, Bessis says “there is a latent consensus among mathematicians about what it means to do math and what it feels like” (emphasis his), and that “If this consensus were to be turned into a definition, it wouldn’t characterize math in terms of what it studies, but as a human activity of a particular nature” (emphasis mine). For Bessis “doing math is a physical activity.”

(Do I see hints of his fellow alum (archicubes?) Simone Weil here? None Enters Here Unless He is a Geometer).

When I was a kid, I thought the Agentic Mode of math was something like “exactness” and “logic,” and that those who enjoyed math must enjoy it because you can be incredibly exact and get the exact correct answer. I wasn’t entirely wrong; that seems to be the Agentic Mode schools promote, and some do learn to enjoy that. But Bessis seems to be pointing at how the Agentic Mode of doing math in school, and the Agentic Mode of a mathematician, are entirely different. The mode of a mathematician is more like meaning-making; you find a way of thinking about the problem such that everything comes together, and you can just see right through the problem. Or perhaps it’s more like the moment in a game when you see the winning move, and take it. I can see the appeal of that.

“This story deserves to be told”

This all reminds me of Wittgenstein a bit. He was a mystic and a romantic stuck in the body of a logician. He wanted to write about art and music because he thought that real learning came through those mediums. But what he knew was logic, and he couldn’t ever quite get to the point where he could explain through logic what art and music could teach (I wonder what he would have thought of tacit knowledge, deep expertise, and embodied cognition?).

Bessis seems to have been struggling with something similar in that he wants to push people to see through the logic, and was frustrated that for so long he didn’t have the words to describe it. There is a similar existential angst both share in that the world at large is focused on the form of an argument, and both Wittgenstein and Bessis want the words to describe how people can see past the words and formalizations (which are mere constructions) into the deeper more embodied reality.

“...no, mathematicians don’t think logically. It is in fact utterly impossible to think logically. Logic doesn’t help at all with thinking.”

This resonates with me as someone who has occasionally enjoyed doing variant Sudoku. Regular Sudoku is rather boring to me. But if you start adding extra logical constraints, it suddenly gets very interesting very quickly. Such puzzles can be incredibly complex and require lots of logic. But in a way it’s not logic at all, but more like flashes of pre-logical insight where you just suddenly see the move you are supposed to make. I’ve written about this elsewhere, but I agree with Bessis that we don’t do logic. I think it is pattern matching all the way down, and Bessis seems to agree.

Einstein: “I have no special talents. I am only passionately curious.”

I also resonate a lot with this quote from Einstein. I wasn’t a particularly “smart” child. In elementary school I did Hooked on Phonics and Hooked on Math because I was struggling. I had mediocre grades, did a special program called Bridges with other struggling students (still don’t know what that was about - did they think I had ADHD?), and was in the 60th percentiles in my elementary-age Stanford test scores. I never did the SAT’s or ACTs in high school because no one told me I was supposed to sign up for them, and I missed the window. I didn’t exactly show a lot of promise.

But as the years went on, things improved. And by my my late 20’s when I was studying my masters at an Ivy League, my class ranked me as the number one person they would go to for help on their homework. Some people have even said I’m the smartest person they know - and some of those people weren’t even my mother! But this is of course a ridiculous claim. My IQ is at most “average college student.”

But what I do have is a desire to think deeply. Thinking is fun. I love wrestling with a particular thorny problem and trying to find a way to represent the problem such that everything clicks into place, and the answer becomes clear. This is my peculiar Agentic Mode with which I wish to engage with the world. It leads me to consume and debate academic content for hours. Extroverts get energy from talking with other people; I get energy from engaging deeply with ideas. Some people get discouraged when they can’t work through a problem; it makes me all the more curious.

This relentless interest in ideas can give the appearance of IQ-smarts, and maybe IQ is a part of the story - but you are delusional if you think it can explain more than 10% of the difference between me and the average reader of this Substack. If I seem intelligent, it comes down to passion, niche curiosities, and a whole lot of time debating people on the internet.1 It is definitely not raw processing power, better working memory, or whatever flavor-of-the-week IQ research comes in these days.

Which brings us back to Einstein’s quote. When you discover the peculiar Agentic Mode/human activity/invisible mental moves involved in a particular intellectual pursuit, you also learn the beauty of it and can’t get enough of it. You develop an endless curiosity and fascination with that mode of engaging with the world which allows you to pursue your intellectual curiosity past the point where others give up. And people will be amazed at all that you know, and impressed with your intelligence. But in a way, that intelligence is total bullshit. You’re not any smarter - you just have access to the Agentic Mode that allows you to pursue ideas like a World of Warcraft player on a 24-hour binge.

Bessis did math, I did psychology. Neither of us care all that much about IQ because we realize raw IQ cannot explain the variance we see. The epistemic and motivational problems of learning a discipline disappear when you learn how to use and find joy in the mental moves of that discipline.

Why nurture over nature

It’s interesting that Anders Ericssen (another student of expertise) was such a believer in nurture over nature. He thought genetics explained approximately 0% of anything, and was quite radical in this view. His view is too extreme for me, but perhaps it is not surprising that those who study expertise might come to question the conventional strong determinism of IQ. We regularly see people of average intelligence do amazing feats, but then still see obvious gaps that makes us realize that even these amazing individuals are nowhere close to the ceiling of their ability. Also, they are completely average in other aspects of life.

I think it’s silly to deny intelligence altogether; as Bessis acknowledges people clearly differ. But when you see someone do a feat as impressive as playing a violin at an expert level, how could you possibly think: “oh they must just be smart”? No, the gap that exists between mediocre and superb is principally a gap in skill and practice, and perhaps access to that peculiar Agentic Mode unique to that skill. IQ can explain some of the difference maybe, but it’s such a miniscule part of the story. Gaps as large as that cannot be explained by genetics. IQ is swallowed by expertise, experience, and relentless pursuit of ideas.

The oral tradition

The idea that mathematicians hide true math because it seems less serious should resonate with anyone who is familiar with how experts treat rules and procedures. Or anyone who, like me, thinks that Rational Choice Theory is a failed theory of decision-making that only sticks around because of the veneer of rationality. All too often these things are mere post-hoc rationalizations of what our intuition already knows is right, or bureaucratic requirement for the sake of having something that gives the illusion of objectivity.

That is all for now. I will share my thoughts on chapter 2 and 3 next Saturday. I suspect early chapters will be the most rich with new thoughts and ideas, and that later chapters will be more about consolidating ideas.

You can try to imagine what it would be like to transform into a bat, and fly through the air using sonar to catch bugs. But you will always be imagining what it would be like for you to do so, because what it is like to be a bat is beyond comprehension. Some go as far to say that the science of the subjective is a contradiction in terms; such a science could not exist even in principle. (Of course, there is some disagreement on this point)

Nagel’s essay has been rather influential on me as it changed my answer to the infamous question, ‘If you could be any animal, what would it be?’ My answer for the last few years has been a bat, so I would finally have a good excuse for not reading Nagel’s essay.1

Good news though, I have now read the essay and can go back to my original answer; manatees (Chilling out in the warm waters around the Gulf of Mexico seems like a pretty good life).

But this essay actually has nothing to do with consciousness, bats, or manatees. It is in fact about vampires.

Just over a year ago, I wrote about L.A. Paul’s famous essay on whether it could be rational to choose to become a vampire. There seems to be a circular reasoning in such a change. A vampire is perhaps as alien to the ordinary human experience as that of a bat; so what could rationally motivate the desire to become an undead being that sucks blood? It seems that a desire to change is inherently irrational.

But of course, my essay wasn’t about vampires at all. It was about games.

A new board game can be incredibly boring when you first start out and are forced to read the instructions to figure out how to play. The rules don’t communicate what it is like to play the game, and so the experience you will have can seem as alien as that of a bat or a vampire. You may wonder about the silly rules, and how there could be anything possibly redeeming in those instructions. The transformation you will go through in the course of playing is very difficult to understand from the instructions alone.

But once you start playing, things change. The game may come into focus, and sometimes there is a moment where everything comes together and you understand why the game is enjoyable. The game manages to evoke a particular mode of engagement that Thi Nguyen calls an Agentic Mode; that is a particular way of using your own agency. And that Agentic Mode has a type of beauty and joy you couldn’t see in the instructions because it must be experienced in the doing.

The particular qualitative and subjective experience of the Agentic Mode the game evokes may be something you’ve experienced before in fleeting moments in real life, and you may have even appreciated its allure and beauty in those moments. However, Nguyen argues that games capture that agency and crystalize it into its purest form. In this way, games are an experimental design for the replication of one of the most peculiar aspects of human nature; agency.

So while bats can enjoy various Agentic Modes that we will never experience and may not even be able to understand in principle, they will never know the diversity of Agentic Modes which are available to humans. A bat is not a human, and it can never pick up a game and experience the Agentic Mode it allows humans to enter. And so while games might not capture anything as alien as zipping through the air on leathery wings and navigating through sound, they can capture many diverse modes which are incredibly peculiar to humans.

Like fatherhood.

Because of course, that essay wasn’t about bats, vampires, or games. Instead it was about transformation, and more specifically it was about the imminent transformation into a father I was about to go through.

As I wrote in the essay

This is the dilemma L.A. Paul highlights: how can I rationally choose to be something I don’t yet know how to value? I believe I want to be a good father. But at this moment, the ability to enjoy fatherly duties doesn’t seem possible or desirable, just as the ability to enjoy drinking blood seems neither possible or desirable.

In March of 2025, fatherhood was Nagel’s bat in that I could not truly appreciate what it was like to be a father. It was Paul’s vampiric choice in that I could not understand how to rationally change my preferences to that of being a good father. And it was also a game in that I was reading the instruction manual and found it boring, and I felt I needed to just experience the game in order to understand the allure and beauty of the Agentic Modes of fatherhood.

Well I am now a father, and my daughter nears her first birthday. So did I transform and learn to enjoy the Agentic Mode of fatherhood?

Well there is no single Agentic Mode of Fatherhood, and the ways you must engage vary month-to-month, and sometimes week-to-week. Babies change so rapidly that by the time you figure your child out, she has already moved on.

Early on, the Agentic Mode of fatherhood is mostly one of responsibility and duty to your wife. The child is hardly one at all - just a blob that cries, eats, poops, and sleeps. You can’t yet see who she will become, and her personality consists of metrics like how often and loudly she cries. So for me, early fatherhood was more just being a good husband by ensuring the newly branded mother of this blob would stay sane.

Fatherhood is a slow dawning for most fathers, and this can sometimes lead to a sense of inauthenticity. Agnes Callard notes that large transformations often feel this way. You have to try to live the transformation before you even see the beauty of it.

“In the beginning, we sometimes feel as though we are pretending, play-acting, or otherwise alienated from our own activity. We may see the new value as something we are trying out or trying on rather than something we are fully engaged with and committed to. We may rely heavily on mentors whom we are trying to imitate or competitors whom we are trying to best.

As many fathers have reported to me, fatherhood starts off as a sort of role play. You merely play the role of father for the first weeks or even months. You don’t yet feel like a father in these early stages. Ironically, for many fathers it’s not until the child can start playing that you can finally stop roleplaying, and just be a father. At that point, the possibility of a relationship finally opens up, and fatherhood is the result.

Thats what happened to me. Once the job of father was more than just “keep child alive” and “keep mother sane,” I started wondering about how I interact with her, and whether those interactions will influence who she will become. And those aren’t any questions; those are the questions of a father.

Of course every parent is different, but this wondering led to an an explosion of Agentic Modes. There are an infinite number of ways to play, and so therefore an infinite number of ways of engaging with your child; each influencing who she will become, and what she will enjoy.

My daughter inherited my relatively flat affect, and it can be hard to get her to smile. God forbid she become like me instead of like her mother! Therefore I’m always trying to surprise her with something new, something she hasn’t seen before in order to get her to express herself. In so doing, I often surprise myself; babbling, dancing, singing, WWE throw downs (onto the bed). She loves it and grins with her mouth wide open. I love it, too. Did you know babbling nonsense words could be fun? If you don’t have experience with babies, maybe the Agentic Mode of babbling is as alien to you as that of flying around and catching bugs using sonar, or of sucking blood through someone’s neck using your vampiric canines. But as they say, don’t knock it until you try it. (err, well…you know what I mean)

As we approach a year, the child now has a personality, and that too is a transformation in the relationship. She likes crawling around with a plastic knife in her mouth as if she were Rambo (it falls out before I can get a picture). She doesn’t really care for the soft touch of a stuffed animal, but instead prefers the harder plastic of remote controls and bottles of lotion, or the rubber of a rubber duck. She loves to sit with you and kick her feet. She has no stranger danger to speak of…except around my dad who is perhaps either too large, or too close to being (but not quite) her father, placing him within an uncanny valley. Not that long ago she was about the size of a QT cup, so I’ll forgive her hesitancy.

As this personality comes into view, all the ways the personality goes wrong also comes into view. My wife and I often talk about how to shape this being. We talk about the ways our parents raised us, and ways that family and friends raise their kids. It is the most productive of gossip; the kind that makes you understand why gossip evolved in the first place. It’s a relief how often my wife are aligned in our judgments of others parenting (both what we like and dislike). She tends to be more strict and cautious than myself, but never too much, and sometimes she surprises me with a slight rebellious streak.

If I were to describe what this stage is like (the Agentic Mode it most resembles), I would say it’s like World Building.

I know, I know. I am a nerd.

If you are not familiar, imagine what it would be like to be J.R.R. Tolkien designing Middle Earth. You have created these hobbits, elves, and dwarves, and you have to make all these decisions in order to figure out who these people are. Will their language be elegant or rough? Their architecture rounded or angular? What virtues will they prioritize? What will they consider sacred and profane?

Tolkien called this sub-creation. For him, myths and fantasy are not just about creating something cool, but about expressing something more fundamental.

“Fantasy remains a human right: we make in our measure and in our derivative mode, because we are made: and not only made, but made in the image and likeness of a Maker.”

So like God shaped Adam and Eve, Tolkien shaped his Hobbits, and we shape our own halfling; trying to express in her and through her our deepest values. We do this in our plans for how we will handle various parenting situations (What if she calls us a bad name? Do we allow her to go on sleep overs?), but also in things we do now, such as the books we read, the ways in which we play with her, and when we do and do not give her attention.

Sometimes these choices are less indoctrination into our values than it is indoctrination into our sense of aesthetics; my wife jams out to Taylor Swift full volume and dances with our daughter in the morning, and then will hum Concerning Hobbits when putting her down to sleep. I only endorse one of these things.

As my daughter begins to hold on to the couch and shuffle her feet, I am once again learning the agentic side of physics; hard surfaces and corners, the anticipation of how a body moves and falls. I am on a constant mission of anticipating the many ways a skull can crack. And of course, its not just corners. Every knick knack is now a death trap. Every time you think you hid every possible choking hazard, she finds a new one.

The umwelt of fatherhood changes too quick to get comfortable; you just have to constantly anticipate that next week you will understand what it is like to be a father of a N+1 week old, even if you cannot yet fully see it. Even if the next stage of fatherhood is as foreign as vampirehood.

Probably these metaphors don’t work for everyone. Maybe I am the only father nerdy enough to think raising a daughter is kind of like writing Hobbits into existence. But I think one of the great joys of fatherhood is to experience all kinds of ways of being; these Agentic Modes that are normally so fleeting in normal life that you have to play games in order to find them. Except with fatherhood, the bounty of all the various agencies of human existence find you; no point system needed.

Disclaimer: These more sappy and personal articles are rare, and frankly, I find them fairly embarrassing. If this is the type of content you are interested in, please do not subscribe. Mostly I write about the science of decision-making and expertise, with occasional delves into the philosophy of science and psychology, and that is what you should expect from me going forward.

Subscribe now

System 1 and System 2 are, by traditional accounts, two modes of thinking which are comparable to Aesop’s The Tortoise and Hare. In the fable, the eponymous critters decide to race, but the hare is so confident that it decides to take a nap halfway through. When it awakes, it finds the tortoise has already finished.

The metaphor is well suited for Kahneman’s central psychological story; a System 1 style of thinking that quickly and overconfidently leaps to conclusions, and a System 2 that is slow and careful, and so eventually gets the right answer.

But my favorite similarity is the part often left unstated; they’re both fables. As Kahneman says of the dual processes in his book Thinking Fast and Slow;

“System 1 and System 2 are so central to the story I tell in this book that I must make it absolutely clear that they are fictitious characters. Systems 1 and 2 are not systems in the standard sense of entities with interacting aspects or parts. And there is no part of the brain that either of the systems would call home.”

But dismissing Kahneman’s Fable as a mere fiction would be a disservice. The famous saying goes that “all models are wrong, but some are useful,” and I think that is true of these fictitious characters, as well. The distinction seems so obviously true to our experience that System 1 and System 2 are not even the only names we have given to it; logic vs. emotion, left vs. right brain, reptilian brain vs the neocortex, and (the most fun to bring up at dinner parties) male vs. female brain.

There have been attempts to move beyond this “Systems” terminology. Keith Stanovich, who originally coined the terms, no longer uses the word and instead talks about the Reflective Mind, Algorithmic Mind, and the Autonomous Mind. There have also been attempts to talk about Type 1 and Type 2 thinking. But none of it has stuck.

There is something interesting here we are grasping for, but so far it seems that we haven’t found terminology that isn’t just pure metaphor. So the field continues to use the fictional “Systems” language simply because it has not yet coalesced around a less false way of talking about it. And until we do, it perhaps makes sense to continue to do so.

But I think I now have a better story, which goes beyond mere metaphor, to actually explaining the illusionary distinction in terms of the actual cognitive processes we rely on.

The central idea is that System 2 does not exist; it is System 1 all the way down.

The shaping of system 2

The basic principle on which the brain operates is pattern matching. Karl Friston and many others say “prediction” instead, but they amount to the same thing. Neurons do not distinguish between these human concepts, and so I will not either; recognition, prediction, construction are just words we use to emphasize different aspects of the same electrochemical process that occurs in the brain.1

This process is described beautifully in How Emotions Are Made (a book I have talked about elsewhere). The patterns we observe, and the context in which we observe them, trigger electrochemical cascades through a network of neurons. These cascades start with the pattern input, and end with some predictive output. This process is System 1. Hardly a system at all except in so far as the entire brain is a system.

For example, when you see 2+2, an electrochemical cascade ripples through your brain leaving you with the answer of 4. Completing this pattern is trivial and easy for most people, and is the prototypical example of System 1.

Now consider the prototypical example of System 2; multi-digit multiplication problems such as 17x24. For many of us, the cascade is barely a trickle. We pattern match to recognize “I don’t know that” and then give up.

But if you actually had to solve it (without a calculator!), how would you do it?

There are a few ways to do it. If you do it by hand, you would write 24 over 17, multiply each digit together, and then add up the sum. Or if you want to do it mentally, I did it by multiplying 24 by 20 to get 480. But 20 is 3 more than 17, so I multiplied 24 by 3 to get 72 which I subtracted from 480 to get 408.

Whichever process you use, I can guarantee there is no need to evoke “System 2” to explain what you did. There weren’t two types of thinking involved. All you have to do is break down the problem into a series of easier patterns.

So what is System 2? It’s just pattern matching in a series. We can distinguish between Direct Pattern Matching to an answer versus a Structured Pattern Matching in an organized series of steps. But they’re both just pattern matching. Why evoke System 2 at all?

Pattern shaping

Of course, identifying that structure is an important part of the process. But that too is just pattern matching. We can call this Pattern Shaping just to give it a name, but we need to be clear that this name distinguishes a function—identifying a structure to bound subsequent pattern matching—but it is not itself a different type of cognition than the pattern matching which we have been talking about.

I recently came across a great example of Pattern Shaping by David Bessis who describes his first encounter with the bat and ball problem: “A bat and ball cost $1.10. The bat costs $1.00 more than the ball. How much does the ball cost?”

Most people try to pattern match and get this wrong. But David Bessis has done enough math problems in his life that when he sees a problem like this, he pattern matches to recognize it as the type of problem you can represent in terms of length. This is a really clever way of thinking of the problem, and very different from how I think.

If you want to see his approach, read his essay (or his book, which is on my reading list this year). The important point is that when viewed in terms of length, the answer was immediately and intuitively obvious to Bessis who has enough familiarity with seeing problems in terms of lengths.

Of course, what is easy for one person will not always be easy for another. Bessis may be able to solve problems by thinking of it in terms of lengths, but I cannot. Not because I am incapable, but because it is a novel way of thinking which I haven’t trained on. Pattern recognition is dependent on experience with those patterns. The cascade in our brain cannot pattern match to things it hasn’t seen enough to recognize. But if through pattern matching you shape a problem into a series of patterns you have seen before, then you can solve novel problems.

The idea that you solve problems by recognizing them in terms of problems you have seen before is something I talk about often, and for a long time.2 One of my favorite quotes from Herbert Simon applies this principle from mathematics to design thinking and the rest of problem solving

“All mathematical derivations can be viewed simply as change in representation, making evident what was previously true but obscure. This view can be extended to all problem solving—solving a problem simply means representing it so as to make the solution transparent.”

What kind of problem solving? Well, all the problems.

In the Recognition Primed Decision-Making (RPD), a model developed to explain the decisions of firefighters and warfighters, individuals “use their experience to directly identify the situation as typical of a standard prototype and to identify a course of action as typical for that prototype.”

In their description of expertise in real world contexts, Dreyfus and Dreyfus (2005) say that “normally an expert does not calculate, or solve problems, or even think. He or she just does what normally works and, of course, it normally works.”

Over in the world of Behavioral Economics, Bordalo et al. 2024 “offer a theory of decisions in which selective attention to the features of the current problem is determined by its categorization in a set of problems the [decision-maker] solved in the past.”

Or as Barry Schwartz and Richard Schuldenfrei say in their new book (which I reviewed), “Well framed is half solved.”

Sorry for so many references, but this is an important concept and may be what defines rationality. When we find a frame that allows us to utilize our existing repertoire of patterns, perspicuity3 into the correct answer is the result.

In a paper Joseph Borders and I have submitted to the NDM conference this year, we call this the Situation Shaping Hypothesis. The idea being that a large part of decision-making and problem solving is about finding a way to represent4 a problem where the answer seems obvious, or common sense. This was one of the original insights from Naturalistic Decision Making: the ways in which individuals made sense of situations often exerts greater influence on actions than does deliberation over a set of predefined options.

Again, I will emphasize that neurons do not distinguish between recognition, prediction, and constructing, and we just as well could have called it the Situation Recognition Hypothesis if we wanted to emphasize the pattern matching aspect. Regardless of the name, the point is identifying a way of thinking about the problem which breaks down the problem into something easier.

To summarize; the most basic function of the brain is pattern matching, and there is no process above and beyond that. However, while we can do Direct Pattern Matching to an answer, we can also pattern match to recognize there is a structured series of steps we could take, which I am calling Pattern Shaping. That identified meta-pattern binds and serializes subsequent pattern matches so that we can do Structured Pattern Matching. But at the end of the day, all three of these processes are just pattern matching.

Conditions for expert matching

In the Kahneman view, System 1 is seriously flawed, while System 2 is careful and logical. This account undermines that view significantly,

System 2 isn’t logical like a computer, but is instead a tower of fast system 1 pattern matches; one intuitive leap to recognize the problem in terms of a series of smaller leaps.

I think those who engage in serious logical problems will recognize this as closer to reality. For example, when I watch Simon of Cracking the Cryptic solve a Sudoku puzzle, I can follow and explain each step in his logic. But when I try to solve the same puzzle, I struggle to identify where the logic applies because I lack enough experience to do the same pattern matching. As it turns out, even when doing logic, pattern matching dominates as we depend on patterns to determine how and when to apply the logic.

This may sound familiar to those who have had the opportunity to speak with experts on how they feel about procedures and protocol. Nice, clear and logical procedures are nice and all, but the principal cognitive problem they are trying to solve is recognition of the nature of the problem. Because once the problem is recognized, the procedures may not even be necessary as the next steps reveal themselves. The procedures, at that point, are only aids for novices after someone more expert than them tells them what the situation is, as procedures can’t help if you don’t recognize the problem.

However, while this view undermines the careful procedural logic of a System 2, it does help to explain why “slow” thinking is often seen as superior to “fast” thinking. Structured Pattern Matching is indeed a corrective for more basic Direct Pattern Matching to an answer. We use such structured thinking precisely when Directly Pattern Matching to an answer is insufficient. In situations where you have insufficient experience for Direct Pattern Matching, you should add structure. And indeed, in our work we find experts tend to do both types of pattern matching; Direct and Structured. Perhaps the greatest misconception of expertise is that it is all intuition and gut instinct when in fact methodical thinking depends on expert pattern matching, and so experts provide some of the best examples of careful and methodical thinking.

But on the other hand, given how the brain works, enough time using the same structure can lead us to skip the Pattern Shaping and Structured Pattern Matching altogether. Even something as simple as 2+2 requires Structured Pattern Matching for one learning math, and even something as complicated as identifying how to put out a house fire can become Direct Pattern Matching when one gains enough experience. This is why many experts, such as firefighters, claim to never make decisions, as they have done the same thing so many times that they can Direct Pattern Match to the correct option.

Conclusion

I named my Substack after Kahneman and Klein’s famous adversarial collaboration, “Conditions for intuitive expertise: a failure to disagree.” At the end of that collaboration, they admit a failure to bridge their disciplines, but in their concluding sentence they invite others to try.

Consider this my contribution to the discussion. When we stop trying to force System 2 to be something distinct from System 1, both the strengths of intuition identified by Klein, and the weaknesses identified by Kahneman, become a little more intelligible.

When you find an effective way to represent a problem, it grants perspicuity into it. This happens not through mystical intuition, but by the simple fact that experts know how to frame a problem in terms of patterns they have seen before, and novices do not. Perhaps if we appreciated this, we’d have a lot less innumeracy in the world as we’d recognize our cognitive limitations as representational rather than computational. Once the representation is there, things click into place.

In fact, Kahneman anthropomorphized System 1 and System 2 for precisely this reason; he wanted to find a representation that made his story obvious. And that duo of characters he used did their job so well that they became embedded in the very way researchers talk about the brain.

But at the end of the day, System 1 and System 2 are as fictitious as the tortoise and the hare. We use such fictions to describe something true about reality, and that can be helpful. But in calling them System 1 and System 2, I think Kahneman gave these fictional characters the veneer of scientific explanation, which is a job they were never capable of accomplishing. And just as worryingly, researchers keep trying to add to the cast of characters, such as System 05 6 or System 37 8. It seems we are asking too much of this fiction, and I think we are in desperate and urgent need to move beyond it before it gets reified even more.

Hopefully this account offers something new: an account that bridges research traditions and replaces an overworked metaphor with a clearer explanation of what’s really going on when thinking fast and slow.

Addendum: I had been recommended against publishing this article because it contains new ideas, and I might get scooped. Or even if not scooped, journals might be reluctant to publish these ideas in the future because they will not be original. This is a constant trade-off I have to make and I am growing tired of it. Help me to feel better about my decision to publish original ideas on Substack by subscribing.

Subscribe now

The first is whether people approximate rational models when reasoning about the world.

By rational models, I mean those set of models invented (discovered?) over the years by philosophers, mathematicians, economists, computer scientists, and psychologists. You have probably heard of some of them; Probability Theory, Bayes Theorem, Classical Logic, Expected Utility Theory, Signal Detection Theory, and others. Very few believe we literally calculate out these equations in our head, but might we still act as-if we do?

The second interpretation is whether people, when they do not approximate rational models, are still rational in a more global sense (believing true things and making good decisions1).

It’s not obvious that a perfect statistician and logician would be a great decision-maker. Nor is it obvious that among the people we consider to be great decision-makers, that they are universally good statisticians and logicians. So there is a question over whether statistics and logic actually define what it mean to be rational.

Whether people reason as-if they are using rational models, and whether those models are even rational in the first place, are both interesting questions. So I thought it would be fun to provide some backstory for this debate, before providing my own take on why I abandoned the study of rationality altogether.

Backstory

In 1967, there was a large analysis of 110 experiments, trying to answer the question of whether people were intuitive Bayesians. That is, whether by default people were basically good statisticians. Their conclusion: yes! They found that statistics provide “a good first approximation for a psychological theory of inference.” The paper was titled, Man as intuitive statistician.

This may be shocking to many of those familiar with modern psychological research. This is definitely not the dominant view of human cognition 60 years later, where most see humans as systematically “biased.” How did we pull a complete 180?

Well, back then the methods were quite different. They often used things like an urn filled with different colored marbles, and subjects were allowed perhaps 1,000 trials over an hour to learn and make predictions about what was in the urn. When given such a set-up, people indeed began to act like they were using Bayes Theorem (albeit conservatively, they didn’t quite update enough). And given that the dominant view of cognition at the time was that the brain computes, then why wouldn’t it compute Bayes Theorem? Or at least come to the same conclusion as-if it were using Bayes Theorem? After all, Bayes Theorem isn’t just any theory, it is the theory which gives you the objectively correct answer about how uncertain something is (at least according to proponents).

Many of the studies that were done to support this idea were done in the lab of Ward Edwards, most famous for inventing (discovering?) Bayesian Decision Theory. As it so happened, one of his star pupils was Amos Tversky. As the story goes, Tversky was brilliant and charming, and therefore the kind of person you never debate in public. And for the first part of his career, he was a disciple of Edwards arguing that humans were essentially Bayesian in nature.

Then, one fateful day2, Daniel Kahneman invited Tversky to present to his class. This was a little strange because the two apparently had never really spoken before (some students thought they hated each other). But Tversky accepted and spoke about a study done in Edwards lab where subjects took poker chips from a bag, and had to estimate the composition of the bag (i.e., how many red vs white chips). The result from the study is that people got pretty close to getting it right, almost as-if they were using Bayes. Sure, they were not quite updating their estimate enough based on the data they were getting and so they were conservative Bayesians, but they did indeed seem to be Bayesians.

Kahneman listened to this talk, and couldn’t believe his ears. The experiment being described seemed patently absurd. Subjects pulled red chips out of a bag and then concluded there are more red chips than other colors? Well, no duh! Why was Tversky emphasizing how closely subjects approximated Bayes when he should be emphasizing how lousy they were at approximating Bayes even in this very simple set-up?

The talk ended, and now it was time for Kahneman to respond, “Brilliant talk,” he said, “but I don’t believe a word of it.” Then, according to those who were there, Kahneman took Tversky “to the wall” and completely annihilated all of Tversky’s arguments. This was shocking. Tversky was brilliant and never lost debates.3

But to his credit, Tversky went home and began to rethink his entire understanding of the human mind. In the subsequent weeks, he began talking with Kahneman more consistently, and the two stumbled on a very different methodology than the types of studies Edwards used. They called it the Psychology of Single Questions.4 This is the more popular (and easier) methodology now dominant in the field, and this method tends to show people making silly mistakes, such as systematically deviating from Bayes Theorem (and other rational models).

This new research was so popular that all of the previous research by Edwards, Tversky, and others seemed to get memory-holed. The new narrative was that people were nowhere close to being rational. Researchers in this tradition would write books about how irrational humans were, and comparisons of humans to Homer Simpson abounded. The idea that anyone ever thought humans were rational came to be seen as absurd and laughable.

Long time critic of Kahneman and Tversky, Gerd Gigerenzer was curious about what happened to Edward’s research, and mailed him to try and understand why Edwards wasn’t still doing research showing people were intuitive Bayesians. As Gigerenzer recounts

In response, Edwards sent a long, emotional letter in which he criticized cheap experiments with little control and thoughtless normative standards that portrayed people as irrational: “You are right in taking the illusionists to task for their kneejerk version of the Bayesian position”. But why had he not voiced his critique in public? Edwards wrote that he had turned away from the field because “if I had stayed with it, I would have to argue in public with them [Kahneman & Tversky]”. In his letter, Edwards referred to Kahneman and Tversky as “my boys” and wrote that one thing he regretted most in his professional life was that he “didn’t clamp down on my boys earlier”. According to Lewis, Edwards was anxious to avoid any public controversy with them. “Amos was the most terrifying mind most people had ever encountered. They were afraid discussing with him” (Lewis 2017, 155) and “no one wanted to get in a fight with Amos. Not in public!” (Lewis 2017, 319).

Do people approximate rational models?

With all that backstory, how might we answer the question of whether people “approximate” rational models?

Edwards and others (including Tversky’s earlier work) showed that in some situations where people are given time to learn, people absolutely do approximate rational models. In the 60’s we had over a hundred experiments showing that people were “intuitive statisticians.” Other experiments have shown they approximate other rational models given the right situations. For example, people do not fall prey to Confirmation Bias when given context, as opposed to being asked to reason about meaningless cards.

But also, Kahneman and Tversky and their intellectual heirs show that in other situations this is not true. In some situations, people vary significantly from rational models. Especially in the methodology that they use, the Psychology of Single Questions. Or in the same Confirmation Bias example, when they do have to reason about meaningless cards.

The disagreement between these two camps are not really about whether the experiments are replicable, but rather they are questions about whether the methods used can really tell us how people typically reason in typical situations.

Kahneman and Tversky provided evidence5 that people reason using heuristics such as Availability, Representativeness, and Anchoring. And that does seem like a pretty good description of what is happening in these experiments. But is that really all human reasoning is? Or might humans rely on different cognitive strategies depending on the situation? Might the heuristics become more informed with time? Might heuristics combine together in interesting ways to create more complicated mental processes? And can the Psychology of Single Questions possibly answer these questions?

The Heuristics and Biases paradigm was designed specifically to surface the reasoning strategies people use. But it’s not clear that most decisions in life really resemble anything like a series of trick questions asked by a psychologist trying to show people are not rational. And if those single questions are not representative of real life, then the very idea that people rely on systematically flawed heuristics is up for debate. People may in fact by intuitive Bayesians if these questions are not representative.

Of course, the same rhetorical point can be asked of Edward’s research; are most decisions in life really like pulling marbles from an urn? That too seems quite different from reality. If real life is nothing like that, then it may be reasonable to conclude that people are not intuitive Bayesians.

As the debates are not about replicability of the experiments, I think it is fair to say that everyone agrees that sometimes people approximate rational models, and sometimes they do not. What is less clear is the boundaries of when this happens, and more importantly, whether most of our life exists inside or outside of those boundaries.

In so far as this is a meaningful line of inquiry, it seems to me like the only way to resolve this debate is to get outside of the lab, and to study decision-making in real life. That way we can understand the nature of real life decisions, how people tend to make those decisions, and whether the decisions made resemble anything like rational models.6

But that leads us to the last question; would it even matter if people didn’t reason as-if they were using these rational models?

Are people rational even when they do not use or approximate rational models

The Heuristics and Biases tradition which Kahneman and Tversky founded is frequently misunderstood, even by prominent researchers in the field, as saying people are irrational. But in fact, Kahneman has said he hates the word.

“I don’t think that people are irrational. I don’t like the word ‘irrational’ altogether,” Kahneman said. “It reminds me of frothing at the mouth and humans are nothing of the kind.” Source

And

MR: You say you don’t like the term irrational.

DK: We don’t like it at all. Because I think it’s a complete misunderstanding. To claim we were studying irrationality – we absolutely were not. All the examples that we deal with were examples that worked on us. It’s just that we knew better because we had thought about the statistics. But every intuition that we described was an intuition that we found appealing, and we never thought we were irrational or stupid or anything like that. I really hate that word. And the word “rationality” in my mind is a technical word. It describes logic, and people cannot be rational in that sense anymore that they can speak perfectly grammatically. Obviously, they don’t. Source

And

DK: What I think has happened is that a lot of our work has been misunderstood as an indictment of human cognition. One of my least favorite words is the word irrationality. I’ve never used it. Rationality is a technical definition of the logic of decision making, or the logic of probability, which people intuitions simply cannot conform to. It’s not feasible for rationality, as defined in this, it is not feasible. So it’s not even interesting to say that people are not rational. Of course, they’re not fully rational, as decision theorists would define rationality, nobody can be. The question is to characterize how people think in a constructive way, and how that generates both correct judgments and error. Most of our judgments are correct, but the errors are not random. The errors are predictable and systematic. Source

I’ve previously shared the only communication I ever had with Kahneman in which he stated that he wasn’t interested in errors for their own sake, but only because the errors allowed us to understand the underlying heuristics.

More recently, Richard Thaler has said, “I don’t use the word irrational. I would say that we’re human and we don’t behave according to the models that economists write down and call rational.”

According to the founders of the field, there is a difference between “not conforming to rational models” and “irrational.” Biases are deviations from rational models, but that does not mean they are irrational.

This in turn would seem to imply that you can be rational even when not conforming to rational models. But yet, I’ve never heard Kahneman or Thaler ever state this explicitly. In Kahneman and Tversky’s original paper on small numbers which launched the Heuristics and Biases paradigm, they called people’s intuitions a “multitude of sins,” “indefensible,” “self-defeating,” and “ludicrous.” And that seems to still be the dominant way of talking about and thinking about biases. Based on how they talk about biases, I personally find it hard to believe that Kahneman and Tversky did think, or that Thaler really does think, that biases are not inherently irrational.

To try and better understand what is happening here, I will refer to Model-Rationality, wherein one’s reasoning conforms to rational models, and True-Rationality, wherein one is reasonable, competent and adaptive enough to believe true things and make good decisions. The question we are trying to figure out is whether these two types of rationality are equivalent.

There are a few different ways of thinking about the relationship between these two types of rationality. I will describe and briefly respond to the most popular views.

1. Full-metaphysical-equivalency: In some deep way, the universe itself dictates that True-Rationality is the same as Model-Rationality. In this view, rational models are not just invented, they are discovered. Bayes Theorem is not just a good model, but the objectively correct model which defines what it means for something to even be considered evidence. This normative realism is much broader than mere moral realism, and necessitates that every bias is de facto irrational, as there is no situation in which you should ever deviate from what a properly applied rational model says you should do.

I’ve gotten into many debates with people like this over the years. Typically rationalists (in the tradition of Descartes) in the rationalist community (in the tradition of Yudowsky). This metaphysical position is hard to argue against because it is a pre-empirical assumption. What scientific evidence could we ever point to so that we can settle this? It is a philosophical debate, not a scientific one.

2. Tautological-equivalency: It’s not that there is some metaphysical alignment between reality and our models, but rather, we define our models tautologically to be rational (e.g., even if you are a virtue ethicist, and you truly believe in doing x, then x is the highest utility option and therefore the rational option as defined by Expected Utility Theory)

This feels like smuggling in normative realism through the backdoor; presupposing some kind of commensurability between theories such that everything comes out according to one’s preferred theory. If such commensurability is possible, then we should be able to just as easily use Virtue Ethics as our standard, and yet no one in the field does that. Regardless, this is either metaphysical or definitional, and so not really something that can be settled by appeal to science.

3. Definitionally-irrational: We define the problem away. There is partial overlap between True-Rationality and Model-Rationality, but we only use the word ‘bias’ when that overlap exists. Therefore a bias is de facto irrational, but not every deviation from a rational model is a bias because the two types of rationality are not always equivalent.

This approach was introduced to me by Dilip Soman. The problem I see with it is in the identification of which situations require rational models, and which do not. I doubt Dilip and I could come to consensus, and our differences would not be purely empirical, but might come down to pre-empirical believes about reality, or of our ethical views. Therefore, whether a bias is present becomes a philosophical debate, and not a scientific one.

4. Pragmatist: Rational models are human inventions, and are applied by humans. In some real sense, the use of rational models is always ‘naïve’ in that there is no “true” way to apply them that gives us an objectively correct answer (except in some toy problems where everything relevant is already predefined). Because of this, sometimes using rational models leads to good decisions and sometimes not, and so measuring deviations from them is an exercise in futility because it’s not an objective standard except in toy problems.

This is the position I take. It is not scientific to ask whether Expected Utility Theory (for example) is objectively rational in any given situation, and so it is the wrong lens to be using when you are studying how people reason and decide.

5. Fully-exclusive: There is no situation in which True-Rationality and Model-Rationality overlap. This doesn’t mean all biases are rational, only that a deviation from a rational model is uninformative. Perhaps instead of rational models, you should obey revelation, a holy book, your heart, Deontological rules, or whatever dear supreme leader tells you to do.

A perfectly valid approach depending on your ethical commitments.

Each of these different interpretations seem to not differ in their understanding of psychology experiments, but in their metaphysical and meta-ethical assumptions. That is, the differences are not resolvable by science.

I think it is rather clear that people are capable of being reasonable, competent, and adaptive, but deciding whether they are Model-Rational when they do so feels intractable and unscientific because it’s not clear if rational models are actually normative, and even if they are, whether they can be applied impartially and objectively such that a comparing people to them is a reasonable thing to do.

So, are people rational?

Does it seem like a silly question to anyone else? Is the term “rational” even meaningful?

Even those who believe the term “rationality” to be meaningful are pretty skeptical about improving it. Kahneman was very pessimistic about the possibility of overcoming biases, arguing that it is much easier to change organizations, processes, and our environment than the cognitive machinery on which humans operate. This forces us to ask whether rationality is a dimension on which an individual can even vary. Keith Stanovich has tried to quantify this with his Rationality Quotient (RQ), but as far as I am aware, this hasn’t led to any breakthroughs in how to improve rationality.

Those in the Rationalist Community may disagree with Kahneman’s pessimism. That community, more than perhaps any other in history, has taken the idea of rationality seriously. So are they the Übermensch we should all aspire to be? And, if they are, why isn’t there copious amounts of research on them? Why isn’t JDM pointing them out as exemplars of what they want to achieve? While I love the community (or at least parts of it), so far it seems the academic community is unwilling to hold them up as the platonic ideal of their field.7

Given these metaphysical and definitional differences, and given the fact that it’s not even clear that you can improve your rationality, can I propose an alternative view?

Ditch rationality. It is not a meaningful term and cannot be studied scientifically. Rationality doesn’t exist in an objective enough way to lend itself to scientific analysis.

I’ll admit two exceptions to this rule.

One, you can study whether the brain itself operates on rational models, and that’s fine. If you believe the Brain is literally computing Bayes Theorem, that is a valid research project you can work on. Even if wrong, it is at least science.

Two, you can study economic rationality within the context of economics or other extremely quantifiable domains. Quantifiable domains are where rational models really excel, and it makes sense to use them as a normative standard there (most of the time). But even if they weren’t truly normative, Behavioral Economists are tasked with making psychology commensurable with economics, and so are forced to use rational models anyways. That is the constraint of their discipline, and so their use of rational models, even if somewhat irrational, can be excused.

Beyond these two exceptions, researchers leave the realm of empiricism and science whenever they start to ask whether people are rational, or compare humans to rational models.

But if not rationality, then how can we study normative decision-making? Well, you probably already know where I am going with this.

Rationality may not exist, but expertise does. All over the world, people develop incredible skills in their jobs, in sports, in art, or whatever else. Expertise is not pure intuition, nor is it rational models. To try and carve out expertise into either System 1 or System 2 would be a mistake. Expertise is fully using all our human facilities to make difficult decisions in the real world.

We know experts are superior at these things because we can see the superiority without needing to reference metaphysical assumptions, or debating the normativity of models. In the domains we study in Naturalistic Decision Making, experts regularly make life or death decisions. Shouldn’t that be taken as a sign that what experts are doing is real? Far more real than the comparison to rational models that currently dominates Judgment and Decision-Making literature?

Those in Heuristics and Biases regularly argue heuristics are rational most of the time, but then focus entirely on those situations where their answers are perceived as irrational. Fast and Frugal researchers may admit heuristics can lead to errors, but they spent their entire time pointing out all the ways heuristics are great. Meanwhile, those in the tradition of Edwards with their balls-and-urns…well, I actually don’t know what they are up to because for some odd reason I was never taught about them.

But at the end of the day, all three are stuck in labs, watching people make decisions in situations which don’t look anything like the rest of life. Whatever they discover about how people decide will always be suspect, all of their claims about the quality of people’s decision-making will be insufficient. They will never be able to assess what the default human decision-making process looks like, or what it looks like when humans are acting Truly-Rational in their day-to-day lives.

If you want to know if humans are capable, competent, and adaptive, or how to train them to be, then you have to study capability, competence, and adaptivity in the real world. You have to study human expertise. When you do this, you will pretty quickly realize that the vast majority of good decision-making in the real world looks nothing like rational models. And not just in high stakes domains like firefighting, but in most of life people are not even trying to be Model-Rational, but to be more expert. Once you see this, you will see rationality for what it is; unscientific chaff ready for the fire.

One of us is , a psychology professor at the University of West Georgia, and author of the Substack . The other is , an applied psychologist, and author of the Substack . Born and raised in the same cultural and religious milieu, we are both unconventional psychologists who tend towards the philosophical. It is perhaps not surprising that Jared, who constantly scours the internet for like-minded souls, would eventually find Jacob.

Perhaps our most relevant similarity is our shared criticisms of debates about causality and free will. We both see libertarian free will as incoherent, determinists as lost in tautologies, and compatibilism as just determinism with extra steps.

We also share similar criticisms of psychology and psychologists. Libet, Harris, and Sapolsky do not understand the stack of turtles on which they stand, and just about everyone else is missing the most important layer of analysis. Behaviorists describe human experience in terms of environmental stimulus and response; evolutionary psychologists in terms of modules; neuropsychologists in terms of electrochemical processes; and social psychologists in terms of traits, tendencies, and affects. There is value in these perspectives (when correct), but the overdetermined causal story they collectively tell seems to be missing the star of the show.

Nevertheless, and despite our shared criticisms, we also view each other with some of that same skepticism.

Jared could be accused of scientism. While religious, he tends to the unorthodox, and describe his faith (only half jokingly) as a sort of pesky persistent doubt. He tends to trust science over scripture, and reason over revelation. So, when Jared approaches the topic of agency, he is interested in the topic only insofar as it bears on the theories and methodologies of psychology and decision science. Indeed, he hadn’t thought much of the term until Jacob pushed him on it after reading Jared’s book review of Games: Agency as Art, in which Jared admitted reluctance to even use the word ‘agency’. For him, the question of agency is pragmatic, empirical, and methodological, whereas the philosophical and religious baggage makes him uneasy. He sees agency as the domain of psychologists.

Jacob, on the other hand, could be accused of dogmatism. He is religious, and outwardly so. He does not start with a hypothesis so much as a declaration of faith: agency must exist. He is less interested in agency for its methodological implications than the existential issue of what must be true about agency for meaning, responsibility, and morality to have bearing on our lives. This makes the question of agency an ontological, metaphysical, and normative problem. For him, agency is firmly the domain of philosophers and prophets.

So, while we both enjoy bashing on the lackluster arguments of Neuroscientists, our starting points are so different that it can sometimes seem we are speaking a different language. Nevertheless, given the importance of agency to our two projects—Jared’s to course correct psychology, and Jacob’s to establish the reality of agency—we decided to collaborate to see if we would fail to disagree.

Jared’s perspective: Agency as a Garden

You can’t not read a word. For anyone who is literate, to see a written word is to read the word. If all psychological findings were as dependable as this fact, psychological science would be a hard science (that is, an easy science).

But most findings are not that robust, and with good reason; we are primarily agentic in nature, and therefore more variable than any scientific law. Non-agentic word reading, and findings like it, are the exception.

While I hadn’t previously used the term ‘agency’, the idea that our psychology is primarily non-agentic is essentially what I have called the Digsite approach. In this view, human behavior is governed by deterministic and discoverable processes such as cognitive biases that cause us to err, and evolutionary instincts that jerk us about. Under this view, stimulus is destiny. There are some set of entities governed by rules1 which determine our behavior. Just as we cannot help but read a word upon sight, latent entities deep in our brain react to stimuli directly, ultimately driving behavior. A stable if-then mapping of causality which could be mathematically modeled.

But what evidence is there for such a view? A study finds that people on average do X, and we are supposed to conclude that the X Tendency is a feature of human cognition? Why? The behavioral average in a study is not a property of the brain, and I don’t know why anyone would make that conceptual leap.

Then an attempted replication finds on average people do not-X, and we are supposed to conclude that the original study didn’t find what it said it found? Again, why? Why should a completely different study provide any evidence about the average of a different study?

The alternative I have called the Garden approach (see here and here). In this view, experiments do not excavate fixed psychological laws; they cultivate conditions under which particular patterns of behavior emerge. Behavior “blooms” when a specific framing, set of values, and sense of relevance is successfully induced. Behavior is not causally determined by modules in the brain, nor is cognition epiphenomenal; rather, the meaning that is constructed through cognition is the driver of our behavior.2

Consider a military medic deployed in Franistan, a war-torn country smushed between Switzerland and Iran. A bomb has just gone off, and they are treating as many casualties as possible. A medivac helicopter arrives and they load it up with patients. Finally, there is just one spot left on the medivac, but two casualties. The first casualty is a fellow soldier who will likely survive even if not put on the helicopter, and the second is a Franistanian child who won’t survive unless he is put on the helicopter. Can we predict who the medic will choose?

If this were simply a matter of pre-existing values in the psyche, it would be straightforward to elicit those values and develop a formula which would allow us to predict who they would save. But this isn’t how it works. Different medics do not agree on what the core trade-off even is, or what values are relevant. One sees it as a choice of saving the most lives, another as the number of life years, and a third sees it as a choice between a soldier who may fight again vs. a child who may one day fight on the other side. Importantly, how they see the situation is not pre-determined by the situation or some set of pre-existing values, but by how they make sense of the situation in the moment. This is why psychologists say preferences are constructed.

Treating values as fixed entities leaves out the star of the show; agency. What varies is not some latent entity inside the person. Different answers do not originate in how medics weigh a pre-existing set of trade-offs, nor are they driven by lawful entities in the brain which pre-determine how the medics understand their context. Instead, the disagreement arises upstream of any such weighting, in the process by which the situation is framed up by the medic and made meaningful in the first place.

The frame that makes a situation meaningful is the core of the Garden approach. An experiment does not reveal a fixed psychological mechanism which jerks us, and therefore psychology is not in the business of cataloging internal mechanisms such as traits, tendencies, biases, modules, emotions, or whatever other type of latent construct. Instead, psychology is the study of how personal and situational context cultivates particular interpretations, and how behavior emerges as narrative coherence with that interpretation. Consistency in this view is the result of the context surfacing similar frames or interpretations, not of activating latent entities.

What prevents psychologists from understanding this view is a misallocation of agency in time. When agency is placed at the moment of choice, we end up concluding that humans are mere stimulus-response machines because the frame is already set at the moment of choice and predetermines the choice. Once a frame has stabilized, there is as much agency in choosing between options as there is in calculating out a formula, and so locating agency once the frame is set will of course make us look like non-agentic automatons acting out the programming of latent entities in our brains. In this misunderstanding, frames become mere biases, values become latent variables, and context becomes noise.

But when you recognize the agency in how we frame a choice to begin with, diverse human responses bloom. Not just in choices between options, but in the diversity in how we make situations meaningful in the first place.

Any serious account of human decision-making must therefore begin not with lawful latent entities, and not even with a choice between options. It must instead start with the agency that is exhibited through how we make sense of and give meaning to the world in which we find ourselves.

Jacob’s perspective: Agency as a Way of Being

In much of psychology, agency is understood as the capacity to make choices, typically framed as rational decision-making among alternatives. This definition is inadequate. Reducing agency to choice or cognition abstracts action from the lived, relational, and meaning-constituting contexts in which it occurs. Treating agency as choice construes action as selection among pre-given options, ignoring how agents actively form possibilities through commitment and engagement. Treating agency as cognition locates it inside the head, as deliberation or calculation, rather than in embodied participation in the world. Both reductions mistake agency for an internal event, leaving unexplained how meaning, responsibility, and authorship arise. Agency is more adequately understood as a relational, purposive activity through which persons actively shape their lives and the worlds they inhabit.

What follows are, for me, six non-negotiable claims, six core features necessary for a coherent account of agency. There is of course more to say about agency than what appears below, but given space constraints, this is the essential structure.

One: Agency is what humans are, not something they have.
Agency is not a thing, capacity, mechanism, process, or trait layered onto an otherwise non-agentic organism. It is not an evolutionary add-on or a psychological module. To be human is already to be agentic. Agency names how humans exist in the world. It is not optional or episodic; any human action is, by default, agentic because it is performed by a person. Even the automatic reading of a word which Jared mentioned is agentic, and is no different than what an expert does when they ‘read’ a situation. Both are fundamentally agentic acts.

Two: Agency is meaningful, purposive self-direction.
Agency consists in acting for reasons or grounds, not behaving due to causes. Agentic action is intentional and oriented toward purposes; it is neither random nor disconnected from meaning. Meaning arises precisely because actions are meant, not merely produced. This does not require deliberative, conscious decision-making in every case. A trained pianist does not decide on each individual note; the pianist intends the song and plays it. Intentionality need not be reflective to be real.

Three: Agency requires real possibility.
Agentic action must be non-necessary; it must be able to have been otherwise in principle. Constraint does not negate agency (e.g., physically placing a resistant toddler on the toilet does not render them non-agentic), but inevitability does. If an action occurs solely because it was fully predetermined by biology, environment, or prior causes, then it was not agentic. Real possibility also requires meaningful alternatives. Choosing between cups in a ball-and-cups game is not agentic if the ball has already vanished; even if one is not forced to choose a particular option, the outcome cannot differ in any meaningful way.

Four: Agency is originative but not isolated.
Agents originate action, yet are always related to everything else. Biology, history, culture, and context provide influences, reasons, and constraints without fully determining action. Agents contribute intention, commitment, and refusal. Individual and social dimensions are co-primary: agency is neither atomistic nor absorbed into context. For this reason, agency cannot be understood as radically libertarian or as entirely determined by circumstance.

Five: Agency is inherently relational.
Agency does not occur inside individuals. Because it is what humans are, it exists in relation to other agents. Agency emerges in the “between,” the space between people where things like norms, practices, affordances, and relationships exist. It is therefore always embodied, situated, and contextually budgeable: possibilities can be opened, constrained, strengthened, or weakened. Meaning, morality, and identity are continually co-constituted through relation.

Six: Agency grounds meaning, morality, and intimacy.
Agency is not one human feature among others; it is the condition of possibility for core dimensions of human life. Meaning exists because actions are intended. Morality exists because agents originate action and can be held responsible. Love, trust, and commitment can all be given or withheld, which is what allows for intimacy to exist.

The overlap

After some back and forth to make sure we understood each other’s jargon, the verdict is in. Our accounts focus on different aspects of the story, and our paths to get here are different, but it seems we have successfully failed to disagree.

Jared with his garden/digsite analogy is making an argument for how psych overstates its explanatory power. This is an empirical, methodological, and practical issue. But Jared’s research focused critiques cannot stay practical and methodological in nature. His claim that behavior varies because frames are agentically constructed rather than mechanistically produced is already assuming a degree of what Jacob has called real possibility, intentional authorship, and meaningful responses to grounds and reasons (as opposed to reactions to causes). In other words, Jared’s whole garden metaphor is presupposing an ontology of agency that mainstream psych does not presuppose. If it’s not presupposing a different form of agency, then framing itself just becomes another mechanism. So, this practical critique of psych itself requires some account of what agency must be in order for sensemaking variability to be real and not just illusion.

Meanwhile, Jacob starts out with the problem of what must be true about agency for meaning, responsibility, and morality to exist and make sense, which is an ontological, metaphysical, and normative problem. But while Jacob is making philosophical claims, those claims cannot stay at the level of theory. Ideas have consequences. If agency is relational, originative, and situated in the between, then it cannot be localized at choice points; variability must arise prior to deliberation; and meaning must be constituted in context. All of these philosophical claims are pushing towards a practical account of behavior that bucks the mainstream psychological methodology of looking for and reifying latent entities, tendencies, and effects as if they were causal entities to be collected and catalogued. So, Jacob’s account of agency that was made to ground responsibility ends up challenging many psychological explanations, and pushes for an understanding based on framing, situated cognition, sensemaking and meaning.

What seems to have happened is that Jared created a garden analogy as a critique of psychology and was wondering about the theoretical implications. Meanwhile, Jacob has been working on the theory of agency, wondering about the practical implications. And then they see each other, their eyes meet, they exchange phone numbers, and an article is born.3

Jacob listed his core claims, and so it is easy to identify at least a few major points of agreement (once you get over the different verbiage we use).

One: Agency is what humans are, not something they have.

Jared: I agree. The basic functioning of the brain is this agentic process we are jointly describing, not a list of modules or latent entities that we have. Moreover, I am convinced that my initial example of something non-agentic (reading a word) is in fact agentic after all. History and experience can make something so automatic that it can seem non-agentic, but is still an expression of our basic constructivist meaning-making capacity rather than some innate module.

Two: Agency is meaningful, purposive self-direction.

Jared: Agreed. Behavior should primarily be understood as responding to meaningful narratives rather than the net impact of being jerked about by various unconscious processes. Our thoughts are not epiphenomenal, and behavior arises because of the meaning we construct through the sensemaking process.

Three: Agency requires real possibility.

Jared: Yes. If it is literally impossible to do otherwise, then it is not agentic. Do such non-agentic processes exist in the brain? That is to say, something truly law-like where no context, belief, or intention can prevent it? I don’t rule it out, but I am not sure what it would be. Not even an infant’s latching reflex is so law-like that there is no other choice for the infant.

As for the claim that the cup-and-ball game where the ball has been removed is taking away agency, I consent on normative (as opposed to epistemic) grounds. This does feel slightly adjacent to my core concerns, but I suppose there are methodological implications; certain lab experiments commit a similar “sin” when they control too much of the context, removing possibilities for different meanings.

Four: Agency is originative but not isolated.

Jared: True. Agency is not a free-floating causal factor independent of the material world. It originates, and has biological, cultural, and historical influences.

Five: Agency is inherently relational.

Jared: Absolutely. There is no all-seeing eye that makes situations objectively intelligible, and meaning is not inherent in objects or people. It is constructed (by an organism) who contextualizes meaning in relation to everything else.

Relationality is so central that it could be the focus of a second article, but let me try and ground its importance here.

Agency cannot exist if meaning is simply out there waiting to be perceived. While information theory is deterministic, the ‘codebook’ used to interpret a signal is not and cannot be pre-determined by the universe. It is precisely in that indeterminacy that we find meaning making agents not controlled by deterministic causal “if x, then y” mappings, but instead find agents determining what even counts as ‘x.’ Something they do through historical, personal, and social contextualization. It is in the indeterminacy of the signal of the world that meaning making pries itself away from determinism, and can instead exist in a (relational) network of meaning constructed by the agent.

Six: Agency grounds meaning, morality, and intimacy.

Jared: This is precisely the thing which I hadn’t considered until we started talking. Prior to that, I hadn’t really thought much of the term “agency” as I was content with my cute little garden metaphor and the methodological implications. Nevertheless, on this point you have also convinced me.

I did not list any criteria, but here a few of my core claims for you to respond to, Jacob.

Seven: Agency is not located at the moment of choice.

Jacob: For sure. There’s that famous Libet study that found people’s brains lit up slightly before people become aware they’re going to make a decision, and so one supposed implication is that we don’t actually make agentic choices. There’s been many critiques of that study, but one is simply that agency isn’t found in the moment of choice like that. The participants exerted their agency in everything that led up to them being in the study. Choice is just one manifestation of our agency; it is not agency itself.

Eight: Psychological patterns do not prove internal latent mechanisms.

Jacob: Psychology wants to be able to predict and control human nature, and it assumes that for this to happen, human nature must be determined. Otherwise, everything would just be chaos! And that’s not true. Just because we’re not determined doesn’t mean we’re indetermined either. And just because we tend to act in predictable patterns does not mean we have to act that way.

Nine: Variability is a feature of agentic life; it’s not statistical noise.

Jacob: Building on the previous point, when people fail to be totally predictable, psychology likes to chalk that up to “confounding factors.” After all, you can’t control for literally everything in a study. But that assumes people are static. The truth is, human research is always “without replacement.” Please are always changing, acting, learning, remembering. If you give someone the same test twice, the second time will be different because the person has done it before. Human experiments will always have variations, because we’re agents, and we change like that.

Ten: Misunderstanding the nature of agency has methodological implications for how research is conducted (such as the limits of lab research) which is plaguing the field.

Jacob: Who would you rather have as a doctor? Someone with an accurate understanding of the body, or someone who still believes in humoral theory? If people are in reality agents, then it’s going to matter if we have a theory of agency that is true or false to reality. Psychology’s determinist assumptions shape its methods, giving us forced-choice tasks, hypothetical decisions with no real stakes, context stripping in the name of experimental control. These methods that assume determinism limit what we study, but also reconstruct people as non-agents from the get-go. Then we act like it’s a big discovery when our experiments say people don’t have agency.

A dialogue on our two approaches

Jared: When we set out to write this piece, the question I wanted to answer was whether we would fail to disagree. In the end, I think wherever we might have disagreed previously, you have now convinced me. If we still disagree on any substantive issue, it is hard to identify where. Our major difference is principally methodological.

Jacob: True. My approach is principally ontological, normative, and theory based, and obviously I think this is better because it’s mine. Your approach is methodological and practical, and it’s not obvious that you think your approach is better merely because it’s yours, as it’s possible you’re more humble than I.

Jared: Au contraire, I am far more arrogant! That’s why I use French when English would do.

Jacob: Regardless, I begin with what must be true for moral life to make sense, and you begin with what must be wrong for psychological explanation to keep failing. The difference in starting points is actually why the convergence and the ultimate agreement is informative. We both started at different points, landed on the same solution, which means that solution carries more weight.

Jared: Agreed. While I have to admit that starting with the normative and a positive assertion of faith still makes me uncomfortable, I am now in the unique position of benefiting from you having done precisely that. I even smuggled in the term narrative coherence into my description of agency which is a term I got from an article you shared which argues for disciple scholars. So, despite my continued discomfort, I come away with a stronger belief that science is best served by variety of perspectives, and that perhaps diverse perspectives are the most valuable precisely when they make me the most uncomfortable.4 Good scientists do not pretend to stand on an objective foundation, but instead must do their best to acknowledge their pre-empirical beliefs, lest they end up like the neuroscientists we criticize.

Jacob: Right. I think we can have a really pragmatic approach to science and use the method or perspective that best fits the question. If we limit ourselves to our scientific hammer, not only will every problem look like a nail, but we won’t be able to solve the problems that aren’t actually nails. I think it’s a mistake to assume that science demands we exclusively use our ‘objective’ scientific method; if what we’re studying requires a different perspective, then science is still benefited from the increased knowledge. Now, I’m not saying that just reading the Bible can constitute ‘good science,’ but truth doesn’t really care about where truth is found, you know? If agency is in fact the reality of being human, it doesn’t matter as much how we discover that reality so long as we get there. I found the truth of the matter through religion; you found the truth of the matter through practical science. Both starting points worked! We both made it to reality. Put that way, I guess it’s no surprise we fail to disagree. Maybe the thing we get out of it is I start giving more credit to science and you start giving more credit to religion.

Jared: What’s funny to me about this is that I have been told that my approach has some overlap with critical psychology and feminist psychology. But your view has its foundation in a typically conservative religious tradition (LDS). I take some comfort in knowing my intellectual adversaries won’t know whether to dismiss my arguments as post-modern feminist, or dogmatic Mormon. Too left-wing, or too right-wing. But in reality, I was completely oblivious to the scholarship of either when I started. I was just trying to be objective and follow the evidence. Therefore, I’m not sure whether our convergence is evidence for or against the value of attempted objectivity in science.

Jacob: I’ve had similar experiences where people see similarities in my religiously-grounded ideas and other branches of psychology that I didn’t know anything about. But then I would learn about a branch and discover some issues with it that I couldn’t overlook, usually regarding agency. And I’m basically of the opinion that if psychology has a false understanding of agency, then anything it has to say about humanity is just pseudoscience.

Jared: I’m not sure I am willing to go that far. Psychology still produces real knowledge despite these flaws.

Jacob: My worry is about a category mistake. If you think you’re studying rocks but you’re actually studying trees, you can produce real knowledge, replicate, do the science thing, but still be wrong about rocks. It’s a problem with misidentifying the object. I think whenever psychology does that it becomes a pseudo-science, because it claims to explain agency while employing non-agentic assumptions and methods.

Jared: I’m sympathetic with where you are coming from. Psychologists are confusing context-specific sensemaking for generalizable latent entities, and this is an extremely confused view that has done damage to the field, most visibly through the replication crisis, but arguably far beyond that. But I would say this; even Newton’s fundamental ontology of the universe was wrong, and yet I don’t consider Newton’s Laws pseudoscience. If you are doing the best science you can possibly do given your understanding of the world, that is science. Any other definition turns all previous scientists into pseudoscientists merely because we know more than them. I am not quite willing to adopt a view of science where the march of science turns everyone who came before me into pseudoscientists.

Perhaps it is better to say that psychology is still just pre-paradigmatic, unsure of what its subject matter even is. Either way, we can agree it is time to move to a more agentic paradigm.

An agentic psychology paradigm

There can be a science of agentic human psychology, it just might be different than what psychologists are used to.

Such a science would shift from causal explanation to understanding, and from deterministic prediction to intelligibility. Rather than an experimental paradigm that controls context, it would focus on how people make sense of that context; what they identify as relevant, how they interpret and give meaning to the situations in which they find themselves, as the meaning itself is what drives behavior; a meaning we create. These constraints certainly would push for a more qualitative science, but perhaps not exclusively, as some consistencies in human behavior can be observed through behaviors such as learned automaticity5, pattern matching, and framing. Such phenomena can be studied and some of it might be modeled.

An agentic psychological science would not only differ in methodology, but in how it influences and intervenes in people’s lives by moving from control to responsibility. Instead of explaining behavior away as the result of environmental stimuli and internal forces, psychology could be something that helps preserve responsibility for actions by helping people understand how agency is exercised in different contexts or constraints. This would lead us away from behavior change tactics that try to leverage subconscious mechanisms, and towards approaches that re-frame and give meaning to behavior in a more permanent and sustainable way.

A science of human life will be different from physics, geology, or paleontology, but it will still be a science. Taking agency seriously does not end knowledge exploration; it redirects it toward meaning-making, relation, and responsibility. Agency is what makes psychology possible: without it, explanation runs out of subject matter and gets reduced to behaviorism, neuroscience, or some other field. Psychology itself becomes mere epiphenomenon. The choice, then, is not between science and agency, but between a science of psychology that studies agency and a science that explains psychology away.

No one likes Drosophila melanogaster aka the fruit fly. At least, no one but biologists who like them so much that they breed them by the millions.

Drosophila is easy to keep alive, breed, and study. Researching them has resulted in major discoveries in genetics, evolution, and biology. This little insect has such a good reputation among scientists that it is often used as a metaphor for a simple and widespread study subject that is productive in generating scientific insights. Every field wants their own Drosophila.

Does decision science have a Drosophila?

One candidate is gambling, which arguably controls for the complexity and context of real life. This makes gambling a productive study subject for generating insights about how people decide.

Not only that, but gambling supposedly teaches us how to make optimal decisions, because we know the formula to maximize gambling wins; Rational Choice Theory (RCT). This is taken so far that, in the Heuristics and Biases research tradition, “rationality” simply means “in line with RCT.” Biases aren’t deviations from rationality; they’re deviations from optimal gambling.

But is gambling a good Drosophila? Even the first researchers to call it the fruit fly of decision research questioned some of its assumptions.

More recently, Barry Schwartz and Richard Schuldenfrei in their book Choose Wisely: Rationality, Ethics, and the Art of Decision-Making make the case against gambling, arguing it is not the essentials of decision-making but instead a distortion of it.

But Choose Wisely felt incomplete to me. If you are looking for critiques of RCT, the book is great for that. But then they spent only a couple chapters building out their own theory. Additionally, while they argue that gambling is a bad Drosophila, they never offer an alternative.

I think we can go further. Rather than write a book review,1 I would like to build and expand on Schwartz and Schuldenfrei’s work because, whether they realize it or not, their theory of decision-making is in many ways just a restatement of the principles of Naturalistic Decision-Making. A field I chose to study precisely because its main object of study is the best Drosophila for decision science.

No inherent frames, but inherent to the frame

A gamble is not a natural representation of a decision, and life certainly does not come prepackaged in that form. In fact, reality does not come pre-packaged at all; a fact which is a constant thorn in the sides of scientists, AI researchers, and journalists. What models and categories we use, the facts and values we identify as relevant, the stories we tell, etc., are determined not by the universe, but by our goals.

Because of this, a gamble can be understood as just one way of framing a complex and messy reality through a process called sensemaking, so that a situation can be made meaningful and tractable.

But a gamble is not the only way to frame a decision, which is important because the way a decision is framed has implications for our behavior. As Weick and Meader2 note,

to label a small portion of the stream of experiences as a ‘problem’ is only one of many options. The stream could also be labeled a predicament, an enigma, a dilemma, or an opportunity. Each of these labels has a different implication for action. If it is a problem, then solve it; but if it is a predicament, then accept it; if it is an enigma then ignore it; if it is a dilemma then define it anyway; and if it is an opportunity then exploit it. To call something a problem is the outcome of Sensemaking.

And I’ll add, if a decision between options, then evaluate options; if a gamble, then calculate.

At the inaugural conference of Naturalistic Decision-Making, a similar insight was noted: The ways in which individuals made sense of situations often exerts greater influence on their actions than deliberation over a set of predefined options. In a paper coming out next year, my colleague Joseph Borders and I call this the Situation Shaping Hypothesis. When a situation is adequately framed, that is to say, when we have made a situation meaningful and tractable through stories, frameworks, and categories that help us to understand what is relevant and why, then the next action to take becomes a type of common sense.

By common sense, I do not mean shared, but rather that anyone who has access to the same frame should also see the next action as obvious. This is because ‘common sense’ isn’t a property of the world, but a product of the clarity of your thinking. And for many of life’s most common problems, we have found a way to frame problems in ways that make it seem like common sense, meaning we don’t have to pause every 3 seconds to evaluate options. As noted by Dreyfus and Dreyfus, and more recently by Bordalo, this if often done by framing a situation as similar to problems one has solved in the past. Though certainly this isn’t the only way it works.

The take-away from all this is that a decision, and by extension RCT, is just one type of frame, and an inferior one at that. Framing a situation as a choice between options is a mere back-up plan when a better, more qualitative frame that turns the situation into common sense isn’t found. This has caused major problems for the field; once decision research took “a choice between options” as its primary unit of analysis, it set itself up to misunderstand the fundamental nature of decision-making.

Substitution, substitution, substitution

Schwartz and Schuldenfrei argue that using RCT to make decisions is an example of substitution—replacing a hard question for something easier. Thus, “where should I go to college?” becomes, “which school is ranked best?”

These two questions differ not only in content, but in the cognitive process that underlies them. The original question demands reasoning to make sense of ambiguity, and the substitute question is asking for analysis to reduce uncertainty.

One way to make sense of this process substitution is with a 2x2 table developed by Zack who distinguishes between four types of uncertainty.

Muhren et al., note that left column issues are resolved through decision-making and objective analysis, whereas the right column issues are resolved through contextualization through sensemaking. They further note that sensemaking must precede decision-making because part of the sensemaking process is determining whether there is even a decision to be made, as well as what kind of data is even relevant.34

Coincidently, this idea that sensemaking precedes deciding first occurred to me while trying to reduce Choice Overload (Schwartz most famous finding) on an e-commerce platform. It occurred to me there are always infinite options, but yet Choice Overload isn’t a constant. Why is that?

We concluded that Choice Overload is not caused by the quantity of options, but by the quality of the frame. By forcing customers to choose before they had time for sensemaking, our platform hadn’t contextualized the choice enough for the decision to be meaningful and tractable. Thus not only do decision-makers substitute one question for another, but as noted in the book, companies also force people into the substitution. Oftentimes to their customers’ own detriment.

But the worst substitution is committed by decision researchers themselves. What they want to know is ‘how people figure out what to do,’ and instead they ask ‘how people choose between options.’ Under this substitution, ambiguity and equivocality don’t even register as potential problems, as RCT is treated as the default correct framing. All of life’s decisions become gambles, and estimating probabilities becomes the foremost tool of rational thinking.

The consequences of this substitution have been drastic in the real world, and just as much so in research.

By assuming a frame, researchers force research subjects to skip the part where they make situations tractable and meaningful for themselves. Then having done this, the researchers force the subjects to make a choice. How many of the biases of human cognition can be attributed to this research-enforced-substitution? Arguably, all of them as the very idea of a cognitive bias is a deviation from a prescriptive model of rationality, and therefore the very idea of a bias presumes there is a correct mathematical or logical framing from which one can be biased.

Decision researchers have substituted an interesting question for a narrow one, and the results have impoverished our understanding of humans and rationality, as well as our understanding of human rationality.

The importance of qualitative frames

I like to think of the Data-Frame Theory of Sensemaking as a reciprocal relationship between Relevance Realization and Framing, where the frame guides a person to identify what is relevant, and what is relevant guides the framing.

This reciprocal relationship is essential to understand. Because if a frame helps to realize relevance, then there are significant disadvantages to using a quantitative frame (such as RCT) as opposed to a qualitative frame.

A quantitative frame is inherently a closed world because that is the nature of an equation. But even if we ignore that, probabilities and utilities are blind to the causal structures and meanings that matter and so do not aid in Relevance Realization. Because of this, RCT will lead one to fail to recognize what facts and values are relevant, and if we cannot recognize what facts and values are relevant, then we are doomed to become bad decision-makers, or as the authors put it; rational fools, if not moral monsters.

However, a qualitative frame (i.e., a story) can guide someone to realize new things by thinking through how things will play out. Qualitative frames reveal relevant facts and values.

A useful example Schwartz gives elsewhere is when a friend asks how she looks in a dress, and you have to decide whether to lie and say she looks good, even though the dress is unflattering. What you should do, he argues, depends on many factors. What is your relationship like? How will the truth impact her self-esteem? Does she have anything else to wear? To ignore these issues is to risk being a bad friend, a bad person, or both.

Such qualitative reasons only reveal themselves when thinking qualitatively. And the result at the end of such qualitative reasoning is often that you figure out what to do without needing to do RCT because you’ve already internalized the relevant facts and values. Such a solid grasp of the situation will guide your conversation better than any mathematical formula that outputs which of a binary choice results in greater utility.

Of course, this means that a qualitative understanding is also essential for researchers if they are to understand how people decide. As Schwartz and Schuldenfrei argue, anthropomorphizing humans is not a mistake, but treating them as gambling machines would be. A true scientific understanding of decision-making will have to rely on the qualitative models that humans use in the real world.

There are exceptions to this. I will give a pass to Behavioral Economics, as the principle purpose of their scientific endeavor is to make psychology commensurate with the existing mathematical tools of economics. Similar with neuroscience which is dealing with an entirely different level of abstraction.

But in cognitive psychology, to mathematically model the psychology of human decision-making is to fundamentally misunderstand the nature of decision-making. To try and model a human is to try and model a modeler. And the problem with that is that a model is fundamentally closed, whereas a modeler has to be open to what else could be relevant, or to finds ways in which it is relevant. This sort of context sensitivity cannot be formalized.

Of course, RCT is an ok retrospective descriptive model because once the frame and all its components are fixed in place, you can often reconstruct a choice in RCT terms. That retrospective fit makes it useful for some types of economic analysis. But unfortunately, it also tempts psychologists into thinking RCT can explain, predict, or prescribe decision-making.

But it can’t do these other roles because RCT assumes a closed world whereas the fundamental nature of decision-making is a bending, shifting, and opening of the frame. RCT inevitably fails as an explanatory, predictive, or prescriptive model as RCT is fixed in a way reasoning is not, cannot, and should not be.5

The better Drosophila

So now we finally come to the better Drosophila.

I briefly considered, along with Simon and Schaeffer, that chess might be the better candidate Drosophila, and was considering the title, “Economics Plays Poker, But Life is Chess.”

Completely unrelated to that line of thought, my book club recently decided on Thinking in Bets by Annie Duke. I finally got around to buying a copy, and had to laugh as I saw the title of chapter 1.

“Life Is Poker, Not Chess.” What are the chances? (maybe Duke can tell me)

Duke hasn’t convinced me of her title, but as I reflected on it, I grew dissatisfied with Chess, as well. Chess has a similar problem to gambling; it does too much of the framing for decision-makers. Chess is a closed world where everything relevant is on the board, the possible moves and goals already determined. Chess, like gambling, assumes away most of the sensemaking process.6

To get around this problem, we need to find open world domains where people make sense of varying levels of ambiguity and equivocality, from mundane problems to truly novel crises.

And then I facepalmed, because, well, that was my original reasoning for studying Naturalistic Decision-Making.

Of course, I didn’t put it in those terms at the time. What actually happened was that I was reading the brilliant Rationality and Relevance Realization which argues for the same ideas I am arguing here, and I saw comparisons to Gary Klein’s Data-Frame Theory of Sensemaking which explains the dynamics of how experts develop adaptive frames. And as I reflected on the comparisons between the two papers, I had an insight, “Oh, Gary Klein has already won The Great Rationality Debate. The rest of the world just hasn’t caught on yet.”

Silly? A little. But I still think it is true. After that realization, I decided that understanding the sensemaking of experts was the future of decision science. I stopped studying Heuristics and Biases (which I had already started to see as a dead end), and a couple years later I found myself working for Gary.

But let’s clear the air because many mistake Naturalistic Decision-Making as the study of intuition, even though researchers in the field often consider the term too vague, and totally insufficient for describing real world decision-making. The key dynamic of expertise is not gut feelings, which Gary himself says you should never take at face value. Instead, it is better described by what Schwartz and Schuldenfrei call automatic reflectiveness where a more tacit (or some might say, intuitive) process aids in determining what to think about, but then the experts think more explicitly through the implications, typically through mental simulation.

But we might as well call this automatic reflectiveness by its more traditional name; sensemaking. And for me, the best model of sensemaking is Gary’s Data-Frame Theory.

Of course, Drosophila is easy to study, and expertise much less so. I can understand criticisms which note this problem. But given the need for an open world domain where ambiguity and equivocality are often present, naturalistic studies of expertise may be the only option. Both games and lab studies commit the fallacy of the All-Seeing-Eye where the world comes pre-packaged, framed, and a rational answer already pre-determined.7 They can be useful to study, but ignore the core issue.

Mundane decisions, such as crossing the street, can be understood as demonstrations of expertise and so arguably could serve a similar role. However, mundane decisions tend to not reveal as much because people have so much expertise that they do not experience ambiguity and equivocality when doing such tasks. That is why the domains that Naturalistic Decision-Making typically studies, such as firefighting, may be better as such domains demonstrate sensemaking in situations of minor to severe ambiguity and equivocality, and use subjects ranging from experienced experts to untrained rookies.

Important to keep in mind, as well, is that NDM serves both the descriptive and prescriptive role that gambling-advocates think gambling has. As we study experts in such domains, there is a good chance our research subjects actually manage to make sense of these situations, and so demonstrate not only how people do make sense of situations, but how they should. Thus RCT is replaced by both the descriptive and prescriptive superiority of expertise.

Naturalistic settings may be the only option.8 And experts may be the best Drosophila we have. Perhaps this is why Daniel Kahneman has said of the work of Gary Klein,

When you read [Klein’s] descriptions of real experts at work, you feel that it is the job of theorists to accommodate what he has seen – instead of doing what we often do, which is to scan the “real world” (when we think of it at all) for illustrations of our theoretical notions.

Conclusion

This piece is too long, so I’ll be brief.

Decision science needs a science of qualitative sensemaking, not just quantitative choice, as our main tool for decision-making is sensemaking. Naturalistic Decision-Making is, at the moment, the field that takes this most seriously: with its Drosophila of expertise, it studies how people actually make sense of ambiguity and equivocality in real world settings.

My hope is that Choose Wisely is a sign that the broader decision-making world is catching up to this insight.

The genesis of this turn in parenting style is that I have occasionally received reports of a niece or nephew being too scared to get on a roller coaster. When I hear these stories, something snaps in my soul. The idea that anyone over three feet tall would be scared of a roller coaster is unfathomable to me. I was ecstatic when I was finally tall enough to ride my first roller coaster, and furious when I arrived at the park and learned they had changed the height requirements forcing me to wait yet another year. But scared? Never.1

There is something so unrelatable about fear of roller coasters that it has forced me to contemplate the emotional landscape of my daughter’s soul. Will she be too nervous to try new things? Too full of anxiety to take risks? Too scared to join her father on a roller coaster?

So, I have decided to preempt my own fears: I throw her in the air and catch her, let her fall and faceplant into the bed, lean her back in her car seat until she is upside down. It has been sublime watching her being so unsure to now giggling with delight as soon as I start hoisting her up by her ankles.

This isn’t too different from what other parents do. After all, roughhousing is older than our species, and you can certainly describe what I am doing in that way. But my theory behind it is perhaps a little different, because I don’t think of it as roughhousing so much as training emotional expertise by teaching new emotional concepts.

Which brings us to the real subject of this essay: Lisa Feldman Barrett’s Theory of Constructed Emotions.

There are two major types of emotion theories which can be explained using a familiar metaphor I have used elsewhere. There are the traditional theories where emotions are basic and universal, and so can be detected and discovered just as how a fossil might be discovered at a digsite. Then there are the less intuitive constructivist theories, such as Barrett’s, where emotions are more like flowers which are grown in a particular context. I am going to try and explain both theories using these metaphors, and hopefully in so doing justify my parenting choices.

Digging for emotions

The traditional approach to emotions takes the digsite approach. In that view, emotions are things in the brain we can discover. There is a basic set (typically joy, sadness, fear, anger, surprise, and disgust) which have a neurological basis as modules, neurotransmitters, networks, brain structures, etc. These emotions push and pull on us: joy makes us smile, anger makes us want to destroy, anxiety gives us butterflies in the stomach.

This view is so pervasive that in Pixar’s Inside Out, emotions literally take the controls of a girl’s mind to steer her. But the view only truly reaches its apotheosis in courts of law which distinguish crimes of passion from premeditated crime. “My emotions made me do it” doesn’t get you out of jail, but it may reduce your sentence.

But increasingly the tide is shifting against the traditional view. The evidence just doesn’t support it.

If you are scared, angry, or sad, it is not because an entity controls you. There is no entity to even be found. Emotions are not “mental organs” to be dug-up, and they leave behind no fingerprint that lets you identify them. In her book and publications, Barrett goes through and systematically debunks the idea that there is any kind of one-to-one relationship between an emotion and anything in the body. There is no single pattern of brain activity, hormones, or facial expression that lets you identify fear; anger can’t be distinguished from joy; and sadness is nowhere to be found.

It is not that there is zero correlation, but as Barrett describes in one of my favorite papers,

[T]he issue boils down to magnitude. How many psychological meanings can a single facial configuration support and still be interpreted as evidence for a typological hypothesis? Likewise, how many different configurations can be experienced as expressing a single category of emotion and still be considered evidence for an expressive prototype with inherent biological meaning? How far can a typological view be stretched before it breaks?

This does not mean “emotions aren’t real.” People really do experience what they experience. It just means there are no emotions to be “dug up” and discovered as unique causal entities. There are no universal or biologically hardwired emotional circuits. Emotions are not like fossils or ruins; they are not pre-existing entities waiting for you to find them.

The emotional landscape is a garden, not a digsite

Which brings us to the alternative; constructivist theories such as Barrett’s Theory of Constructed Emotions, which is where I will focus. In this approach the emotional landscape is less digsite, and more like a garden where many diverse things can be grown.

This requires explanation and an example.

Our bodies are incredibly efficient at allocating resources through a process called allostasis. This process involves predicting resource needs (neurotransmitters, hormones, glucose, blood, or whatever else) and allocating those resources as needed. These changes are objective and measurable, and sometimes we can even sense these changes. But they are not emotions.

For example, when preparing for a routine morning run, your body predicts your muscles need extra oxygen and will redirect oxygen rich blood away from less needed muscles—such as your stomach—to those which need it more. Your stomach might even protest this by repeatedly contracting. You’ve likely noticed these contractions before, and may have described them as “butterflies in the stomach.”

But these “butterflies,” despite typically being associated with anxiety, don’t have a one-to-one relation with that emotion. They are not caused by an emotion. They are not even evidence of an emotion. It’s just blood being redirected.

Consider a story from someone far more reputable than either Barrett or I: Spiderman, or at least the actor who plays him on TV. While making a meal with Gordan Ramsey, Tom Holland shares how a director once asked him to distinguish between two physical sensations: being nervous before getting on a stage versus being excited before getting on a roller coaster. You might contemplate the question yourself before reading on.

As Holland relates, the physiology is the same. A fast heartbeat, a tension in the chest, butterflies in the stomach, etc. What distinguishes stage fright from the excitement of a roller coaster isn’t physiological.

So, if emotions are not a physiological process, what are they?

They are interpretations or frames. Just as how you might frame a roller coaster as exciting or scary, you might frame your own physiological reactions as excitement or fear. This is why many experience roller coasters as exciting, and why I can find beauty in sorrow. The emotion isn’t physiology, but the interpretation of the physiology within the given context. Emotions are the product of our sensemaking; a multimodal summary of the situation in which we find ourselves. That is to say, emotion are mental content as opposed to mental process.

Which brings us back to parenting.

Hanging my daughter upside down by her feet isn’t just about bonding with her. I am teaching her that when she feels all these internal sensations that some of her cousins (bless their souls) experience as anxiety, stress, or even nausea, she can instead experience it as thrill, excitement, or challenge. By allowing her to experience various physiological states in a fun and safe environment, I am giving her a new emotional concept which she is already using to make sense of her body and world.

This is why I say the emotional landscape is a garden, not a digsite. The traditional approach says fear is something that happens to us, and we can detect and measure it. In contrast, the garden approach says that there is a near infinite diversity of emotions for any given physiological state, and you get to decide which to foster and grow as a response. Will it be nettles and nerves, thistles and trepidation? Or will it be roses and adrenaline rushes, levity and tiger lilies?

The bounds of our emotional experience are not our physiology, as physiology under-determines which emotion we are feeling (e.g., both thrill and fear are biologically the same). What matters in the Theory of Constructed Emotion is the interpretation of that physiology through the context in which we experience it, which we learn through a lifetime of socialization and experience.

Growing a good garden

Since emotions are not universal cognitive processes and physiology doesn’t bound emotional experience, emotional experience is extremely varied and malleable. We can get a glimpse of this variety in other cultures. The Utku Inuits don’t have a word for anger, and Tahitians have no word for sadness. The Germans have backpfeifengesicht (face in need of a fist), and Ilongot headhunters have liget (a type of exuberant aggression).

You can even invent emotional concepts. Here are two I thought of while preparing for this article.

• Bag-bliss: That moment of perfect still coziness of being back in your warm sleeping bag after taking a midnight bathroom break when camping somewhere cold.

• Dream-loss: The longing for a dream one has just woken up from and already forgotten.

Two completely novel emotions you have never heard of. But can you feel them? They are as real as anger and sadness; I have experienced them.

Different cultures even define emotion differently, with some cultures considering emotions to be purely social, or as physical descriptions of a situation or object, or may not even distinguish between thought and emotion. These descriptions are just as valid as our western conceptualization of “emotion,” which is itself a replacement of the earlier concepts of passion and sentiment. There can be no single objective way to define or categorize emotions, there are just all the diverse ways people construct and experience them. Because emotions are not systems in our brain, but mental content just like “thought”, diversity is to be expected.

But this brings up an interesting question. Given all the ways emotions can take shape, you might start to wonder whether some “gardens” are healthier than others. Is emotional health just about “positive thinking,” or is there more nuance to this?

To answer this question, it is important to keep in mind that our capability for emotion didn’t evolve so that we could enjoy the various flavors of emotion for their own sake. Emotions are ways of making sense of the world, and that sensemaking guides our actions. When we construct an instance of fear, that fear comes bundled with all the other associations to the concept we have learned in our life, such as avoidance. This isn’t inherently bad! Sometimes avoidance, and therefore fear, is appropriate. I don’t want to train my daughter to embrace every risk. That’s potentially worse than living in fear.

What I want for my daughter isn’t positive thinking, but adaptivity. I want her actions to be appropriate to the situation. For this, she’ll need more emotional concepts than just “fear,” but also worried, insecure, and, of course, thrill. The more emotional concepts she has access to, and the more refined and specific they are, the more adaptive and fit to the context her actions will be. She’ll know when to run, when to fight, when to seek help, and when to plunge ahead and embrace the risk. A diversity of emotions is just a diversity in ways of making sense of the world, and so underlies adaptive behavior. Barrett calls this skill emotional granularity.

Emotional granularity is not about detecting different emotions (that would be the traditional view), but rather consider how some can distinguish between two situations; the agony of bodily harm, and the hurts-so-good pain of healthy exercise, and others cannot, likely leading the former to have more consistent and enjoyable exercise routines. Or consider boxers and MMA fighters who will sometimes hit themselves to get into the right emotional and physiological state before a match.

Part of what is recognized in these examples is physiological (pain!), but also much of it is mental and informed by the context. The same physiological response experienced by different people, or even in different contexts, can lead to vastly different interpretations. The ability to detect subtle differences between situations and respond adaptively is the essence of emotional granularity. When physiological responses such as pain are treated not as basic brute facts happening to us, but as mental content we created to summarize the current situation, adaptive behavior is the result.

This may sound familiar to long-term readers.

In my own field of Naturalistic Decision-Making, a central finding is that experts rarely make decisions in the sense of deliberating between options. Instead their decision-making is better described by the Recognition Primed Decision-Making (RPD) model where the expert is able to recognize situations, and then construct an appropriate frame of the situation which implies the action to take. Because of this finding, expertise is sometimes understood as a type of situational discernment, where a greater ability to make subtle distinctions between situations is the defining characteristic of expertise.

Isn’t this just the same as emotional granularity? And note that I do not mean emotional granularity is analogous to expert situational discernment. Rather, what I want to hypothesize is that they are the same skill, just different domains. A medical expert can recognize and discern illnesses using subtle cues, and construct an appropriate frame (i.e., a diagnosis) which implies the appropriate actions to take. Similarly, an emotional expert can recognize and discern situations using subtle cues, construct an appropriate frame (i.e., emotion) which implies the appropriate actions to take.

It seems to me that emotional granularity is just RPD with emphasis on internal physiology and phenomenology.

Given this hypothesis, another way to talk about my roughhousing is that I am training my daughter to be an emotional expert. Though she is only 6-months and isn’t even interested in trying to communicate yet, I can teach her new emotional concepts which she can use to make sense of her body and world so that she can respond more adaptively.

Final thoughts

If psychology were in mortal peril and I had to assemble a team of elite researchers to save it from imminent demise, I’d pick Lisa Feldman Barrett as team leader. Her theory of Constructed Emotion has been a paradigm shift in not just how I view emotion, but also the mind, brain, and the context in which it exists.

Her theory has helped to inform my critiques of psychology, because if even emotions are “gardens and not digsites,” then certainly the existence of all the tendencies and effects which litter the field are also up for question. It has also informed my study of expertise, and there is much interesting work to be done connecting her theory to the Naturalistic Decision-Making paradigm in which I work.

And of course, her theory has even informed my parenting.

But this is not to say I am absolutely certain of the theory. I have questions around why babies smile and giggle, and whether my splitter inclinations should make me doubt myself. There is also the old bugbear Relevance Realization; how do our bodies predict resource needs before we have even conceptualized a situation in emotional terms?2 I am also not an emotions researcher, and don’t know what I don’t know. I feel some skepticism is warranted.

Yet this is the theory I am betting on when it comes to raising my daughter. I want her to understand that there are no emotional entities that push and pull her in a series of jerks, and that physiology is not destiny. I want her to be an expert not just over her emotions, but through her emotions by cultivating diverse emotional possibilities. And I have started training her in this ancient craft so that so that when her stomach goes topsy-turvy while waiting in line for her first roller coaster, she’ll recognize it as something from her earliest memories and will know the appropriate reaction is not avoidance, but smiles and laughter.

But as was recently pointed out to me, the best model for ALL cats is, in fact, not a cat. Any particular feline is far too overfit to be a stand-in for the whole genus. There are just too many dimensions on which a given cat may differ from all others, and so every cat will be an outlier in one way or another. This is as true for cats as it is for humans.

Because of this, a perfectly normal cat does not exist. It is an abstraction, a fiction.

Do not interpret me as saying this is bad. Sometimes a good fiction helps to make sense of a diverse population of instances. The fictional perfectly normal cat may be less ideal for understanding a particular cat, but it is helpful for understanding the set of all cats.

This trade-off between a highly specific model representing a small number of instances versus a more fictitious model which represents the larger set of all instances is known formally as the Bias-Variance Trade-Off. But I will give it the less complicated name of the General-vs-Specific Trade-Off: highly specific and context sensitive models do not generalize, but generalizable models are not specific and context sensitive.

The question I want to answer here is this: does the General-vs-Specific Trade-Off apply to decision-making?

I am going to claim that the answer is no. The trade-off is largely irrelevant as there is typically no need for generalizable models when making decisions in real life.

How to not make decisions just like an expert

One thing to note is that decisions do not exist in an objective sense. Reality is not a multiple-choice test. If we perceive options, it is because we have modeled the situation as one in which there are options. A decision is a fiction (like our perfectly normal cat) which we have layered on top of a messy, complex, and formless reality to give it structure and normality. Decisions are tractable and we know how to solve them, so modeling a situation as a decision makes our life easier.

The process by which we give structure to a messy, complicated, and formless reality is called sensemaking, and the subsequent structure is sometimes called a frame or mental model.1 But just because we can frame a situation as a choice between options doesn’t mean we typically will. In fact, it is rather rare and exceptional to do so. Since most of our lives happen in familiar domains, most of our “decisions” (if you can call them that) don’t require us to generate multiple options. Instead, our familiarity means there is typically one clear answer.

For example, how many decisions have you made today? Did you decide to adjust how you are sitting? To not commit an act of murder? Whether to get the milk or cereal out first? Or did all those “decisions” feel so natural that to even label them a decision misrepresents your experience?

Deep understanding of a situation results in effective frames which bypass the need to consider multiple options. The example I gave in What is Naturalistic Decision-Making is that how someone got lost implies how to search for them. So rather than considering multiple search methods, Search-and-Rescue tends to focus on how they got lost in the first place; did they wander off trail, miss their turn, or fail to enter the trail in the first place? Increased specificity in how someone got lost results in increased clarity in how to search.2 This means someone’s level of expertise is better measured by how they distinguish between situations than by their ability to weight options.

Since a good framing implies the answer, expertise must be understood not in terms of abstract models which generalize, but instead as a deep understanding of how to frame the particulars. As Dreyfus and Dreyfus argue:

We must be prepared to abandon the traditional view that runs from Plato to Piaget and Chomsky that a beginner starts with specific cases and, as he or she becomes more proficient, abstracts and interiorizes more and more sophisticated rules. It might turn out that skill acquisition moves in just the opposite direction: from abstract rules to particular cases.

Particulars, not abstractions, underlie expertise.3

This way of thinking about human cognition collapses the descriptive and the prescriptive, as well as problem solving and decision-making. And in that collapse, abstract generalizable decision models lose any advantage they might have had. Abstractions cannot be more informative than a frame which makes it obvious what to do.

And the trade-off?

But what about the General-vs-Specific Trade-Off?

It is true that expertise doesn’t generalize across domains. Years of Chess experience won’t make you a better Go player. The context-specific mental models of experts are just too domain specific and narrow for that.

But conversely, Expected Utility won’t make you a better player either. Nor is there any general model (mathematical or otherwise) which will make you a better player at both games. Your best bet is to actually learn Chess and Go.

I think what is going on here is that complex domains can be incredibly sensitive to context.4 And while a particular situation is unlikely to be an outlier on any given dimension, each situation consists of a thousand dimensions and so every situation is an outlier in one way or another. Typical situations, like typical cats, are a fiction. And as a result, abstract models which are tied to what is typical or average, will (typically and on average) fail.

Chess and Go are incredibly complex, nuanced, and context specific, more so than many assume, which makes them less amenable to generalizable abstractions than one might naively expect. But reality is obviously far more complex and fractally nuanced than either.

But there is a second and even more important reason the General-vs-Specific Trade-Off fails to hold water when it comes to human decision-making which is this; humans are not models, we are the modeler. We do not rely on any single mental model, but are dynamically sensemaking, framing, and adjusting our mental models depending on the needs, constraints, and particular context in which we find ourselves. We are not beholden to a single general model or framing, but can craft a wide variety of context-sensitive frames on the fly.

Because of this ability, we bypass the General-vs-Specific Trade-Off entirely. You simply don’t need generalizable models when you can, in the moment it is needed, construct a new frame which makes sense of what to do.5 Finding out how experts do this so consistently is (what I consider) the central question of Naturalistic Decision-Making, and is why I find expertise a more interesting topic than rationality.

How do humans find the right model in the moment?

Mathematical models do not change. Once the math is written, it is set in stone. A variable once present is always present, and an absent variable always absent. Newton’s equations didn’t change themselves when Einstein came along.

But humans are not like that. If cognition is an algorithm, it’s an extraordinarily strange one which self-organizes, adapts, and changes based on context and changing information.6 I suspect our brain works more like a cloud of starlings than like a computer. As I describe elsewhere, “to understand the collective intelligence of that moving mass, you have to understand how the birds interact with each other, with the wind, the predator birds they are avoiding, the ground, trees, etc. All the various interacting features of the system (living or not) contribute information to the entire system.” This ability requires an open ontology where anything can become relevant.

But of course, humans are even more open to novel variables than starlings. Consider how we can conceive of things which no one else has ever conceived. Our ancestors had no need to understand quarks or quasars. Yet humans can add these to our ontology and make decisions based on their existence.

Humans not only add to our ontology, but also ignore. We create what Gary Klein calls Just-In-Time (JIT) models.7 These fragmentary (non-comprehensive) models are constructed in the moment that they are needed. For example, a cardiologist working on heart structure will only consider the aspects of the heart relevant to the problem at hand. It’s not that the cardiologist has multiple mental models, but rather, their rich mental model of a heart takes multiple forms depending on which information is relevant.

This ability is why expert mental models, despite being incredibly particular, do have a type of generalizability. While skills don’t transfer between domains, they can transfer across problems within the same domain quite well because experts can adjust their mental model based on the needs of the situation.

It is this ability to dynamically frame a problem based on what is relevant in the moment which most amazes me about expertise. How do we realize that something is relevant without already knowing it is relevant? How do we identify the right frame without already knowing it?

I think the circularity required for relevance realization, insights, and expertise should shock you. It should set off red flags which say, “CIRCULAR LOGIC!” in all caps. It is so counterintuitive that relevance realization is possible that I can understand why people think abstract models which generalize should be superior, or even try to do away with models and representations all together (as ecological psychologists do). After all, how can you be sure that your context-specific model that you constructed on the spot is going to solve the novel problem you just came across? How do you know you are considering all the relevant variables?

Yet despite the circularity, realizing relevance is a central cognitive process, maybe even THE central cognitive process. I consider it the most interesting problem not just in decision-making, but all of psychology. Relevance Realization is what makes us modelers and not just models.

Of course, relevance realization sometimes fails. Novices don’t know what they don’t know, don’t notice what is missing, and can’t interpret warning signs because their JIT models are insufficient for guiding relevance realization. And in tough situations experts will struggle with this too. Overcoming the circularity is difficult.

So, what is the solution? How do you realize relevance? Or at least, how do you train yourself to be able to realize relevance?

The full circle

Well, you don’t fall back on abstractions! Instead, you get more specific and concrete. Do a deep dive into the context. Explore corners of the domain you hadn’t looked at closely before, dig into details with more granularity than you had henceforth been willing to do. Find contrasts which will throw into relief the relevant aspects you had previously glossed over.

This has become immensely clear in some re-working we are doing of Gary Klein’s framework on insights which he wrote about in his book Seeing What Others Don’t. For years Gary has been collecting stories of insights from interviews, books, and his personal life. Insights from firefighting, financial markets, science, or even fantasy baseball. The list currently stands at 138 stories, some of successful insights, and some of failures to have an insight.

While coding these insights, it occurred to me that not a single one of the insights involved going more abstract. In every case, the insight came from getting more familiar with intimate details of the problem than anyone else. For example, every person that predicted the 2007 financial crisis did so because they were deep in spreadsheets.

I think of insights like science. Specific hypotheses are falsifiable because they tell you what an anomaly or deviation from the theory is (they tell you what is relevant), whereas vague and abstract theories cannot guide your attention to any deviation. There is nothing to grab onto which can change your mental model and send you off in a new direction. It is only by getting specific than you can have a revolution in your understanding.

Similarly, you need concrete specific mental models to figure out what is wrong with them. This is why in every single case of an insight which Gary has collected over the years, deep familiarity with the topic (call it expertise) is the necessary pre-condition for the insight. A fact which has made me much more reluctant to use LLMs—I do not want to outsource the deep familiarity that is the pre-condition to having insights.8

But what about therapists who have less familiarity with a patient’s life than the patient? Is this not a situation where less familiarity is better?

I think that is the wrong way to frame the problem. Despite intimate familiarity with their own life, patients do not meet the conditions for expertise. They are like a firefighter who only ever fights the same fire, in the same house, using the same tools, day after day. They may be incredibly familiar with that fire, but they are not an expert at fighting fires. Their experience is impoverished and so they suffer a sort of relevance-blindness.

By analogy, patients who commit the same problems over and over again also have a type of relevance-blindness, whereas the therapist has seen enough to know how things could be different. The therapist has a richer mental model of cause and effect than the patient.

So, I will amend my original statement; you need to get deeply familiar with a domain to make expert decisions and have insights, but deep familiarity requires cross-contextual understanding. You cannot understand a phenomenon if you have only ever seen it in one context and have only ever interacted with it in one way. Reality is far too context sensitive for a single context to be sufficient for deep understanding and insight. Similarly, you will never learn to develop effective JIT mental models if you do not practice realizing relevance in changing circumstances.

Of course, introducing variety can be risky, especially in high stake domains like firefighting. But this is why mental simulation is so essential to expertise. Despite mental simulations being based on our own mental models, they still brings things to the foreground9 we hadn’t before considered, and so help with relevance realization. And, as I have argued elsewhere, stories are open ended models capable of going in unseen directions (quite unlike mathematical models). There is a reason mental simulation is an essential component of Recognition-Primed Decision-Making, and why scenario-based training can be so effective.

“Meow means woof in cat”

Academics are so intensely interested in trying to find the most normal and typical thing that they have three different terms for it; mean, medium, and mode. And when they do experiments, how do they determine whether their results are statistically significant? When something deviates too far from what would be considered normal.

I don’t hold this against the academic. The fiction of normal has its utility because it generalizes quite well. I quite like abstractions and generalities and depend on them when doing research.

But decision-making is not like research and so the General-vs-Specific Trade-Off is largely irrelevant. Reality is complex and context sensitive, and when making decisions it is far better to have dynamic mental models which adjust to the context than abstract models which generalize. Expertise, insights, and good decision-making come from not increasingly better abstractions, but from increasing familiarity with all the subtle details and nuances of a situation.10

When it comes to decision-making, the best questions are not about how we might be biased from rational models, but about how we are able to expertly form effective mental models in the moment they are needed.

But I was writing. It just all sucked. I was overwhelmed with everything I wanted to say, and articles would balloon past my ability to manage them. My voice was inconsistent, and my prose clunky and repetitive. I had some occasional successes, but by and large it was one failure after another.

Then LLMs came out.

I don’t know how much of my Substack was written by an LLM. But what I do know is this: every article uses it less than the previous. In fact, my article coming out later this month will be completely LLM-free. Not a single query was written for that article.

This Substack wouldn’t exist without LLMs which taught me things I should have known but didn’t (one paragraph ↔ one idea), corrected quirks in my writing (don’t hedge every sentence), and helped me structure articles so that arguments flowed. I am sure an actual editor could have taught me more, and one day I aspire to hire one. But ChatGPT is what I had, and it worked.

But this article is not about LLMs.

Today is the one-year anniversary of A Failure To Disagree, and so also marks the one-year anniversary that my friends no longer have to listen to me complain about how I wish I wrote more. And it is my improvement as a writer that I most want to celebrate. I am now writing things I couldn’t have a year ago. Not only that, but it is an enjoyable struggle. Writing and re-writing a sentence over and over again until the structure of the sentence enhances its meaning instead of hindering it can be cathartic. And while I am no G.K. Chesterton and never will be, I am no longer the whining-would-be-writer, and never will be again.

I wasn’t going to do a one-year anniversary post, but Selectionist (who is worth checking out) did one, and his goals and aims were so different from mine that it got me thinking about what I would say. And then having decided what I would say, I thought, “why not say it?”

My main goal

My main goal with this Substack is to poke and prod at misperceptions, confusions, and barriers which keep psychology complacent, and with a particular focus on behavioral science, decision-making, and expertise.

Psychology is operating at maybe 60% power. We’ve got some good stuff, but also a lot that is rotten which prevents us from progressing. I want to stick my finger into these festering wounds and make people take notice. I want those of us in this field to be a little more restless. Not just about individual findings, but about the foundational assumptions. Pre-registration and replications are great, but they won’t ever cause a scientific revolution.

There are consequences to adopting this goal.

First, what I write tends to dance on the edges of scientific understanding. I focus more on questioning the science than following it (ex. 1), and am as interested in unproven theories (ex. 1,2) as I am in well-replicated ones. I may even talk about theories I am unsure I even understand let alone endorse (ex. 1). The result of all this is that I will inevitably be wrong. The ideal error rate for this Substack is not 0.

Second, I think my audience size has a relatively small upper bound. Science communication is popular, but I am not doing science communication. Not really. My focus is more public sensemaking (ex. 1,2) with the goal of getting others engaged in the same activity. I once planned to do a PhD, and my wife told me that was a stupid idea, and she was probably right. So consider this Substack the best alternative I could think of. I am actively collecting people to create my alternative to academia.

Third, because I try to write for an audience that can (and should!) push back, I assume readers already have some context. I will occasionally write intro pieces (ex. 1,2), but even with those, my goal is to give context for later meatier discussions. I also try to be just provocative enough to draw these people in so that they will seriously engage with the ideas (ex. 1). I don’t want passive readers. The Substack is called A Failure To Disagree because that is the goal, but it is not the strategy.

Henrik Karlsson once wrote, A blog post is a very long and complex search query to find fascinating people and make them route interesting stuff to your inbox. My goal is not to dissimilar from that. Science is fundamentally social, and I am trying to create those social dynamics here.

The goal is not the purpose

But of course, I didn’t get into psychology because I wanted to change psychology. The main goal of this Substack is, at the end of the day, a side-quest to things more meaningful and important.

So occasionally when I have something to say, I will allow myself to deviate from my main goal and write something a little more personal, such as my articles about sorrow, fatherhood, and identity. (Fatherhood is great, btw)

Such articles are intentionally rare because I do not want subscribers who are chiefly here for such content. But I am human, and sometimes something pulls at my heart strings and says, “You should write about this,” and so I do. I hope they are valuable to someone out there.

I also exist off Substack, and have a page for podcasts I have done, and another for articles I have written which were published elsewhere. Some of that is more accessible science-communication-type-stuff, but I think you’ll find a lot of what I do is public sensemaking.

The future

I am pretty happy with how things are, but I do have two big ideas I want to work towards

• Adversarial collaborations

• Do science on the internet

I would love to become a hub where people come to publicly disagree with each other and put ideas to the test. Not antagonistically, but in a generative way.

Life is busy, and Substack isn’t a real job, so I may not be able to get around to doing these things in the coming year, but it is where I eventually want to go. If you are interested as a reader or collaborator, let me know!

Stats

Now for the fun stuff. Stats!

If you count this article, I have published 16 articles and one talk. I published 5 articles last October when I launched, skipped November, and then have done one a month since. That is a total of 51,421 words, for an average 3,213 words per article.1

My most popular article was What is Naturalistic Decision-Making with a total of 4070 views.2

My most underrated articles are my two launch articles: Stories as Context Sensitive Models (245 views) and ‘Games: Agency as Art’ — A Behavioral Scientist’s Book Review (153 views). I would love for people to poke and prod at and tell me if they disagree with them. Though I lost subscribers with my Little Letter Republic article, so maybe I should choose that one.

I currently have 657 subscribers. Unsurprisingly, my most accessible “introductory” type articles cause the largest jumps in subscriber count, while longer philosophical essays about the science of psychology don’t even show up in subscriber count.

I have subscribers from 33 states and 55 countries. Special shout out to India which will one day, I predict, be the behavioral science capital of the world. And also a shout out to Brazil who subscribed even after my hour long talk on the different decision sciences. And a special apology to Mexico (only 2 subscribers). How have I failed you? I lived in Chiapas for 2 years, and it hurts how much I miss it. No other foreign country is as close to my heart as Mexico. I will have to find a way to remedy this situation.

My biggest failure is that I still have not written about Simone Weil. I just can’t keep the article to a reasonable length. Basically my entire essay on fatherhood started off as just the intro for that essay before I decided it was too long and spun it off as its own thing. But rest assured, one day I will finish it.

The biggest success was an honorable mention from Adam Mastroianni of Experimental History for my article Psychology Experiments Are Gardens, Not Digsites. An article in which I tried to channel my inner Adam through introducing a couple of neologisms.

The thing which most inspires gratitude is the special few paid subscribers. I offer no benefits, and yet you gave me money anyways. That is a special kind of endorsement which I take far more seriously than the money itself. Ideally I would like to use this money to run experiments. I have a couple of (very) rough ideas around agentic modes, GeoGuessr, the sociology of decision researchers, inside/outside views, and a few other ideas related to expertise/decision-making which have been percolating in the background.

So here is to one year of Failure. And onwards to many more!

I chose that subtitle on a whim right before pressing publish. I regret doing that. Not because the subtitle was wrong or misleading, nor because I am opposed to click bait titles in principle (clearly not). But rather because I didn't follow through on explaining what I meant. That was pretty rude of me. I am a firm believer that if you have a click-bait title that you should fully follow through with it. I will do better here and convince you that even Daniel Kahneman would agree with my my title for this article.

This will not be a comprehensive critique of Cognitive Biases. I just don’t have space to talk about the theoretical1 and methodological2 issues. I am not even going to spend much time defending heuristics and intuition. Nor will I talk about the lack of imagination that researchers have on different ways they could study, conceptualize, and address errors in decision-making. Instead, I will focus on the relationship of biases to rational models, and whether we should care.

Biases don't exist

First, let’s agree on what heuristics and biases even are because it is something that is very easy to misunderstand.3

A heuristic is a rule of thumb: an easy to implement guideline for complex situations. Most people are going to be familiar with various prescriptive heuristics; measure twice, cut once; A bird in the hand is worth two in the bush; never pee onto a flat rock. All good advice.

The heuristics psychologists talk about are descriptive rather than prescriptive. That is, they describe how people do think instead of prescribing how they should think. For example, when deciding between two risky options, people tend to avoid the option with the more memorable risk. We call this the Availability Heuristic, and it is typically adaptive: if you can easily recall someone dying, then it is very likely a dangerous thing.

But sometimes this heuristic can lead you astray. The most famous example is the thousands of people who decided to drive instead of fly after the 9/11 attacks. Driving is vastly more dangerous than flying, and some have calculated that more people died after 9/11 from choosing to drive than people who actually died on 9/11 itself. This seems like a pretty major failure to understand probabilities. Even if there was a plane hijacking every week of the year, flying would still be safer.

When heuristics lead to mistakes from rational theories (such as from probability), we call it a bias. So instead of talking about the Availability Heuristic which is the cognitive process used that led to the error. We might instead say the Availability Bias, emphasizing how the heuristic led us astray from Probability Theory.

But here is an important point to take away from what I just described; biases don't exist. Biases aren't things in the world and they are not cognitive processes. Biases are an artifact of what we are measuring against.

Some time ago, Gary Klein, who had a friendly but adversarial relationship with Kahneman, was going to grab dinner with him and asked if I had any questions for the Nobel Prize winning scientist. I wrote up some hard hitting questions to ask, including extensive notes in my email to Gary so that he could fully understand why I thought the questions interesting.

But when the time came, the dinner conversation turned to other topics. It was a meeting of friends, not academics, and Gary didn’t want to disrupt the conversation with my hard-hitting questions.

So instead, after he got home, Gary forwarded my entire email to Kahneman, and Kahneman responded. I was both starstruck and mortified. Here is the opening of Kahneman’s letter.

Hi Jared,

You seem to think that biases are a thing, and that I study that thing and am committed to the existence of biases. If you read a few chapter of ‘Thinking Fast and Slow’, you will find that this description is incorrect. What we really try to study are the psychological rules that describe human thinking.

Ouch. Nothing like one of your idols telling you that you have misunderstood his work!

But here he confirms the first part of my click bait title; biases don’t exist. And I don’t mean that empirically we have discovered they don’t exist, but rather, by definition they do not exist. They are not the type of thing which have existence. They are like a correlation coefficient in that if you try to use a correlation coefficient to explain something then you have fundamentally misunderstood what a correlation coefficient is. Correlation coefficients and biases don’t exist in a way that they have causal force in the world.

When you decide to drive instead of fly because of a recent high-profile crash, we can use the Availability Heuristic to explain what happened. The heuristic has causal power in the world. But biases have no causal power. It’s like saying an arrow missed the target. The property of missed is not in the arrow, the bow or the archer. You cannot explain why the arrow missed by evoking a process called missing. That is circular.

Similarly, a bias just means you missed the target, and so can't be used to explain anything. But in this case, the target is what you would have done if you had appropriately applied various prescriptive theories such as Decision Theory, Bayes Theorem, or Probability Theory.

To recap:

Bias: When a heuristic leads us to make a different decision than a rational theory.
Rational theory: What some academics decided was the correct way to make a decision (sometimes also called a Normative Model).
Heuristic: Psychological rule we follow which explain the bias.

Jonathan Baron listed 53 biases in his book (4th edition). Wikipedia lists more than 2004. That is a lot of ways that people systematically miss the target!

But remember, for every bias there is an underlying target (a model) from which it deviates, and it is essential to keep that target in mind. Why? Because at some point you need to decide whether you actually care about hitting it.

Personally, I often don’t care about these models which is why I tend to define biases not as “the systematic ways that human reasoning and decision making deviate from rational models”, but instead as “the systematic ways that economic models are wrong about how humans reason and decide.” Same meaning, completely different emphasis.

Who died and made academics the arbiters of rationality?

I am sympathetic to the list of rational models that academia has developed over the decades and centuries. They are certainly good for some situations.

But these “rational” models are not capital-R Rational. They were not delivered to us on Mount Sinai, written on stone by the finger of God. Academia has not solved the problem of rationality, and it does not have the answer to life, the universe, and everything. If you live your life according to the maxim WWHED (What Would Homo Economicus Do), I think you are seriously hampering your ability to make good decisions, and will wind up believing and acting on very bizarre and irrational things.

This is partly a moral argument. I am not a utilitarian, and I have to admit that the fact that Jonathan Baron considers my moral philosophy a bias does rub me the wrong way.

But even more so it is an argument based on the the very premises of Heuristics and Biases. When Kahneman and Tversky set out to disprove the assumptions underlying economic theory, they didn’t intend to say that the rational theories that economists relied on were capital-R Rational and fully general decision-making formulas by which to live your life. The word ‘rational’ was a technical term which they got from Rational Agent Theory. Kahneman and Tversky were excited about biases because they falsified the idea that humans reasoned using rational models, and also provided a methodology for discovering how people actually reason.

This is why Kahneman has said that his work has been misinterpreted as an indictment of humans when that isn’t really what his work showed. He goes on to clarify that the term “rational” is a technical term and that he hates the word “irrationality” and has never used it.5

And there it is! My clickbait title has been fully endorsed by the man himself; biases don't exist, and humans are not irrational.

Now with that context, here is Kahneman’s full letter.

Hi Jared,

You seem to think that biases are a thing, and that I study that thing and am committed to the existence of biases. If you read a few chapter of ‘Thinking Fast and Slow’, you will find that this description is incorrect. What we really try to study are the psychological rules that describe human thinking. We often illustrate these rules of thinking by showing they lead to characteristic mistakes. We do so not because we are interested in mistakes for their own sake, but because correct thinking is the default case, which requires no explanation6. This is not a universal rule, but it is often the case that one can learn a lot about a system by examining circumstances in which it fails. For example, memory is studied by looking at forgetting.

All best,

Danny

Biases are not the object of study, they are a methodology for understanding the object of study (i.e., heuristics). They are a research tool, not a diagnosis. Just as how you might study forgetting in order to understand memory, or visual illusions in order to understand perception.

But as Kahneman himself has noted elsewhere, the medium became the message. Rather than studying biases to understand heuristics, many researchers began to study biases in order to prove human were irrational. And despite himself, Kahneman seemed very interested in this work.

And so perhaps my title was click-bait after all. Despite explicitly claiming otherwise, Kahneman seemed interested in “mistakes for their own sake”, and did seem to think people were irrational.

And that is my principal disagreement with Kahneman and the entire Heuristics and Biases approach. Biases may be a good tool for studying heuristics in a lab, but I do not think they are a good tool for studying human errors in real-life situations. The principle reason for this being that despite all the hubbub around these models, they are not actually rational in the vast majority of real-world settings.

You would think there would be lots of empirical evidence about how the best decision-makers all use rational models. But yet the literature is lacking on this point, and the literature that does exist tends to show the opposite. It is not the best of the best, but the newest of the new who use rational models; novices will sometimes use rational models because they have so little knowledge of a domain. But as novices gain experience and start to understand and appreciate concrete specifics, they put away childish things and start making decisions based on a deep understanding of the context.

Similarly, you would think there would be lots of literature on how biases lead to many real-world errors. And indeed there is. But most of it is post-hoc; identifying errors and then describing the error as a bias as opposed to focusing on the underlying process which caused the mistake. Worse, some will even claim the mistakes were caused by a bias as if biases were the type of thing which have causal power.

I find this type of research extremely dangerous. Yes, heuristics can underlie various types of errors, but they also underlie expertise, adaptive cognition, and the entire learning process. Blaming the very thing which facilitates excellence is perilous and I do not think current research practices have adequately addressed this.

But maybe you are not convinced that rational models are not capital-R Rational, and the moral argument doesn’t bother you. Well, let’s get even more specific then.

No Free Lunches…well, almost none

Even by their own standards, the rational models are not always rational. For every bias, there is always an exception where the thing that has been declared “irrational” is actually optimal.

This is proven by a set of theorems called the No Free Lunch Theorems, which are simultaneously both trivial and controversial. The theorems state that no model can be optimal across all possible situations; for any model that says A is optimal, there is a possible world where B is actually optimal. Because of this, it's logically impossible for a model to be optimal in all possible worlds.

If you take the theorems seriously, the takeaway is this; you cannot always rely on the same model. There are no free lunches when it comes to rationality. You have to put in the work not only to figure out what is optimal, but to figure out which model can get you that optimal result. Why else do you think we have things like Fuzzy Probability Theory and Non-Aristotelian Logics? Because there are always exceptions where the traditional model doesn’t apply. Remember, all models are wrong! Yet somehow we are supposed to believe there is some set number of models which are the standard for rational decision-making? Do not mistake the entirely of reason with the single field of statistical inference!

Some people feel this in their bones and get anxious when rational models are naively applied. But others think to themselves, “Well maybe there are some exceptions, but by and large the traditional7 rational models will work for most situations and we can easily identify the exceptions.”

When I hear these people, I swear I can almost hear the roar of the T-Rex from Jurassic Park.

Consider two examples.

First, Confirmation Bias, one of the most popular and supposedly well replicated of the biases. Everyone knows that Confirmation Bias is irrational, right?

Well, no. When uncertainty is high, confirmation isn’t only not irrational, it can be optimal as it can be more informative than falsification. This isn’t to say Confirmation Bias is always rational. But do Behavioral Scientists really fully understand the types of situations where it is and isn’t? Should we de facto label all instances of Confirmation as bias or irrational? Had you even considered asking when it might be rational and optimal?

Another example is Expected Utility Theory: a theory so popular that some people base their entire moral philosophy on it. For fans of the theory, Jason Collins has a bet for you: “Suppose you have $100 and are offered a gamble involving a series of coin flips. For each flip, heads will increase your wealth by 50%. Tails will decrease it by 40%. Flip 100 times.”

Do the math. Expected Utility says take the bet.

But common sense says that I am likely playing a trick on you, and you should be careful.

Turns out the bet is non-ergodic, which is a term I barely know how to pronounce let alone describe, so read Collin’s full article here. But the takeaway you need to know is this; if you calculate expected utility it looks like a great bet, but you will probably wind up poorer than you started if you accept the terms.

I use non-ergodicity as an example not in spite of its mathematical complexity but because of it. Had you ever heard of the term? Do you think the average Behavioral Scientist going around declaring people biased has heard it, internalized it, and can tell when they are in a non-ergodic situation? If not, are you sure they are sophisticated enough to know when rational models should and should not be used?

A bias is, by definition, a deviation from a rational model. But for every given rational model, there are times when the bias is more optimal than the model. And frankly, I do not trust people to know when those exceptions apply.

Some people come up with work arounds to this criticism, saying that biases are only biases when they are actually irrational. If Expected Utility doesn’t give you the “correct” answer, then it is not a bias to deviate from it. I find such reasoning very circular and dangerous. The bearers of rationality are no longer unbiased equations but researchers. This leads to situations where a behavior previously considered irrational is later declared to be rational because researchers finally understand the logic of it. I find this problematic. Declaring something irrational shouldn’t have such a high correlation with the ignorance of researchers.

So specific you can only find it in a lab

But maybe there are some domains where these exceptions don’t apply? Maybe there are entire areas of problem solving where rational models are, in fact, capital-R Rational?

Well, it can’t be any quick paced domain as rational models are too slow, laborious and complex. Certainly, for the domains we care about at ShadowBox where I work the rational models just don’t cut it.

We can also ignore domains which are too simple, as such models will not be needed anyways. You don’t need Expected Utility Theory to pick out the right cereal at the grocery store. Similarly, we should ignore domains where intuitive expertise is likely to develop and perform as well as the rational models. Especially since using abstract rational models can get in the way of learning in such domains.

Ditto for domains where values are not clear, stable, and well-ordered. If you are still trying to figure out what you want, or your values are too dynamic, then rational models will just fail you entirely.

It also can’t be a domain where there is too much non-linearity or too many unknown unknowns. In domains which we call complex or chaotic, the list of rational models will lead you astray and will get in the way of the sort of probing, sensing, and acting that you need to do in order to make sense of what is happening.

So, we need a domain that is slow and complicated in a very particular way; you can’t develop expertise, relationships are linear, values are stable and well-ordered, and you have all the information but can’t quite compute it.8

That is a very narrow slice of life! In fact, the only domain I can think of that meets all those criteria is lab experiments which are purposefully built to have these criteria so that people are forced to rely on flawed heuristics. And perhaps that is the fundamental issue with the Heuristics and Biases approach! We use lab experiments to evoke biases to discover how humans reason. But real life isn’t like these experiment, and so generalizing the mistakes to other domains is extremely theoretically and ethically fraught.

Unfair? Let me relax the constraints a bit. Surely there are situations such as stock picking or internet debates where rational models are helpful?

I guess? But they are hard to identify. Even in domains where you should be using models, you typically don’t use the default rational models, but instead something much more context specific. But sure, there are Goldilocks Zone, like Tetlock style forecasting and maybe some others, where I would recommend using Bayes Theorem. My point is not that the rational models are useless. In fact, I would recommend everyone spend some time learning probability, Expected Utility, Bayes Theorem, and logic. These are really great tools for thinking clearly!

But as you learn them, recognize that these are not general purpose tools. The situations in which rational models should be used for decision-making are far fewer than many assume. Rational models are not the default right way to reason, but a very exceptional use case in extremely constrained settings. More useful for post-hoc explanations of a decision than for making a new one.

In a now famous debate between the economists Richard Thaler and Kevin Binmore, Binmore argued that intuition becomes more rational over time as people learn from their mistakes, a point I agree with. Thaler pushed back saying that might be all well and good for grocery shopping, but there isn’t enough feedback for major life decisions like marriage or retirement. Thaler claimed victory with a quip: Binmore’s highbrow theories about the rationality of intuition were only good for “buying milk.”

It is a clever come-back. But yet, I wouldn’t recommend choosing a wife or husband based on Expected Utility Theory. Certainly, I wasn’t trying to avoid biases when I decided to marry my wife. I was trying to construct the future I wanted for me and my kids. Such qualitative as opposed to quantitative approaches still strike me as vastly more capital-R Rational than the rational models that are so popular within academia. And not just for small decisions, but rather, even more so for life’s most important decisions.

Conclusion

So is there anything left to salvage of biases?

Maybe. I sometimes use the term “Natural Bias” when talking about the current craze around “all natural” products and the aversion to chemicals, GMOs, and vaccines. Such thinking seems to me systematically flawed and confused. To be opposed to chemicals is just non-sensical, and vaccines and GMOs have saved billions of lives. All hail Norman Borlaug! 500 million, but not a single one more!

But of course, from the perspective of the those skeptical of scientific consensus, I am the biased one.

And that right there is the difficulty of the whole thing. If you don’t have an objective definition of rationality that everyone can agree on, and that can be applied perfectly and without fail by researchers, then you’re not doing research, you are simply imposing your own standards and calling it rationality.9 And even when you are right, this is a dangerous thing. More rhetoric than science.

This is why I cringe when the word cognitive bias is used. Not because the term is useless, but because misuse and misunderstanding are so pervasive, and I’m not sure the concept of biases serves a useful function outside of the lab. As a cognitive scientist, I would much rather talk about the actual processes underlying cognition than trying to explain human behavior by guessing what the mythical Homo Economicus would have done in the same situation.

In 1985, the US military wanted to understand how to train decision-making when they ran into an little snafu; the literature on the topic was useless.

Imagine you are in a complicated battlefield situation, and you have to use Bayesian Decision Theory to calculate the best course of action in the next 2 minutes. Good luck!

Based on experience (and a bit of common sense) the military concluded that real-world decision-making couldn’t work like the slow and cognitively taxing theories that were so popular in academia. But the alternative wasn’t clear either. If great generals and tacticians weren’t using these academic models, how were they still making such high quality decisions under extreme uncertainty and time pressure?

The Army decided to fund research into decision-making in quick paced and highly uncertain domains to find out. One of the teams that won the contract was led by Gary Klein1.

Klein’s team knew they wanted to study experts in real world conditions outside of the lab, but tagging along with the army on the battlefield was out of the question. They needed a population of experts they could study that wouldn’t put their lives at risk. They decided on firefighters. Not only were firefighters experts in quick decision-making, they also had the advantage of lots of down time at the station between the fires when they could be interviewed. They seemed like the perfect subjects.

But during the very first interview with a firefighter, not even a real interview but a practice one, they ran into their own little snafu. Klein recounts the conversation:

I said, "We're here to study how you make decisions, tough decisions."

He looked at me, and there was a certain look of not exactly contempt, but sort of condescension, I would say at least, and he said, "I've been a firefighter for 16 years now. I've been a captain, commander for 12 years, and all that time I can't think of a single decision I ever made."

"What?"

"I don't remember ever making a decision."

"How can that be? How do you know what to do?"

"It's just procedures, you just follow the procedures."

My heart sank, because we had just gotten the funding to do this study, and this guy is telling me they never make decisions. So right off the bat we were in big trouble. Before I finished with him, before I walked out, I asked him, "Can I see the procedure manuals?"

Because I figured maybe there's something in the procedure manuals that I could use to give me an idea of where to go next. He looked at me again with the same feeling of sort of condescension, (obviously I didn't know that much about their work) and he said,

"It's not written down. You just know."

There are many moments one could choose as the birth of Naturalistic Decision-Making, but I would nominate this conversation. It was the first major insight into how experts make decisions in naturalistic settings; to them, it doesn’t feel like a decision at all.

Klein’s team had assumed firefighting would be so quick paced that firefighters would deliberate between maybe only 2-3 options at most. But instead they found they didn’t deliberate between any. Which isn’t to say what they did was easy, as they still regularly ran into situations where they didn’t know what to do. But the “generate and compare options” paradigm that academia taught wasn’t how real world decision-making happened.

Around the same time, other researchers in similarly high stake domains were discovering the same thing. In 1989, these researchers gathered in Dayton, Ohio not realizing what they were staring. At that first conference, they hadn’t yet conceptualized themselves as a distinct field of study, and they certainly didn’t have a name. But at that conference a few themes emerged

1. They were interested in complex, real-world (naturalistic) environments characterized by time pressure, uncertainty, ill-defined goals, high personal stakes, and other intricacies. Fields like firefighting, law enforcement, tactical decision-making, medicine, aerospace, etc.

2. They were interested in experienced experts who consistently performed well despite complexity. Not novices, and certainly not undergrads.

3. How people made sense of situations often mattered more than deliberating over predefined options.

The third theme stands out a little. While the first two themes identify similarities in what and who they studied, the third was more an insight that many of them had independently discovered, and which Klein had started to pull the string on in that initial practice interview with the fire commander.

Later, the field was christened Naturalistic Decision Making (NDM). That Dayton meeting was just the first of many2.

How to make a decision without deciding

It wasn’t just that one fire commander who didn’t make decisions; they all said the same thing. The researchers actually stopped using the word “decision” and started asking about “tough cases” just so they wouldn’t have to retread the same ground over and over again. Later this approach of asking about tough cases got systemized into the Cognitive Task Analysis (CTA) method, which we still use today.

What this proto-CTA helped reveal was that when the firefighters insisted they didn't make decisions, what they meant was that they were not deliberating between options. Instead, the situation itself implied what to do.

Consider a fictional Search-and-Rescue (SAR) story about a missing hiker.

At first, the SAR team assumes he took a wrong turn, and so they bring dogs to each junction on the trail to see if they can pick up a scent where he might have gone astray. But the dogs pick up no scent.

Next, they assume he went off train for a shortcut. They search along the sides of the trail, looking for broken branches and footprints on the edges of the trail, but still nothing.

At this point, they start to question whether he was ever on the trail in the first place. They talk to the family, and his mother says something interesting, “I don’t understand how he got lost, he just got a new GPS watch. We tested it the other day.”

Ah. Well if he was testing a GPS, he probably wouldn’t have entered the trail at all! The SAR team tries a new search pattern, looking at straight line routes between the start and end points of the trail. And along one such straight line route, they find him. Stuck in a ditch and unable to move.

What I like about this story is that each theory implies how to search: wrong turn → check junctions; shortcut → check trail edges; direct navigation → follow straight lines. The challenge isn’t figuring out which search technique to use. That is always obvious because how he got lost implies the way to search. Instead, the difficulty is in understanding how he got lost in the first place. In other words, and to paraphrase that third theme, the ways in which the SAR team made sense of how the hiker got lost exerted greater influence on their actions than did deliberation over a set of predefined options.

While I do not want to diminish the expertise of SAR teams, I use that domain as an example because it is intuitive to even non-experts. There is an “if X, then Y” relationship between the situation and the action to take. Of course, in many complex domains the relationship between “X” and “Y” is much less obvious. And even knowing the right questions and actions to take to figure out which “X” situation you are in requires quite a bit of experience. But this dynamic is why it feels like protocol instead of decision-making.

Klein’s team formalized this in the Recognition-Primed Decision (RPD) model which explains how recognition of the situation primes the appropriate action. The RPD model has become the dominant way to understand decision-making among experts across every highly uncertain and quick paced domain you can think of. If you want to know more, you can read my essay on the RPD model here, or the original (updated) article here.

Flawed intuitions

Kahneman and Tversky may be more familiar names than Klein, and readers may notice a tension between these thinkers. And indeed there is. The Heuristics and Biases (HB) programme that Kahneman and Tversky founded was based on the idea that intuition is riddled with systemic errors that they called biases. These biases are so persistent that Kahneman doubted they could be mitigated, and he instead preferred to replace humans with algorithms where possible.

As you may imagine, this is the exact opposite of Klein who is fascinated by stories of experts doing amazing things. Consider a famous story that Klein likes to recount.

During an interview a firefighter claimed to have ESP, and Gary asked him to tell him why he thought he had this paranormal ability. The firefighter recounted a situation he couldn’t explain. During the incident in question, they were in a burning house when he got a bad feeling. He didn’t know why he had the bad feeling, but it was strong enough for him to act. He ordered his crew out of the building, not knowing exactly why he gave the order.

As soon as he and the team were out of the building, the building collapsed. Turns out the fire was in the basement (which they hadn’t known about).

To get to the bottom of the story, Klein used the CTA interview method and drilled down into the details of the situation. He was able to help the firefighter to recount various anomalies that he had identified during the course of the incident which had triggered the bad feeling; a fire that was too hot, too quiet, and not as responsive as it should be. While each anomaly wasn’t enough to trigger his conscious mind to realize where the fire was, it was enough to make him doubt that he understood the situation fully, or perhaps to recognize the signs of a fire that was beneath him rather than in front him (it is difficult to know which). Through this subconscious detection of important cues, he was able to intuit the danger of the situation. No ESP required.

This is not the only incident where an expert in a domain came to believe they had ESP. Experts often struggle to explain what they do because so much of what they know and do is tacit. We have heard terms like sixth sense, spidey sense, or even magic as experts try to explain what they do. Mostly these terms are used tongue in cheek, but some do come to believe there is something mystical happening due to the difficulty of explaining their own tacit knowledge.

So how do we square this circle? How do we make sense of intuition that is so brilliant that it seems like ESP, and at the same time make sense of intuition so flawed that finding those flaws led to Kahneman’s Nobel Prize?

That was the question that Kahneman and Klein set out to solve when they decided to collaborate on a paper.

Kahneman had long been an advocate of “adversarial collaborations” between researchers who disagreed. He has done a few over the years, but the one he did with Klein he described as his “most satisfying.” It took them 6 years to write their paper, and many hoped they would fail (such was and is the bad blood between the two fields). But fortunately for all of us the two got along famously, and ended up writing “Conditions for intuitive expertise: a failure to disagree”. The very paper which inspired the name for this Substack.

Klein has always disliked the name of this paper because they actually disagreed quite a bit! But nevertheless, they did agree on some basic facts which they summarize at the end of their paper, and which I will try to summarize in one sentence;

Intuition starts flawed, but can be trained to be adaptive in environments with high reliability and rapid feedback.

The reason Kahneman and Tversky found so many flaws is that they were focused on non-experts (undergrads) in artificial and novel domains (a lab experiment). In situations like that, the subjects don’t have time to develop expert intuition. Conversely, Klein’s subjects had decades of experience and training, and so their intuition was highly adapted to their domain.

Despite this ‘failure to disagree’, the fields to this day are in tension. NDM researchers believe the post-hoc analysis of biases so common among HB research can be extremely damaging as it often dismisses the very intuition that enables experts to succeed in the first place. Meanwhile, HB researchers view NDM with skepticism due to its emphasis on its qualitative methodologies, and its rejection of the more systematic approaches that HB tends to prescribe.

The two ended their article with an invitation that has yet to be fulfilled: “[Daniel Kahneman] is still fascinated by persistent errors, and [Gary Klein] still recoils when biases are mentioned. We hope, however, that our effort may help others do more than we have been able to do in bringing the insights of both communities to bear on their common subject.”

So what is expertise?

A common misperception is that expertise is just pattern matching and intuition (“System 1”). But that is too simple a story. A pattern that is relevant in one situation may not be relevant in another. The heat and loudness of the fire, the position of a rook relative to a castle, the way a victim carefully words what they say…these patterns are only relevant in certain contexts.

Another important thing to note is that even though it's not transferable, expertise is adaptive. What I mean by that is that expertise in, say, Chess, does not transfer to Go, but it does adapt to novel chess boards a grandmaster has never seen before. This also cannot be explained by a story of simple pattern matching.

I think of it this way; there is not a library of pre-existing patterns the expert is trying to match to, but instead, the expert’s mental model of the situation helps to generate patterns to look for. So the key to expertise is not primarily the pattern matching, but the effective mental models and mental simulation that allow the expert to predict the patterns they should see, as well as to notice when things are not developing as they expected.

There is a bit of circularity here that is worth pointing out; the situational understanding shapes the patterns the expert looks for, but the patterns they notice shape their understanding. It seems like you can’t know what is relevant until you know what is happening, and you can’t know what is happening until you identify what is relevant. This relevance realization problem—knowing what matters before you know what’s happening, and vice versa—is, to me, the heart of expertise.

How do experts develop this ability? The easy answer is “through experience,” which is correct but incomplete. Sometimes someone with years of experience can still be a dunce. It has to be the type of experience that helps to build up one’s mental model of the situation. That requires an active mindset where they wrestle, think through, and predict, all while getting feedback, whether real or simulated. True experts are constantly pushing themselves in ways that others do not.

Conclusion

What I have covered here is rather paltry compared to the richness and depth of NDM. The field has covered so many fascinating domains, from astronauts, to fighter pilots, to military. I also said very little about CTA, and didn’t even mention some of the most important theories such as the Data-Frame Theory of Sensemaking and Macrocognition.

But hopefully what I HAVE said is enough to at least intrigue those who are unfamiliar. I personally think NDM is the most important work happening with regard to how humans actually make decisions, and so it is important from both a theoretical and applied point of view. I wish more people would dig and discover its depths!

However, I have said a lot. So I will end with a final thought from Daniel Kahneman.

“Klein and I disagree on many things. In particular, I believe he is somewhat biased in favor of raw intuition, and he dislikes the very word "bias." But I am convinced that there should be more psychologists like him, and that the art and science of observing behavior should have a larger place in our thinking and in our curricula than it does at present.”

There is a particular type of LinkedIn post that makes me cringe. Well, actually there are many LinkedIn posts that make me cringe, but there is one in particular I would like to focus on.1

These posts always have the following form: "Look at what this company is doing. They clearly don’t understand Behavioral Science. They’re doing X when they should be doing the exact opposite!"

The classic example, which I have seen multiple times, is a post about how Wikipedia asks for donations by emphasizing how FEW people donate. Behavioral Scientists see this request and conclude it is wrong because Social Proof clearly states that you are supposed to emphasize how MANY donate.

Wikipedia had so many people writing to them about this “mistake” that they actually released a statement saying something to the effect of, “Shut up! We tested it. This works better.”

Maybe my nerves have over-updated on this one example, but every time I see a post similar to these criticisms I wince a bit. Maybe some of them are justified, but it seems more likely the company ran a test and found the opposite of the famed effect than that the person criticizing the company has additional insight into the context. Just as in the Wikipedia example.

However, it’s not the overconfidence that bothers me so much as the entire theory of psychology underlying these posts.

And maybe I am weird in the way I think about it—I suspect this post could be quite controversial—but I don't think psychological effects are properties of the human condition that we have discovered. Rather, I think they are something we prompt through the context of the experiment. More like flowers cultivated in a garden than fossils at a digsite. That’s why we call it choice architecture and not choice archaeology, and behavioral design instead of behavioral excavation.

So here is my argument: stop thinking of psychological effects as truths we have dug-up and discovered about human nature, and instead think of them like flowers which grow in certain contexts and not others.

Once you see psychology in this way, it becomes hard to take these confident LinkedIn takes at face value. Not only that, but the current ailments in psychology (the twin crises of replication and generalization) become much easier to understand. Because if this view is right, then the question isn’t whether Wikipedia violated some universal human principle, but about whether Wikipedia readers see “social proof” in the same way as the subjects in our lab. (Hint: They do not)

II. Gardens not digsites

Just because you use a trowel in both gardens and digsites does not mean they are the same.

At a digsite, such as ancient Mayan ruins or a Cretaceous era fossil bed, Archeologists and Paleontologists dig to find what was already there without regard to the current conditions of dirt, sun, and moisture. In fact, the current conditions are a nuisance to be removed as quick as possible so that the ancient and immutable truths can stand on their own without being disrupted. Such ancient truths cannot be changed by context, only obscured by it.

By contrast, a gardener isn’t uncovering hidden truths about a seed. The plant isn’t ever the mystery to be solved; the conditions necessary for growing the plant are the mystery. In some conditions the plant will grow tall and fast, in another short and slow, and in yet a third, it will wither and die. The question is always about how to shape the context of dirt, sun, and moisture to get the desired response.

So, by metaphor, the question is this: when we observe a psychological effect in an experiment (e.g., Loss Aversion), are we unearthing some fundamental property buried in the human mind? Or are we crafting the context in ways that get us consistent reactions?

Let’s return to Social Proof as a well replicated example. The idea is that if you emphasize how many people do a certain behavior, the person to whom you are emphasizing this fact will be more likely to do that same behavior. This effect is so common that it is basically a slogan for elementary kids; “everyone else is doing it.”

But the thing is, it's not universal. Sometimes people resist the crowd. Sometimes the fact that something is popular is exactly what makes it suspect. Remember, hipsters exist. And just about everyone has a little hipster in them.

If Social Proof were like a fossil buried in the ground—a fixed and immutable structure—we’d expect it to show up reliably. But it doesn’t. It varies wildly by context, culture, subculture, framing, motivation, salience, the perceived intentionality of the message, etc. Social proof is less like a tibia and more like a tulip. It is not something we dig up and discover about the human condition, but is instead a behavior that grows in certain conditions but not in others.

Loss Aversion, Default Bias, Fresh Start Effect, Sunk Cost Fallacy…I don’t believe any of these findings are inherent to the human condition. They are not in the brain ready to be discovered. They are simply what happens when someone perceives the context in a certain way. These effects can disappear or even reverse given the right context. Just consider the opposing nature of Primacy Bias and Recency Bias, or of Status Quo Bias and Novelty Bias. Or compare Loss Aversion to the risks we take when we gamble, or those taken by adrenaline junkies. All these effects that psychologists find and label are not laws of human nature that we have discovered. They are not the default human condition independent of any context. They’re just what crops up when we manage to craft the context in a very specific way such that people see the pay-off differently.

Of course, to argue against myself, it is true that some variation is consistent with the digsite approach to psychology. No one expects psychological laws (if such a thing were to exist) to be immutable. But how much variability does that approach allow before we should reconsider whether they are laws at all? How far can such a view be stretched before it breaks?2

When I look at experimental procedures, I see tightly controlled conditions that we designed. Less like a digsite than a manicured garden with carefully planted cues and instructions. Or to use another metaphor, an experiment is more like a stage on Broadway than a half excavated amphitheater in Greece. And it makes sense to me that we see consistency in such a situation; but not the kind of consistency you get from a physics experiment. Instead, it’s the kind of consistency you get when you give an actor some dialogue to read and put them on that Broadway stage. I promise you they will perform! If you set up the conditions, you can control how people perceive the situation in which they find themselves and get consistent reactions. But that doesn’t make The Script Reading Effect a fundamental property of human nature.

With the replication and generalization crises as bad as they are, perhaps it is time to reconsider the idea that psychological effects are deep human truths, and instead recognize that they are artefacts of how someone sees the context. i.e., their frame. These frames can be cultivated by the experimental design, and the shared history of the participants, but they are not fundamental properties of the brain. A study doesn’t reveal “what people are like.” It only reveals what people do under the conditions we designed. As Avel Guénin–Carlut puts it: “Psychological effects should be understood as brought about by the experimental context rather than revealed properties of human cognition.”

III. Psychological Effects Are Frame-Dependent, Not Fundamental

Take this real-life example: a friend recently told me his child lost a helium balloon and was still feeling down even after being given a new one. The child, confused, asked, “Why am I still sad even though I have a new balloon?”

My friend passed the question on to me, adding, “Don’t just say Loss Aversion.”

I would never dream of it. I think it is absolutely silly to assume that Loss Aversion explains much if anything at all, and even sillier to assume it applies to a particular child. The child could have anthropomorphized the balloon and felt like they lost a friend. Or maybe they felt responsible and ashamed. Or maybe the child really just thinks losses are worse than gains! Each is a plausible answer that could apply to some children some of the time. But none of these frames are universal. Each depends on how the child sees the situation, not on some universal principle of human cognition that exists independent of the context.

For example, if a parent says, “This is Bozo the Balloon, take good care of him,” or if the parent says “Don’t lose it or I’ll beat you with a belt,” the child will be upset when he loses the balloon, but for very different reasons. The presentation completely changes the perception and therefore the experience. The emotional landscape the child inhabits is shaped by so many factors that it would be a fool's errand to conclude that it is just a single effect (loss aversion) that dominates the reaction of all or even most children.

Some might argue that these two examples (the anthropomorphization of the balloon and the threat by the parent) are somehow not neutral cases, and that the child has been primed to see the balloon in a certain way. They might argue the context is getting in the way of understanding the child’s true reaction to a lost balloon. With the typical digsite mentality, they think that such context needs to be ‘dug away’ and controlled for as it can only obscure the deep human truths which is a child’s natural reaction to losing a balloon.

But this is my entire point. There is no default relationship with the balloon independent of the context. There’s no neutral way to relate to anything. Every relationship is a frame, and every frame is shaped by the child’s context. Whether the child treats the balloon like a friend, a responsibility, or a meaningless object will determine their emotional response, and as a result, their emotional response doesn’t reveal something universal about cognition, but rather their emotional response reveals something about the frame they’ve adopted.

There is no escaping context and so all behavior must be understood in light of that context. When studies provoke consistent behavior, it’s not because they’ve uncovered a universal law of human cognition. It’s because they’ve built a reliable garden that is able to grow certain responses with consistency. The experiment is shaping how they think, rather than revealing how they think.

IV. Scaling it back

Now that I have (hopefully) made my point clear, let me try and scale it back.

This doesn’t mean there are no human universals, and nothing ever which we can “dig up” about the human mind. But a capacity, such as the ability to anthropomorphize, while being something we can discover, is not the same as a behavior which is something we evoke in an experiment. What matters in most experiments isn’t whether someone can see something a certain way, but whether we can get them to see it a certain way. And that depends on our ability to design effective interventions which cultivate certain frames.

Consider an infamous failed replication. In the original study, researchers found people were more likely to buy French wine when French music played in the background. However, a follow-up study found no effect.

Would a replication renew your faith in the finding? Would a second failed replication consolidate your disbelief?

For me, it is neither. Obviously French music can influence someone if it successfully activates a patriotic, nostalgic, or cultural frame. But also just as obvious is the fact that it won’t work on everyone—they might not recognize the music, might not care, or might even resist the perceived manipulation. I don’t even need to run the experiment to know that both results could happen given the right experimental design and the right population. Either result is trivial.

Once you see experiments as tests of our ability to consistently cultivate certain frames, that is as gardens instead of digsites, replication and non-replication often become rather trivial. The experiments we run are tests of whether the garden was strong enough to provoke a certain frame consistently. Turns out many of them aren’t. Boohoo. So sorry for psychology’s loss, whatever it was.

Which brings us back to my original Wikipedia example; is it surprising that sometimes social proof (or any other psychological effect) works and sometimes not?

Not at all. Social Proof isn’t a psychological law we discovered, but a tool for evoking a certain type of frame. In some contexts it is reasonable to do what others are doing, and those are situations where we should expect Social Proof to work. But in other situations, such as when Wikipedia readers are grateful for an underfunded resource, then an emphasis on how few people donate can be a more effective strategy for getting them to loosen their purse strings.

I think this is why so many behavioral change interventions can feel more like common sense than a cognitive trick. More “the subjects finally did the thing because we made the thing easier to do”; and less, “we exploited a brain module which misfires under specific conditions.”

In fact, to all you Behavioral Scientists out there, I would say that if you haven’t found a way to understand the effectiveness of an intervention in a way which feels like common sense, then you probably haven’t understood your own intervention. Behavior change isn’t voodoo or subliminal messaging. Behavior change techniques should be intelligible even to someone who doesn’t have a degree because behavior change isn’t about manipulating brain modules, but about shaping how people see the context.3

Am I taking this view too far again?

Consider the following possible claims:

1. People are always loss averse.

2. People, by default, are loss averse except under extraordinary circumstances.

3. People tend to be loss averse.

4. People will be loss averse if you manage to successfully get them to frame the situation in a way that makes losses seem worse than the gains are good.

By my reckoning, claims 1 and 2 are false, and if they were true we likely wouldn't need an experiment to confirm them. Claim 3 is so vague that it is unfalsifiable unless we can somehow find a representative sample of all possible choices. And finally, claim 4 is true. Claim 4 may even sound trivially true at first glance, but I do not think it is trivial what Kahneman and Tversky found in (for example) the Asian Disease problem. Treating mathematical identical choices differently is indeed an interesting finding.

However, somehow the classic finding is almost always interpreted as being about claims 1-3. This results in people on LinkedIn saying, “McDonald’s emphasized that you can get an extra toy if you buy two Happy Meals. But they should have framed it as losing two toys if you only buy one!” Completely ignoring the the rest of the context which would make a Loss Averse framing less effective in that situation.

So, in a sense, yes. We dug and discovered Loss Aversion. But having discovered that it can be the case that people can treat mathematically equivalent results differently depending on the frame, the question has now shifted not to whether Loss Aversion replicates, or even how common it is. But it instead shifts to be about whether we can design experiments that reliably reproduce the results if we so desire.

Choice Architecture, not Choice Archeology.

Behavioral Design, not Behavioral Excavation.

Gardens, not digsites.

V. Let go of the bones

If I have overstated my case, it is intentional. This is at least my 7th iteration of this essay, and the only way I could make it work, it seems, was to scream into the microphone. As I have said, I do think there are things we can discover and dig up about the human mind, and I am perhaps a little nervous in taking this argument too far and saying there is no module-like entities we can discover.

But I do believe that the strongest impact on how people behave is how they perceive their situation, not the various latent constructs of tendencies and effects which we find in psychology textbooks.

Not recognizing this can be dangerous as we begin to reify experimental averages as if they has some material reality. We call the average in an experiment x̄ (X-bar). But x̄ isn’t real. It is not a module in the brain. It doesn’t generalize, and even replication can be pretty shaky. A psychology based on x̄ will be perpetually in crisis. Don’t do x̄-Psychology. Don’t reify x̄ into some property of the human brain. x̄ is a result grown in a particular context, not a cognitive process you can discover.

Sometimes the job of psychologists and behavioral scientists is to excavate the human mind and find fundamental principles. But other times we’re just gardeners trying to find the right set-up to promote a certain kind of growth and reaction. Maybe part of the problem with the current psychological paradigm is that we are very bad at telling when we are doing which.

What makes humans distinct from all the other animals in the animal kingdoms is that we are the most flexible. Not physically—that distinction probably goes to our liquid feline friends—but in how we adapt to any environment. We are not blank slates, but we’re the most blank slate of any animal. Other animals are not deterministic, but they’re more deterministic.

This flexibility and adaptivity is made possible because of our ability to take advantage of our context. Humans don’t ignore the environment, nor are we merely resilient to it. Instead, we integrate it into everything we do.

This is great news for humans. But it is also terrible news for psychologists. How do you study something as adaptive, and therefore as context-sensitive, as humans?

In Part I, I argued that what we have left something out of our study of psychology—namely context. And this context is flooding back in and disrupting our field. I further argued that if we want to develop a truly paradigmatic psychology with robust theory capable of replication and generalization, we need to expand the boundaries of our discipline and stop treating context as something separate from the subject we study, but as core to it.

Consider physics. A physicist might run an experiment on how billiard balls bounce off each other, and they will model this system using variables such as angles, weight, friction, force, etc. However, in that model, the physicists would probably not include a term for the gravitational pull of Jupiter. Sure, the gravitational pull is interfering with the experiment in some minuscule way (it is “leaking” in). But not enough to affect the results or invalidate the model. So they treat it as irrelevant context.

But keep in mind, the gravitational influence of Jupiter is still the type of thing physicists study. They might ignore it in a given experiment, but that is a temporary disinterest, not a declaration of irrelevance. When the gravity of Jupiter becomes significant, physics has theories to account for it. It is still physics.

I argue psychological context should work the same way. If something influences human cognition and behavior, then it should be the type of thing psychologists study. We might ignore it temporarily within a particular study or subdiscipline, but that shouldn’t imply permanent disinterest. Our theories of context need to be just as strong as our theories of reasoning. Or better said: they should be so well-integrated that the distinction between a theory of context and a theory of psychology starts to disappear.

But what would that actually look like?

I have no sure answers. Instead, I’ll outline three different ways we could think about the problem of context, not as an exhaustive list of possibilities, but as an exploration of what I consider most promising. Each of these approaches reflects a fundamentally different theory of what context even is:

1. Context as external and measurable

2. Context as subjective interpretation

3. Context as relational and co-constituted

Context cartography: the brute force approach

It is worth starting with Cartography because it is in some ways the simplest of the three approaches. The idea is this: exhaustively map every potentially relevant factor so that we understand how studies relate to each other in context-space.

The problem, it is argued, is that psychological research isn’t commensurable—our studies are not apples to apples comparisons. Research articles don’t explicitly mention all the relevant variables, and they often use different terms for the same thing. The result is that our methodologies are so ill defined that it is impossible to tell whether one study is actually a replication of another or not. Tyler Cowen has a really great line about this.

“Every [Behavioral Economics] paper is this kind of little isolated universe, with its own model, with its own data set. They're kind of like…Leibniz Monads; you're not sure how they interact with other papers."

So what’s the solution?

The cartography approach says: brute-force it. Exhaustively log, track, and measure everything.

This is what Linnea Gandhi is doing at Wharton. In her research, she is building on Duncan Watts ideas on integrative experimentation to create what she calls Nudge Cartography; “As the name suggests, we aim to map studies of various nudges. This work will provide researchers and policymakers with a framework for finding the most effective nudges for a specific use case.”

Her team1 reads, re-reads, and re-re-reads hundreds of studies to exhaustively catalogue every potentially relevant factor. On their official website, they say they have analyzed 226 studies and identified 384 variables they think matter in those studies2. This lets them plot each study into a 384-dimensional space, mapping how interventions cluster, which dimensions matter, and what areas remain unexplored.

Nudge Cartography is the experimental methodology that I am most excited about within the traditional approach to Behavioral Science. Mega-studies3, the other major experimental methodology being developed at Wharton, while interesting, rarely are relevant to me. Their hyper-optimization approach reveals very little about human nature outside of that very specific context.

Nudge cartography aspires to fix that problem. Instead of 100 interventions in one ill-defined context (as per the mega-study), you get one intervention across many clearly defined contexts. This is an extremely valuable effort and I wish it were getting more buzz.

Of course, Nudge Cartography has some downsides. The principal one being how laborious it is. I have a lot of respect for the research assistants doing this work. Every time I talk with one of them, I have to ask, “How many variables are you up to now?” Making psychological science commensurable is an exhausting task. Does anyone really want to try and log 384 variables? And what happens when you discover a new variable that matters? You have to go back and re-read all those papers to see whether that variable is present. If a study doesn’t clarify whether it was present? You call the original researchers and ask.

For me, this is too demanding to be a scalable solution for the entire field. I also wish it were more cognitive (see next section). But I may be wrong about the impossibility of codifying so much context, and regardless, it may be the most important effort happening in Behavioral Science right now. I recommend following Linnea and her team closely. Even if it doesn’t become what she hopes, I think we will learn something valuable about context itself. And that makes it a valuable effort.

Modeling the modeler: context as interpretation, not terrain

In physics, when we want to understand how billiard balls bounce off each other, we model the system by specifying variables and writing equations. We know what matters, and how it matters.

But in psychology, we’re not modeling a well-defined system—we’re trying to understand how a person makes sense of the situation: what they notice, what they ignore, and what stories they construct. More poetically, we are modeling the modeler, we are mapping the mapmaker. “Context” isn’t the 384-dimensional terrain, but the subjective interpretation of that terrain.

Consider a study that works in India but fails in Brazil. The difference in replication isn’t because of the the geo-coordinates, the culture, or even the translation of the study intervention—at least none of these variables are directly causal. Rather it is how people interpret those features in the moment. The context isn't some list of objectively defined features, but the subjective interpretation of those features. So unless we model how people interpret that context, we’ll never understand why certain interventions don’t generalize.

The psychologist who, I believe, has most directly confronted this issue of interpretation is Walter Mischel in his work on personality. In many ways, Mischel reverses the causal arrow that many imagine when thinking about personality. Rather than saying a child is high on an aggression trait, and that is why they are aggressive to adults, Mischel says that the child’s consistent perception of adults as threatening leads to aggressive behavior. The aggression trait isn’t a causal force, but instead just a label we use to summarize the behavioral pattern that results from the child’s perception.

Mischel justifies this view by pointing out that a child’s aggression is situation specific. A child that is consistently aggressive to adults is not necessarily consistently aggressive to other children. In such a case, the child must perceive children as friendly, and adults as less so (understandable), and therefore we conclude the behavior is driven by perception, not a generic “trait.”

This critique led to the Cognitive Affective Processing System (CAPS) model, which explains behavior not through traits, but by showing that people consistently construe (interpret or frame) similar situations in similar terms. For example, a child that always assumes kids are friendly, and that adults are a threat.

This consistency in construing situations in similar terms allows for the creation of ‘if-then’ rules of behavior4, such that important environmental features (e.g., the presence of adults) operate like cues in a habit loop leading to consistent behavioral patterns we can label and describe as personality.

Kahneman called Mischel’s work a “scandal” because it exposed the inadequacy of the existing trait based models. Existing trait-based models must treat predictable deviations from averages not as something to be explained, but as an error to be explained away5. But according to Mischel, such variability is not an error, but part and parcel of personality. The child’s variability in aggression is their personality.

You may be more familiar with Mischel’s more famous experiments involving marshmallows. Somehow, our interpretation of these experiments has actually gotten worse over time. Today, people say the experiments showed personalities stability, but Mischel hated that interpretation. Mischel wasn’t testing traits—he was teaching skills.

In the experiment, the researchers were teaching the children skills for resisting temptation by having them re-frame the situation they were in (e.g., ‘Imagine a picture frame around the marshmallow and that it is just a picture of a marshmallow.’). Mischel believed these reframing skills, once learned, were something you could have for the rest of your life.6

CAPS is elegant, and (in my opinion) likely to be more accurate than the more traditional trait theory since it provides a causal account of personality. But it’s also more work for researchers. To understand how your subjects interpret the world, what they consider relevant, and why, you have to use methodologies more advanced than a survey. This is probably a major reason why CAPS never fully caught on: more accurate, less tractable. Modeling the modeler is a much more difficult task than estimating average behavior.

Still, we can learn from Mischel’s criticisms because they apply well beyond personality. Much of psychology, like the personality psychology Mischel was critiquing, is based on tendencies and averages. Prospect Theory claims people tend to be loss averse. The Bandwagon Effect describes what people tend to do. Even the debate about whether nudges “work” is really about averages. Indeed, our entire experimental and statistical paradigm is built around averages.

But a science built around averages becomes nearly unfalsifiable because you have a built-in escape valve. Averages aren’t theories or explanations, and certainly not a guarantee. They’re just summaries. They tell us what happened across a sample, not why or how. Psychologies reliance on averages is perhaps the thing most dissatisfying with our current approach to the study of the mind.

This is why I’m increasingly drawn to theories like CAPS, and Gary Klein’s Data-Frame Theory of Sensemaking which take framing seriously. It’s also why I like Cognitive Task Analysis which is a method for understanding what people noticed, when they noticed it, and how they made sense of it. It treats context and frames as core parts of cognition, not confounds to be controlled away.

Because it seems to me that if we could develop sufficient methodologies for understanding how people frame situations, that we might finally develop true causal accounts of behavior, understand what context is actually relevant in any experiment, and finally understand our problems with replication and generalization.

Transactional worldview: expanding the bounds of the mind

In the first installment of this series, I argued that we should extend the boundaries of the mind. But have we really done that so far? Perhaps these first two approaches do not go far enough, and we can get much more radical. Perhaps we can reconceive the mind itself. To think of it not as being located in the brain, but as a relationship between the brain, body, and environment.

This is hard to think about because we are so used to thinking of the brain as the seat of all cognition. But maybe trying to locate the mind in the brain is a category error, a vestige of our latent dualistic worldview. Rather than thinking of the mind as a thing with a location, we should instead think of it as a process or relationship between many moving parts—only some of which are neurons.

Is that too radical?

Maybe. So let’s scale it back to put everyone on more familiar ground. If you think of cognition as a type of information processing, then we should acknowledge that there is information processing in the tools we use, in how we shape our environments, how we position our bodies, etc.

This is the core of what I’ll call the Transactional Worldview: cognition doesn’t happen inside a brain or outside in the world, but rather arises in the dynamic coupling between the two. No central controller, but instead a series of tight feedback loops between brain, body, and world forming a system where meaning, action, and perception co-create each other to produce psychology main subject of study: cognition.

Some metaphors might help.

Think of a murmur of starlings in flight.

You cannot reduce the intelligence of the murmur of starlings to a single bird. Instead, to understand the collective intelligence of that moving mass, you have to understand how the birds interact with each other, with the wind, the predator birds they are avoiding, the ground, trees, etc. All the various interacting features of the system (living or not) contribute information to the entire system.

The idea is that perhaps cognition is a bit like the murmur. Rather than the information processing happening entirely in the brain as if a single controller was doing everything, perhaps much of the information processing is happening outside the brain, and is distributed in the environment such that intelligence is emergent from the brain-body-environment system.

This way of thinking is a central premise behind 4E cognition, an increasingly popular way to view the functioning of the brain. The titular E’s each push the boundaries of cognition outward:

• Embodied: Our thinking is shaped by our bodily form, sensory systems, and physical interactions, and therefore our thoughts are not solely the product of abstract mental processes.

• Embedded: Cognitive processes are shaped by and embedded within the physical, social, and cultural environment and the body's interactions with it.

• Extended: Cognitive processes can be distributed into the environment through tools, external memory aids, and social interactions, which are all integral parts of the cognitive system existing beyond the skull.

• Enactive: Cognition arises from the dynamic interaction between an organism and its environment.

These ideas are not necessarily dependent on each other, but in bundling them, the advocates of 4E cognition push the boundaries of the mind out as far as possible. Each of the E’s chips away at the traditional boundary between mind and world, and suggests that cognition isn’t in the brain, but is instead a relational phenomenon—a dynamic process enacted between brain, body, and environment.

4E Cognition has a rich intellectual history that I can’t do justice here. But I’ll mention one important intellectual predecessor which might help make it more intuitive.

Edwin Hutchins is the developer of a tradition called Distributed Cognition. In his work, he showed that a ship’s navigation isn’t in any one sailor’s head. No single sailor understood everything that was happening. Instead, the intelligence that was capable of navigation consisted in the coordination between charts, instruments, crew, and action. Without the entire system, navigation would fail.

The cognitive process itself extended across people, artefacts, and systems.

As I mentioned, each of the E’s has their own rich tradition and intellectual history, but this essay is dense enough without going into all of that. My aim isn’t to explain 4E, but to highlight the worldview they collectively suggest.

Personally, I see each of the 4 E’s, and their intellectual predecessors, as bugbears of traditional psychology. I have no idea how true any of them are, but they poke and prod and then nag at me to ask one very important question: how much of what we call cognition is actually done outside the brain? What if our perception of affordances doesn't result from cognition but instead precedes it? What role does the structure of the environment play in memory? Does any one sailor truly understand the navigation of the ship, or is navigation itself distributed throughout an entire process?

The Transactional Worldview suggests that context isn’t what surrounds cognition, but is instead part of what produces it. The environment isn’t a contaminant, but a co-author. Whether you want to take this literally or metaphorically is up to you. But I suspect that even as a metaphor, it may offer more explanatory power than many of psychology’s traditional constructs like traits, tendencies, and effects.

The takeaway for the Transactional Worldview is this; if we want to stop underfitting the context, we have to stop overfitting the brain. Behavior is not the output of internal mental machinery. It is the emergent property of a tightly coupled system. This is an idea worth taking seriously even if, like me, you are unsure how far to take the ideas.

Conclusion

So here we have three major approaches to rethinking the edges of the mind by treating context quite differently:

1. As external and measurable, which we can systematically map (Cartography).

2. As subjective and interpretive, shaped by how people construe situations (Modeling the Modeler).

3. As relational and co-constitutive, where cognition emerges through dynamic interaction (Transactional Worldview).

I don’t pretend this list is exhaustive, nor am I an expert in any of them. They are simply the approaches I have come across that seem to have the most bite. If you know of others, drop them below in the comments. Also, let me know which you find most plausible, as I suspect people will differ wildly on that account.

I also didn’t talk as much about methodology, ontology, or the nature of causality, each of which could fill another article. I may explore those issues in a future post depending on interest. In the meantime, I’ve included resources below for each of the three approaches, as well as other work that shaped how I think about context.

Resources

Nudge Cartography

• Research Cartography: How to Build a Map to Navigate the “Do Nudges Work?” Debate and Beyond by Linnea Gandhi. A talk on Nudge Cartography.

• Beyond playing 20 questions with nature: Integrative experiment design in the social and behavioral sciences by Almaatouq et al. One of the major inspirations for the Nudge Cartography.

• The Nudge Cartography website

Modeling the modeler

• A Cognitive-Affective System Theory of Personality: Reconceptualizing Situations, Dispositions, Dynamics, and Invariance in Personality Structure by Mischel and Shoda. Describes their CAPS theory of personality.

• Daniel Kahneman interviews Walter Mischel - I think the full interview is floating around somewhere, but I couldn’t find it before this went live

• Data-Frame Theory of Sensemaking by Gary Klein. His take on how we develop frames. Very influential on me.

• Use of the Critical Decision Method to elicit expert knowledge: A case study in the methodology of cognitive task analysis By Hoffman et al. The major interview technique used in Naturalistic Decision-Making

Transactional Worldview

• 4E Cognitive Science. A Brief overview.

• Ecological. A brief overview (let me know if you know of a better overview)

• Distributed Cognition by The Decision Lab. I was very pleasantly surprised they had this!

• Foundations of an Ecological Approach to Psychology by Harry Heft. I took the term “Transactional Worldview” from here.

• Externalism. A closely related term.

Other things which have shaped how I think about context

• Context reconsidered: Complex signal ensembles, relational meaning, and population thinking in psychological science by Lisa Feldman Barrett. A very good paper arguing against mechanistic approaches to psychology.

• Context Changes Everything: How Constraints Create Coherence by Alicia Juarrero. Fascinating, but hard to read.

• Process Philosophy. I wish I was better at thinking in terms of processes than objects!

“Why did the chicken cross the road?” isn’t usually a very deep question. But in Behave, Robert Sapolsky uses it as a jumping off point to expand on causality and the boundaries of scientific disciplines.

He walks through various possible explanations of the chicken’s behavior: a psychoneuroendocrinologist says its due to estrogen levels, as there is a sexy rooster on the other side; a bioengineer instead describes the mechanics of chicken legs; and the evolutionary biologist explains it in terms of mating behavior and selection pressures.

Each explanation is valid and correct, but also none is complete on its own. Sapolsky’s point is that siloed thinking will prevent you from fully understanding the behavior. Just because your job is to study hormones doesn’t mean you get to ignore culture, cognition, evolution, or anything else that might be relevant.

I’ve always loved Sapolsky’s build on this classic joke as a reminder that science operates through these overlapping but partial perspectives. Each discipline stacks like lenses in a microscope: from cells to circuits to cognition to culture, each lens brings a different layer into focus. No single frame captures everything, but together, they give us a more detailed picture than we otherwise could have gotten.

But while interdisciplinarity is necessary for a complete understanding of pollo locomotion, that does not mean we should collapse disciplines. Estrogen and culture are different kinds of things, and so require different tools and methodologies. So while the disciplines are arbitrary in one sense, in another they are also reflections of genuine differences. You can’t study cultural narratives with a syringe.

Besides this ontological (and therefore methodological) reason for the divide between disciplines, there is also an epistemological one; focus. Each discipline—as well as every model and theory—ignores the rest of the world, not out of ignorance, but because of the impossibility of bringing everything into focus at once. Each discipline, model, and theory has a blind spot where there exists the entire rest of the world, and these blind spots are features, not bugs. The blind spots are what afford us the ability to zero in on the part of the world we are most interested in.

This intentional ignorance is actually where science gets its power. The common saying is that all models are wrong, but that statement can be misleading. In truth, all models are intentionally wrong. They deliberately bracket out most of the world so that we can focus on some small part of it. Science advances not despite blind spots, but because of them. The ability to be temporarily ignorant of the rest of the universe is the goal of good science.1

This can seem paradoxical, but it is quite common sense when you think it through. Each discipline is defined by both a boundary and a lens. The boundary is what brackets out most of the world as "not relevant”, or as “something to be controlled”, and so therefore acts like a dam holding back a flood of all the possible things we could focus on. The lens, on the other hand, is what brings the particular relevant aspect of the world into sharp focus, be it neurons, narratives, or natural selection.

We need both: lenses to reveal, and boundaries to isolate. The boundaries prevent distraction and dilution; the lenses help us detect detail and structure. Together, they allow science to temporarily ignore the complexity of the whole in order to understand a part. Because if a psychoneuroendocrinologist had to account for bioengineering and evolution every time they ran a study, they’d never get anything done.

The two questions I wish to ask in this essay are the following: has psychology placed its boundaries in the right place? And does it have the right lens?

II. Context is that which leaks in. But sometimes it floods.

Disciplinary boundaries are necessary, but they are also leaky. Causal forces from other levels of analysis leak in through the edges. And worse, because they’re from a different level of analysis, we don’t have the right lens to bring them into focus. The evolutionary psychologist is not a cultural psychologist, and so will have a hard time bringing specific cultural practices into focus when they don’t fit in with the clean evolutionary model. As a result, culture is always getting in the way of their true subject of study.

This leaking in from a different level of analysis is what we call context. Context isn’t part of the model—that we call content—instead it’s what the model left outside the boundaries, but is leaking back in anyways. It is the residual interference we failed to bracket out. An interdisciplinary leakage that our lens isn’t built to handle.

Of course, context isn’t a real thing out in the world. It is instead an artefact of how we draw our boundaries. Since the world doesn’t come pre-sliced, true isolation is impossible. If you could fully isolate a system so that there was no context or leakage, you’d isolate it from yourself too. You couldn’t observe it at all. Context is therefore inherent to science. The inevitable byproduct of having scientific disciplines and scientific models which attempt to isolate and focus on one small aspect of reality.2

But while some context leaking in is ok and inevitable, it can sometimes become too much. Less a drip, and more a deluge. Our subject of study is so context sensitive to the leaking that the subject just doesn’t sit still long enough for us to get our bearings. It's like the dam has broke.

When this happens, we have to ask ourselves whether there might be something wrong with our boundaries. Perhaps the issue is not that we are unsure how to control for context and isolate it, but rather, maybe what we wanted to dismiss as context is actually the content we should have focused on. Maybe we got the boundaries wrong. Maybe what we wrote off as noise was actually the signal, and our models are just underfit.

Or to put it more eloquently, maybe we built the dam in the wrong damn place.

Because while the boundaries around our models and between our disciplines serve a purpose, they’re still human-made. The world doesn’t actually care about the distinction between the content of our models and the context that leaks in—we invented that distinction when we put the dams up in the first place. The difference between content and context is pragmatic, not actual.

So now I ask again: has psychology placed its boundaries in the right place? And does it have the right lens?

Because it seems that psychology has some pretty systematic errors. The leaking of context into psychology is positively diluvian. Everywhere you go, you hear a magic phrase; context matters. A phrase that is a tacit acknowledgement that, in our attempt to bracket out the rest of the world and put some dam boundaries around our subject of study, there are gaps large enough for the world to leak back in and disrupt our focus. It’s a phrase that means that what we have left outside the dam boundaries is flooding back in making it difficult to study the thing we are interested in. A phrase that means the dam boundaries are not holding.

And after hearing ‘context matters’ so many times, I’ve started to wonder that maybe the problem isn’t the lack of rigorous controls of the context. Maybe instead, psychology is just systematically underfit. What we dismissed as noise is signal, our context to be controlled is actually content to be studied.

Maybe, that is to say, we put the dam in the wrong damn place.

III. Are all soft sciences just leaky sciences?

Of course, it isn’t necessarily the case that there are better boundaries. Psychology isn’t the only discipline with a major leak. As you move up levels of analysis—from atoms to cells to societies—there seems to be an inherent increase in leakiness. The boundaries blur, and things get messy.

The philosopher William Wimsatt has this great series of five charts showing different ways the world could have been. The x-axis represents increasingly larger levels of analysis, and on the y-axis you have how regular and predictable the entities at that level of analysis are—or said differently, how context sensitive the entities are.

In the first chart, you’ve got a clean, predictable world where each level behaves as well as the one below. Sociology is as lawlike as physics. In the next chart, things fall apart—levels become unpredictable, disappear, and bleed into each other in a random walk. Then in the last two charts he imagines the world becoming less leaky at higher levels of analysis, or a world where distinct levels never emerge at all.

However, in Wimsatt’s third chart—the one that most resembles our reality—things get fuzzier as you move up levels. The edges get sloppier. By the time you reach society, the models are barely holding anything back. The dam boundaries are about as water tight as a cheese grater.

This is one way to understand the difference between the hard and soft sciences. In the hard sciences, like physics and chemistry, the units are more easily isolated, and so context doesn’t leak in and overwhelm the model. You can actually do math because the units of analysis are independent enough to behave according to the basic rules of arithmetic.

But in a soft science like sociology or anthropology, your units of analysis are not quite unitary. And building clean generalizable models of something which can be divided in so many ways, and influenced by so many things, means you'll always have some leakage. Mathematical models become increasingly inaccurate as the boundaries get leakier, and the discipline becomes increasingly context sensitive.

I think of soft science as like a science of raindrops on a window. Raindrops on a window are not mysterious. But if you draw your boundaries at the raindrop level of analysis, you may just lose your mind. A raindrop appears on the window, splatters into fragments, coalesces with others, then splits again. One raindrop plus one raindrop equals… one raindrop. But only until it doesn’t.

Raindrops are not clean, unitary objects, and they’re shaped by all kinds of interactions: surface tension, wind, temperature, the shape of the glass, etc.. You can’t model them with simple arithmetic. Instead, if you want to keep to the raindrop level of analysis, you have to settle for tendencies. You identify patterns, describe edge cases, use qualitative insight. In essence, you do soft science. It’s not that the raindrop level of analysis is wrong. Indeed, it may give you exactly what you need. But the level of analysis itself just doesn’t lend itself to hard science kinds of analyses.

And maybe that’s just the nature of all the social sciences, including psychology. The boundaries around neurons are crisp and clear, but the boundaries around psychological constructs are not. Psychological explanations just don’t hold steady the way chemical or cellular ones do.

Behavioral models likewise. Behavior bleeds into beliefs, which bleed into norms, which bleed into culture. The object of study won’t sit still because its very nature is to change, adapt, and interact.

So we conclude that, in psychology, the subject matter is simply too unstable, too context-sensitive to ever allow anything approaching hard science.

And I’m sympathetic to that conclusion. There is definitely some truth to it.

But I am also discontent, and I think you should be too. If the boundaries are arbitrary, then it is always possible we’ve drawn them in the wrong place. Maybe if we re-drew the boundaries, things would come into focus, and the leaking wouldn’t be as severe. Maybe we’re doing a science of raindrops at the raindrop level of analysis when we could be looking at weight or volume or something else entirely.

Psychology isn’t a healthy science, and maybe that is because we have the wrong damn boundaries.

IV. Dissatisfaction with leaky psychology

In Ethan Ludwin-Peery’s essay, Why Psychology Hasn’t Had a Big New Idea in Decades3, he proposes different ways to re-conceptualize psychology. He argues that instead of pure psychology we might need to study psychology + neuroscience, or psychology + economics, or do deep dives into memory, value, or even plants. Or perhaps none of the above. Whatever it is, he argues we need to re-conceptualize the very boundaries of psychology.

The reason he thinks such a re-conceptualization may be necessary is that he believes psychology is pre-paradigmatic. The comparison he makes is to alchemy.

Alchemists made measurements, ran experiments, and had causal theories. Sometimes they would even share their findings so that others could then replicate what they had found (though ambition often led them to stay silent about what they found). And at the end of the day, alchemists did in fact make many discoveries.

But was it science?

Alchemy was so conceptually confused that, despite many of the trappings of science, we are reluctant to grant it the title.

However, that's probably just gatekeeping. Bad science is still science. Ethan thinks the best term is ‘pre-paradigmatic’, which implies they were attempting science by proposing theories, testing hypotheses, and all the other things a scientist should do…but without a sufficient understanding of their subject to even agree on what it would mean to do good science.

One way to make this claim more intuitive is to recognize that alchemists didn’t even know what counted as a replication. Their categorization of the world was so confused that they had no clear sense of what variables to control for. When burning the egg of a newt to extract the essence of water, did the phase of the moon matter? The species of wood used to heat the fire? Whether the newt was pregnant? Lacking a framework to distinguish relevant from irrelevant factors (i.e., clear boundaries), their experiments were inherently ambiguous. If two alchemists got different results, they had no way of knowing why. Experimentation by itself is not enough—you need a paradigm to help make sense of it all.

Without a paradigm, for alchemists, it was not just that Ceteris Paribus was an unrealistic ideal, it was that they didn't even know what it would mean for all else to be equal. They didn’t actually know what was relevant. They didn’t have clear boundaries to tell them what kind of stuff they were interested in, or the right kind of lenses to bring that stuff into focus.

But then centuries passed, and humans learned a few things. Alchemy turned into chemistry. Not through an increase in methodological rigor, but by redefining what the subject of study even was. Earth, water, air, and fire turned out to be phases of matter, not the basic elements of reality themselves—and with this reconceptualization of the subject, a real paradigmatic science could emerge. One with a unifying framework that helped scientists to run controlled replicable experiments and have consistent interpretation of the results.

And the question is whether psychology is perhaps a bit more like alchemy than chemistry.

Of course, psychology isn’t quite as lost as alchemy. I think that is a tad too insulting. But it might still be pre-paradigmatic—or at least lacking a good paradigm. Scientific in method, but lacking the conceptual framework for full explanations and replications.

In mature sciences, claims come with relevant provisos. “Sugar dissolves in water” assumes you don’t freeze it instantly. “Vinegar and baking soda create a volcano effect” assumes you’re in earth-like conditions, and so it doesn’t count as a replication if you do it on Pluto. These provisos are an implicit part of the paradigm, left unstated simply because there are simply too many of them. But because an alchemist from Pluto does not understand the paradigm, the provisos would need to be spelled out so that they could verify the chemist’s claim experimentally.

But in psychology…

In psychology it is just assumed that provisos exist, and we don’t hold it against a theory if it happens to accidentally run afoul of one of them. After all, you can’t expect someone to define ALL the context, right?

Ceteris paribus in psychology, as it was in alchemy, doesn’t just fail as an ideal—we are unsure how to even define it. Our paradigm, such as it exists, is ambiguous about all the possible provisos.

This is why generalization has to be handled on a case-by-case basis. A good behavioral scientist never assumes a finding generalizes because all our theories are sometimes theories.

People are loss averse…sometimes. People obey authority figures…sometimes. Jared is agreeable…sometimes.

No psychological theory is as law-like as anything in chemistry because we don't know the provisos. Context leaks in, and we can’t cleanly isolate causes, or pin down universal mechanisms. Many of are most important findings are just glorified averages. (e.g., people tend to be Loss Averse)4

And maybe part of the reason for this unsatisfactory situation is that we’re stuck using categories as confused as earth, water, air, and fire—studying the phases of the elements of reality, but not the elements themselves. If we are not ever studying the right type of things, of course the models will fall apart. Of course we can't define the provisos, consistently replicate findings, or predict how things will generalize.5

If the dam boundaries are in the wrong damn place, of course there is going to be excessive leaking, and all we’ll be able to say is that “context matters.” Would not the alchemists have said the same when their own experiments didn’t replicate?

V. Pushing the boundaries to create a science of context

In psychology, our current level of analysis (the mind) was developed in the 60’s during the cognitive revolution. It is recent.

To be sure, the study of the mind persisted even through the dark days of behaviorism, and also in the decades and even centuries prior. But nevertheless, our current level of analysis wasn’t really formalized until relatively recently, and other boundaries have been drawn before and since, and may be drawn again.

The variety of possible boundaries, and how recent they are, is worth remembering. There may be something inevitable about the atomic, chemical, and cellular levels of analysis, but perhaps not for psychology. You'll notice that neither the mind or behavioral levels even make an appearance on Wimsatt’s chart about the five possible worlds. Those levels are different somehow.

And all this makes me wonder what would it take to redraw our boundaries? To move beyond endlessly repeating "context matters" without ever saying exactly what that context is? To build models that can replicate not by ignoring context, but by learning to incorporate it?

Maybe psychology’s leakiness is inevitable. Maybe, like Kevin Munger suggests, we need to abandon replication as the cornerstone of the social sciences.6 After all, if physicists struggle with modeling three bodies in space, why should psychologists—who study something far more multi-causal—expect to ever have anything less ambiguous “people tend to be loss averse”?

And yet, I can't shake the sense that psychology systematically underfits the complexity of human behavior by treating context as noise rather than structure. That our models leak not just because human behavior is messy, but because we drew our dam boundaries in the wrong damn place. Because if it changes behavior, then shouldn’t it be the very content that we study?

Perhaps the reason cognition and behavior don’t appear in Wimsatt’s chart is that they are not the result of compositional layering. Wimsatt’s chart shows levels that emerge by combining entities of the same kind: atoms into molecules, molecules into cells, cells into organisms, etc. Each level nests within the next, preserving ontological similarity.

But cognition isn’t built that way. It doesn’t arise from the stacking of like onto like. Instead, it emerges from the interaction between different kinds of things, namely, the organism and its environment.

Cognition, in this view, isn’t a “higher level” built atop the others. It’s a relational phenomenon—something that exists between levels rather than above them. That may be why it resists being plotted on the chart: it breaks the compositional metaphor that underlies it.

So I propose we push our boundaries outward. That we stop dismissing context as a nuisance, and start treating it as part of the object of study. That we build a science where context is no longer background noise, but foreground structure.

In an older essay I wrote that applied behavioral science is the science of context, since it doesn’t study psychological mechanisms, but how those functions work in specific contexts. But now I wonder why we would hold only applied psych to that standard? Maybe psychology as a whole needs to become a science of context. Or rather, perhaps psychology needs to redefine what it even considers to be content and context.

We don’t need to tear psychology down or insult it. There are areas that we understand pretty well, such as vision, habits, and others. Applied fields like Naturalistic Decision-Making show that even without perfect replication, we can create trainings based on psychological research which prepare people for life-and-death decisions. Psychology is not as hopeless as alchemy once was.

But for many of the questions we care most about, it feels like we are trying to build a science of raindrops.

In Part II, I’ll explore some of the most promising paths that I know of for turning context into content. I am not convinced that any definitively is the answer. But I am also not convinced any of them are not. So it is worth the effort to at least think them through.7

UPDATE:

The talk is in English and starts at 2:25. Here’s the video:

After the presentation, several folks asked me to write more about Naturalistic Decision Making. And I promise I will. But my writing tends to follow my whims more than my will. So my next two posts will actually be about context, and I am very excited about it. Those posts will be more ambitious than my typical post, as I will challenge the very foundations of our current paradigm. So be on the look-out for that.