Strategically delusional

Why it pays to be overconfident

Asked which is the one bias he would most want to eliminate if he could, Daniel Kahneman answered: “Overconfidence”. Kahneman’s viewpoint can be understood considering that many of the world’s greatest catastrophes, avoidable conflicts and ill-conceived enterprises have been linked to this bias. Within the broad category of “cognitive biases,” overconfidence is somewhat unique, as it doesn't appear to be the consequence of “random errors.” If overconfidence does stem from errors, these errors appear to systematically promote self-serving beliefs.

Overconfidence and self-esteem

The evidence for the pervasiveness of overconfidence is robust. Individuals frequently overestimate their abilities and their standing relative to others. To illustrate: the majority of drivers consider themselves more skillful than the average driver (Svenson 1981); most students regard themselves as more intelligent than the average student (Gabriel et al. 1994); and in a well-documented survey, more than 90% of U.S. university professors considered themselves to be superior teachers compared to their colleagues (Cross 1977).

Why do we cultivate such glaringly inaccurate beliefs about ourselves? The prevailing interpretation suggests that we derive pleasure from thinking we are better than we actually are. In the popular science book Mistakes Were Made (But Not by Me), psychologists Carol Tavris and Elliot Aronson, for example, attribute our propensity for overconfidence to the need for ego-preservation and the enjoyment derived from ego-enhancement and increased self-esteem. For a considerable time, these motives have been the primary explanations for overconfidence within the field of psychology.

The book Mistakes were made (but not by me) posits that overconfidence is fuelled by self-esteem—an explanation that is popular in social psychology.

This notion resonates intuitively. In a widely cited exploration of the psychological effects of self-esteem, Baumeister and his colleagues remarked:

Self-esteem has a strong relation to happiness. Although the research has not clearly established causation, we are persuaded that high self-esteem does lead to greater happiness. - Baumeister et al. (2003)

Costs of overconfidence

There is, however, a problem with attributing overconfidence to a self-esteem motive: Self-esteem is a benefit that occurs in our head, but overconfidence is bound to lead to costly mistakes in the real world. It seems counterintuitive that evolutionary forces would have designed us to enjoy beliefs that lead us to make costly decisions.¹

The fact that overconfidence can be costly is obvious. In his book Perfectly Confident, psychologist Don Moore provides an illustrative example. He recounts the story of Jeffrey Rubin, a professor at Tufts University, who had set a goal to climb the hundred tallest peaks in New England. While attempting one of the last peaks, Mount Katahdin, he encountered dense fog. Despite recommendations to turn back, Rubin pressed on, displaying confidence and determination despite the evident risks. Tragically, Rubin did not survive the climb; he succumbed to hypothermia and exposure.

The Mount Katahdin in Maine

Many decision-making situations demand an accurate assessment of one's abilities and limitations. Should one attempt to climb this mountain? Should one try to swim across that river? Should one opt for a more challenging black ski slope? In such contexts, overconfident beliefs can lead to costly, sometimes catastrophic, outcomes. In the realm of economics, empirical studies suggest that overconfident traders are prone to excessive risk-taking (Barber and Odean 2001) and that overconfident CEOs often make ill-advised decisions concerning investments or mergers (Malmendier and Tate 2005).

Benefits of overconfidence

If overconfidence incurs real-world costs, it must also offer real-world benefits to have been selected by evolution. To identify these benefits, we need to distinguish between two kinds of situations. First, there are scenarios where you interact with natural elements, such as deciding whether to climb a mountain or swim across a river. In these settings, overconfidence offers no benefits² and may even lead to fatal errors. The second category involves social situations where you interact with other individuals. In these contexts, overconfidence can have an impact because, unlike nature, people can adjust their behaviour in response to yours.

In social situations, game theory offers a significant insight: rationality can be self-defeating (Sugden 1991). The rationale for this lies in the potential for irrational behaviour to strategically influence others in your favour. Take, for example, the classic “Game of Chicken,” where two opponents drive their cars towards each other at high speed, with the first to swerve deemed the loser. This scenario has been dramatised in well-known films like Rebel Without a Cause (1955) and Footloose (1984). A viable strategy to win this game is to act in a way that credibly demonstrates your irrational commitment not to swerve—by removing your steering wheel, for example, signalling that you are incapable of changing course.

Because rationality can at times be counterproductive, it's not always advantageous to hold accurate beliefs. Overconfidence, then, can serve as a commitment device, signalling a credible intent to act as though you are strong. Even if others are not entirely convinced of your actual strength, this stance may still accrue benefits for you. For instance, someone might refrain from engaging in an argument or a physical confrontation—even if they would likely win—simply because the trouble is not worth the risk. Projecting confidence, therefore, can offer advantages in social interactions.

Confidence may help you deter others by making you committed to act as if you were strong.

Self-deception to deceive others

This explanation, however, presupposes that your beliefs are visible to others. In reality, our beliefs reside solely in our minds. Given that you can't directly observe whether I genuinely believe I am tough, I may attempt to appear tough even if I don't actually hold that belief. In turn, you might be sceptical of my claims of toughness. Consequently, the challenge I face is how to credibly convey to you that I genuinely believe in my own strength and will act accordingly, rather than hesitating or retreating at the last moment.

De Niro rehearsing sounding tough in the classical scene from Taxi Driver (1976)

Biologist Robert Trivers proposed a solution to this problem in 1976: our tendency to self-deceive, to think we are better than we are, may serve as a mechanism that enables us to deceive others more effectively. In the preface to Dawkins’ The Selfish Gene, he wrote:

If … deceit is fundamental to animal communication, then there must be strong selection to spot deception and this ought, in turn, to select for a degree of self-deception, rendering some facts and motives unconscious so as not to betray – by the subtle signs of self-knowledge – the deception being practiced. —Trivers (1976)

Commenting on this assertion, psychologist Steven Pinker remarked, "This sentence... might have the highest ratio of profundity to words in the history of the social sciences" (2011). A key idea from Trivers is that in communication, deception is widespread. This is a reasonable assumption. As stated by Dawkins (1976): “It may well be that all animal communication contains an element of deception right from the start, because all animal interactions involve at least some conflict of interest.” These conflicts of interest exist because, as pointed out by game theorist Thomas Schelling, most social interactions are mixed-motive games where players have some convergent and some divergent interests. Jane and John may for instance have a joint preference to spend the evening together, but Jane may prefer going to a good restaurant while John prefers to go to the cinema. To secure their respective preferences, they may find it advantageous to exaggerate them. As Trivers implies, self-deception may make them more convincing negotiators.

Why self-deception may work

For Trivers’ explanation to work, we have to be more convincing when believing our own lies than when being self-aware of the gap between the truth and what we say.

When people make claims, we exert epistemic vigilance to assess their degree of credibility. Suppose John, a student, presents an excuse to his professor Jane for handing out an assignment late. Jane will assess the credibility of John’s excuse: is it likely to be the real reason of John’s missing the assignment deadline? Jane can ask questions, prodding whether John has credible answers to her inquiries or whether his answers are confused. Furthermore, she can assess his non-verbal communication—how he is delivering his answers—that may signal his state of mind, his confidence.

von Hippel and Trivers (2011) list three types of cues that Jane can use to assess John’s credibility: nervousness, signs of suppression and cognitive load. Nervousness can be associated with deception arguably because deceiving comes with the risk of potential costs if found out. Signs of suppression can become apparent when people try to mask signs of deception, such as signs of nervousness. Finally, a person’s cognitive load may substantially increase when trying to hold a consistent fictional story together while being aware of the truth at the same time. In all this process of interrogation, John may benefit from believing his self-serving story. That way, he may avoid giving away cues of deception and, therefore, seem more credible and genuine than if he was self-aware.

The benefits of delusion

Recent studies support the notion that self-deception can indeed facilitate the persuasion of others. Schwardmann and Van der Weele (2019) designed an experiment in which participants were incentivised to convince others about their performance in a given task. The researchers found that participants were generally predisposed to form overconfident beliefs. In a related study led by my former PhD student Alice Solda, participants displayed a tendency to select information that bolstered the perception of their strong performance (Solda et al. 2020). In both investigations, the formation of overconfident beliefs proved advantageous when participants were faced with the task of persuading others about their capabilities.³

Some of the most common situations where we need to persuade others are when we have to negotiate how to share the costs and benefits of a collective effort. If self-deception can help convince others, negotiators may benefit from overestimating how much they deserve. It has been found, for instance, that the sum of participants’ perceived contributions to a joint effort tends to be more than 100%. It is the case in couples, when partners are asked to assess their personal contribution to household chores (Ross and Sicoly 1979) and in groups of scientists asked to assess their personal contribution to a published article (Schroeder et al. 2016).

Conclusion

An important insight from this body of evidence is that we are not designed to form objectively accurate beliefs about ourselves. Instead, we see the world through rose-tinted glasses. We tend to think of ourselves as slightly better, slightly more deserving, and slightly more moral than we actually are. We think that way because slightly delusional beliefs come with strategic benefits as they help us convince others. If we were consistently objective in our self-assessments, we would be at a disadvantage compared to others who confidently put inflated claims forward.

It is easy to see that such generalised overconfidence and self-serving beliefs are a cause of many conflicts, from the workplace to the international sphere, in which those involved think they deserve more and are confident in their chances to win. Daniel Kahneman's assertion that eliminating overconfidence would have a significant impact may indeed be well-founded.

This post is the first of three posts on how and why our beliefs are wrong but for often good reasons. The next posts will delve into how our reason really works.

References

Anderson, C., Brion, S., Moore, D.A. and Kennedy, J.A., 2012. A status-enhancement account of overconfidence. Journal of Personality and Social Psychology, 103(4), p.718.

Barber, B.M. and Odean, T., 2001. The internet and the investor. Journal of Economic Perspectives, 15(1), pp.41-54.

Baumeister, R.F., Campbell, J.D., Krueger, J.I. and Vohs, K.D., 2003. Does high self-esteem cause better performance, interpersonal success, happiness, or healthier lifestyles?. Psychological science in the public interest, 4(1), pp.1-44.

Bénabou, R. and Tirole, J., 2005. Self-confidence and personal motivation. Psychology, Rationality and Economic Behaviour: Challenging Standard Assumptions, pp.19-57.

Burks, S.V., Carpenter, J.P., Goette, L. and Rustichini, A., 2013. Overconfidence and social signalling. Review of Economic Studies, 80(3), pp.949-983.

Cross, K. Patricia (1977) “Not can, but will college teaching be improved?” New Directions for Higher Education, Vol. 1977, No. 17, pp. 1–15.

Dawkins, R., 1976. The selfish gene. Oxford University Press.

Gabriel, M.T., Critelli, J.W. and Ee, J.S., 1994. Narcissistic illusions in self‐evaluations of intelligence and attractiveness. Journal of Personality, 62(1), pp.143-155.

Malmendier, U. and Tate, G., 2005. CEO overconfidence and corporate investment. The journal of finance, 60(6), pp.2661-2700.

Pinker, S., 2011. Representations and decision rules in the theory of self-deception. Behavioral and Brain Sciences.

Ross, M. and Sicoly, F., 1979. Egocentric biases in availability and attribution. Journal of Personality and Social Psychology, 37(3), p.322.

Schroeder, J., Caruso, E.M. and Epley, N., 2016. Many hands make overlooked work: Over-claiming of responsibility increases with group size. Journal of Experimental Psychology: Applied, 22(2), p.238.

Schwardmann, P. and Van der Weele, J., 2019. Deception and self-deception. Nature human behaviour, 3(10), pp.1055-1061.

Solda, A., Ke, C., Page, L. and Von Hippel, W., 2020. Strategically delusional. Experimental Economics, 23, pp.604-631.

Sugden, R., 1991. Rational choice: a survey of contributions from economics and philosophy. The Economic Journal, 101(407), pp.751-785.

Svenson, O., 1981. Are we all less risky and more skillful than our fellow drivers?. Acta Psychologica, 47(2), pp.143-148.

Tavris, C. and Aronson, E., 2007. Mistakes were made (but not by me): Why we justify foolish beliefs, bad decisions, and hurtful acts. Harcourt.

Trivers, R. Preface in Dawkins, R., 1976. The selfish gene. Oxford University Press.

Von Hippel, W. and Trivers, R., 2011. The evolution and psychology of self-deception. Behavioral and brain sciences, 34(1), pp.1-16.

1. See my post on the optimal design of a subjective reward system.

2. For overconfidence to increase our chances of success relative to an appropriate level of confidence, you would need to explain why the highest chances of success cannot be reached with the appropriate level of confidence.

3. The notion that overconfidence can function as a signal of genuine confidence has also found empirical support (Anderson et al. 2012, Burks et al. 2013).