The Statistician’s Dilemma
Should you tell the story behind the numbers?
As Nancy Huston brilliantly explains in L’espèce fabulatrice, our culture and identities — surname, first name, birth date, words, langage, mariage, job, ethnicity, political affiliation, etc. — are woven of stories. If facts just tell, stories sell. Religions, political parties, the media and big companies all exploit human penchant for narrativity.
We’re in some way wired to prefer the comfort of a coherent story to the dizzying void of absurd or uncertainty.
Quite unsurprisingly then, quants, actuaries, statisticians, business analysts — generally speaking data-oriented professionals are expected to “tell the story behind the numbers”, to “provide original insights”, to “connect the dots”, “uncover trends”.
In my opinion, if too dominant, this storytelling injunction may conflict with an important deontological principle: the search for truth.
By search for truth, I mean the intent to produce observations that are:
- As fact-based/objective/rational as possible
- Unbiased
- Communicated along with all the methodological hypotheses/shortcuts that were made, and their consequences
One doesn’t have to look very far to see where the conflict arises, as it’s transparently carried in the expression itself: to “tell the story behind the numbers”. The expression implies that there is a story, and that numbers can capture it.
First, we must keep in mind that “not everything that counts can be counted, and [that] not everything that can be counted counts” [1].
Then, the main issue at stake is that readers expect stories of a particular genre — stories in which you’ll find:
- A nearly omniscient narrator
- Protagonists who take actions
- Actions that shape the course of events
- A smooth series of events framed in “cause-and-effect” relationships, which will feed the reader’s appetite for associative coherence and cognitive ease
- Compelling graphs that show why things couldn’t have turned in any other way
With such a recipe, you’ll build inspiring “success stories” and didactic explanations of “why [you name it] failed”.
We clearly see that the desire for storytelling can be a slippery slope leading to subjectivity and a compendium of biases [2], among which:
- Narrative fallacy (hindsight bias and a posteriori adjustments of our beliefs)
- Outcome bias
- Underestimation of the role of luck in the course of events
- Human inability to deal with nonevents
- Conflation between correlation and causality, or between “not rejecting an hypothesis” and “having proved that this hypothesis is true”
Who would like to hear a story in which “despite very poor decisions of our leaders, our company was very successful — probably because of luck”? Or one in which “we took drastic actions, but cannot tell if they had any impact because what we observe may just be noise and not signal”?
“An unbiased appreciation of uncertainty is a cornerstone of rationality — but it is not what people and organizations want” [3], and storytelling induces a very particular lens through which reality will be perceived.
One the other hand, what could a servant of the sacrosanct truth reasonably say? He could quote Socrates — “The only thing I know is that I know nothing” — or Nietzsche — “There are no facts, only interpretations” — , but would we be better off?
If too dominant, the focus on biases and exactitude leads to a form of cowardice that paradoxically disqualifies the quant/actuary/statistician as a collaborative partner to make business grow. Hence the need for a mutual effort of both story “writers” and story “readers” to meet at a middle ground.
As readers, we could (continue to, if already started):
- Keep in mind that numbers don’t tell the whole story
- Be open to counter-intuitive insights, nuance, and demanding intellectual effort to grasp complex patterns
- Accept that, where we would have wanted our initiatives to have an impact, luck and randomness may have played the bigger role
As writers, we could (continue to, il already started):
- Detect the frontier where simple becomes simplistic, and try not to cross it
- Underline the sensitivity of our results to the assumptions made
- Provide readers with different scenarios illustrating the uncertainty of the results to the volatility of the variables at stake
- Avoid the aforementioned biases ourselves
Those combined efforts could help us reduce our collective illusion of understanding.
Acknowledging that things aren’t as explainable as we would want them to is unpleasant and feels like a step backward, but I think it can in the end improve the quality of our thinking and of our decisions.
[1] : Credited to William Bruce Cameron, according to quoteinvestigator.
[2], [3] : See Daniel Kahneman’s gold mine Thinking, Fast and Slow (part III, Overconfidence).
