10 Effective Approaches to identify false information and refute it effectively
The term “fake news” has become a common term in today’s information-overloaded world. During and after the election of Donald Trump he often called news fake. Even if it was candor information.
But how can you identify false information effectively?
Generative AI is becoming more and more powerful and so are deep fakes. They are realistic to a point where it is hard to distinguish fact from fiction.
 |
|---|
| Photo by Nijwam Swargiary on Unsplash |
After my last article on the prosecutor’s fallacy, I want to shed some more light on the topic of misinformation and share some strategies with you to identify and refute it. As a main resource for this article, I’ve used Calling Bullshit by Carl T. Bergstrom and Jevin West.
Navigating the Landscape of Misinformation
Why has “fake news” become so common?
The key-word here is engagement. On social media platforms engagement can be directly translated to more views, hence, more revenue from ads. There are some phrases that lead to more engagement:
- “… will make you …”
- “This is why …”
- “… we can guess …”
These phrases are often used in posts and clickbait articles - the titles offer a lot but don’t deliver once you engage with them.
Let’s have a look at the different types of bullshit in our digital world.
Numbers and Nonsense
Numbers play a special role in the realm of bullshit: numbers moved from a completely different context to underline a statement, misused statistics, or mathiness.
Mathiness is a form of bullshit where apparently mathematical formulas are used that are not actual formulas. This is a common strategy as numbers and formulas convey mathematical rigor and can strengthen a statement.
But it isn’t always deliberate misuse of numbers that convey bullshit. It can be also context-dependent.
Think of metrics.
As soon as a metric becomes a target, it is no longer a good metric. This is Goodhart’s law.
People are very creative in achieving goals and exploiting system errors. “Gaps” are quickly found in the system, which are then exploited to achieve the target figure. One example Carl T. Bergstrom and Jevin D. West mention is in car sales:
Setting the number of cars sold as a target for sales staff seems logical because it’s linked to the dealership’s turnover and profit.
However, pushing salespeople to sell as many cars as quickly as possible might lead to giving discounts and to unsustainable sales practices, which could ultimately harm the dealership’s profit rather than improve it.
Data Quality also Matters
Bullshit in the context of data is often caused by an error in the data selection - the selection bias. An example is our behavior in the world of dating. It seems like nice people are often unattractive and people we are attracted to are often not very nice.
But why is that?
It boils down to data selection. We all have minimum standards for both: niceness and attractiveness. But the minimum standards aren’t Independent: we are more likely to tolerate somebody being a bit less nice if they are hot.
This is not the end of the story, because unless you are a sexy billionaire, chances are that not everybody will date you.
This is best summarized in a visualization.
 |
|---|
| Source: Calling Bullshit by Carl Bergstrom and Jevin West. The visualization shows the selection bias in dating. |
If you just look on the selected data (people who you would date and people who also would date you), it seems like there is a negative correlation between attractiveness and niceness. But it is caused by the selection of the data. The whole data set has a normal-distribution.
Visualizations can really help us understand patterns in data better. But they can also be used in a sneaky way to convey different stories.
Let’s have a look.
Bullshit in Visualizations
Visualizations are a great way of presenting data. Our visual-focused brain is hungry to find patterns in visualizations.
And yes, if used properly, visualizations are powerful - every Data scientist and Data Analytics Expert knows that.
However, there are many downfalls when working with data. Again, some are caused by a lack of competence in doing visualizations and there are sneaky ways to convey different stories with the same data.
- Glass Shoe: This refers to visualizations that are forced into a non-fitting visualization type. Only the professionalism and authority of this type of visualization are used. Examples are the periodic table, subway map, and venn diagrams.
 |
|---|
| Source: Calling Bullshit by Carl Bergstrom and Jevin West. This visualization is a particular perversity: the very good way of arranging the data of the periodic table is abandoned in favor of a particularly creative data representation. |
- Misused bar charts: It is important to remember that bar charts exist to compare absolute values. To make this clear, they should always contain the value 0. Otherwise, the principle of “proportional color” is disregarded (the areas in a chart should always be proportional to the data).
- Line charts with two vertical axes: They should also often be viewed with caution - depending on the scaling of the axes, different stories can be told. If several vertical axes are shown in the same diagram, it often makes sense to include the 0 in both diagrams to create a clear picture.
- Distortion of the x-axis: This has great potential to distort a statement and convey a different message.
- Binning in bar charts: Depending on how the bins (interval of a bar) are constructed, a wide variety of stories can be told.
 |
|---|
| Source: Calling Bullshit by Carl Bergstrom and Jevin West. Different binning in bar charts leads to completely different statements in the data visualization and interpretation of the patterns. |
- 3D charts: This has been very common in the 90s. The zenith of the bullshit of 3D plots are 3D pie charts. While the two-dimensional version is already difficult to interpret, the 3D version (without providing any added value) is even more difficult to interpret.
Identifying Bullshit
Critical thinking is key when calling out bullshit and fake news. There are some simple approaches you can use to support your critical thinking
1. If something sounds too good (or too bad) to be true, it most likely is.
This is a simple, yet powerful approach. Most of the time our intuition is good at identifying bullshit. We shouldn’t let mathiness, bad data quality or misused visualizations fool us.
It is not necessarily the case that the false information was spread with malicious intent. It may also be that the statement has gradually changed.
When facts are shared across multiple channels, there’s a risk of losing accuracy, like in a game of “Chinese whispers.” It’s important to check the source to see if something is true.
2. Check the Credibility of the Source
Does the article, post, news, etc. contain a source and can you trust the source?
In the scientific field, using journal rankings is a good way to distinguish between good and bad sources.
3. Fermi Approximation
In the context of fake news and bullshit, numbers are often completely off.
That’s where the Fermi Approximation comes in handy.
It’s a simple method for detecting bullshit by making rough estimates without knowing the exact numbers. For example, rounding the population of a country to the nearest power of ten for a quick estimate.
With these approaches you can boost your inner sensor of identifying fake news and misinformation, but how to prove that it’s wrong information?
Let’s have a look.
Calling Bullshit
4. Reductio ad absurdum
To use this approach, simplify a statement to such an extent that it becomes almost absurd and is then refuted.
5. Provide counter-evidence
This is a common strategy used in science (e.g. mathematics) to prove that a statement is wrong. But you can use this strategy in different contexts too.
Do you know of articles, posts, news, etc. that say otherwise?
Use this information to your advantage.
6. Show an analogy
Using an analogy can help show that a statement is wrong because it compares it to something different but similar in a way. When we see that the analogy doesn’t make sense or isn’t true, it helps us understand why the original statement might also be wrong.
It’s like saying, “If apples can’t fly, then it doesn’t make sense to say oranges can.”
7. Re-visualize data
The way data is visualized has a very big effect on its interpretation. Do you remember the bar chart example from above? That’s exactly it.
With the “right” visualization, almost any statement can be underlined. If you present data in a new way (as objectively as possible), you can refute bullshit.
8. Null Model
Building a null model (a very simple model that can do without data, for example) can be an effective way of debunking bullshit.
If the null model already shows similar behavior to the data-based model, then it can be assumed that the data has no additional effect and the modeled statement comes from statistical artifacts.
If you are calling bullshit on something someone is passionate about, things can become ugly. To avoid this, use these two rules.
9. Separate the Issue from Identity
People mustn’t be personally attacked, because only then is it realistic for them to be convinced of an alternative solution.
10. Find Common Ground
This is a rule that applies across the board. Once you have found a common denominator, it is easier to make joint assumptions based on it.
Conclusion
Fake news and bullshit is somewhat omnipresent.
But if you think critically and don’t let mathiness, bad data quality, or misused visualizations fool you, you can quickly identify bullshit and refute it with strategies such as reductio ad absurdum, re-visualizations, or using a null model.
Continue honing your critical thinking skills and be vigilant in evaluating information in the digital age and you won’t fall for fake news easily.
Thank you for reading until the end! I hope you’ve enjoyed my article and learned something.
For this article, I’ve used the book “Calling Bullshit” by Carl T. Bergstrom and Jevin West as a main resource. I can highly recommend reading the whole book. Please support your local library and buy from them.