Predicting the future is popular, especially at the start of the year. But how do we know which predictions might come true? And how can we get better at it? Eric Shepherd looks into his crystal ball.

"Prediction is very difficult, especially if it's about the future!" said Niels Bohr, the Nobel laureate in Physics and father of the atomic model, and social scientists seemed to agree. As if to prove Bohr's point, the eminent physicist Stephen Hawking admitted in later years that some of his early predictions about the universe were not entirely correct.

Despite the difficulties inherent in planning for the future, humans are the only species on the planet that make deliberate long-term plans. We’re strongly motivated by the desire to contain the risks that we associate with the uncertainties of future events.

The impetus to enhance our ability to make reasonably accurate predictions is massive. Businesses operating in volatile economic environments are eager to anticipate changing customer tastes or technological breakthroughs. Likewise, governments have much to gain if their estimations about the repercussions of political alliances can be understood.

One of the biggest hurdles to overcome for anyone in the prediction game is to overcome our cognitive biases, be they conscious or unconscious. But we can train ourselves to think more rationally while attempting to predict the future. Today, we have many more scientific tools at our disposal now than previous generations ever did. The accuracy of insights derived from reliable data is considered far superior to those arrived at through subjective methods.

Researchers at Ivy League universities have devoted considerable time and effort to this challenge. In a prominent study aimed at improving geopolitical forecasting, they recruited experts from various fields. Sadly, they discovered that the predictions made by these forecasters, over several years, were only marginally better than chance and worse than even basic analytical algorithms.

The research did, however, lend credence to the argument that a richer set of inputs yields better results. Individuals who took several viewpoints into account appeared to make better predictions than those who stuck to a single perspective. This is a clear call for decision-makers to pay heed to diverse voices that can prove valuable in avoiding blind spots.

After scrutinizing hundreds of thousands of forecasts on events of worldwide significance by educated participants, the researchers generated some key takeaways. They concluded that if the underlying factors mentioned here are given due importance, then making predictions need not be a random process.

Being smart helps

The research participants with particularly high aptitudes displayed a tendency to be more precise in their predictions. However, raw brainpower seemed to provide a significant advantage at the early stages of the investigation. As the novelty of partaking in a new exercise wore off, higher intelligence levels seemed to matter less than before.

Specialized knowledge benefits

Possessing specialized expertise in a particular discipline seems to improve the odds of making accurate projections. While initial research did not point to a direct correlation, specialized capabilities in a specific field do appear to have a positive effect on prediction results.

Practice makes perfect

It takes a certain amount of practice to excel at anything. Prediction abilities are no different. It is worth noting that 'superstar' forecasters, who came up with predictions that turned out to be mostly right, became that way over time.

Teaming up works!

Sharing information among team members rendered better accuracy. This was determined by placing forecasters in a group or asked to make individual predictions by themselves. A Forbes article claims that when recruits were placed in groups for forecasting, they outperformed individuals doing the same task.

Talented individuals thrive together

The quality of predictions was found by researchers to improve when ordinary participants were placed in teams. However, over time, the polarization in this group increased as members displayed an unwillingness to collaborate. On the other hand, the team of 'superstar' forecasters appeared to work well in unison with each other.

Flexible mind-sets improve predictions

Researchers also observed that people with the ability to look beyond their personal prejudices showed higher accuracy in their predictions. In contrast, individuals with rigid thinking displayed poorer performance. While authorities on human psychology consider these characteristics to be in-built, being able to keep an open mind primarily depends on the circumstances.

Training avoids reasoning errors

Even experts can miss the mark by a wide margin when predicting the future. This is because of the very human propensity to overstate just how different the future will look from the present. The good news is that we can be trained to consider alternate scenarios and avoid common biases and errors in thinking. In studies, forecasters who received probability training, on the statistics of past cases, achieved better results than those who don't.

Haste lays predictions to waste

Participants, who invested more time in reflecting before articulating their predictions, produced fairer outcomes. Abilities improved further when these individuals were placed within a synergistic group.

Revisit beliefs to strengthen forecasts

The participants who came up with the best predictions were those who possessed the intellectual humility to alter them when new information was offered. They updated their suppositions based on fresh evidence and had to make fewer course corrections going forward.

Conclusions

These findings bring us to an exciting frontier where we can realize how to get better at forecasting by taking genuine stock of our abilities and shortcomings. A central challenge encountered by researchers was that, despite being trained in probabilistic reasoning, the predictions people made were no better than those generated by algorithms. This led researchers to suggest that reliance on statistical models is a viable method of refining predictions.

But in an era where many human functions have already been taken over by technology, a balance must be struck between humans and machines. Computers are not currently able to exercise the kind of judgment that individuals can. But surely, we are capable of fostering aggregation algorithms that can distill collective human wisdom. In due course, an optimal combination of data and brainpower will yield more than just the sum of its parts.

Now, for our 2020 predictions, all we need is an algorithm to make that combination a reality. Or a new crystal ball.