Where did it all go right for predictive analytics?

donald trump

Republican Donald Trump has won the US presidential election, much to the apparent surprise of opinion pollsters, which had predicted a win for Hillary Clinton of the Democrats. 

Most traditional opinion polls had placed Clinton ahead by 1 or 2 per cent at least, with the average – or poll of polls – showing a significant enough lead for Clinton that she would win by some margin.

However, that clearly has not transpired, and in fact it is Trump who has won by a 1 to 2 per cent margin, and now media outlets are all scratching their heads this morning and wondering why they ever listened to, or even quoted, opinion polls.

The problem with traditional opinion polls is that they rely on talking directly to voters, who, being human, are capable of lying, or at least concealing their true intentions and behaviour – whether it is before or after the event.

So even the exit polls – opinions collated immediately after the event, often standing outside the voting booths – can be inaccurate.

But while doubts about the validity of traditional polls are now being raised – because almost all of them suggested a Clinton win – there was at least one artificial intelligence engine which correctly predicted a Trump victory.

MogIA, an AI system developed by Sanjiv Rai, who founded Indian start-up Genic.ai, has reportedly correctly predicted the past three presidential elections.

In a report broadcast by CNBC before the Trump win, Rai was quoted as saying: “While most algorithms suffer from programmers/developer’s biases, MoglA aims at learning from her environment, developing her own rules at the policy layer and develop expert systems without discarding any data.”

Rai added: “If Trump loses, it will defy the data trend for the first time in the last 12 years since Internet engagement began in full earnest.”

MogIA is said to use 20 million data points from public platforms such as Google, Facebook, Twitter and YouTube – social media, basically – and then makes predictions based on conversations people are having, as well as their other online behaviour.

It could be said that MogIA uses an approach which owes more to the internet of things than to the networks of people traditional opinion pollsters have so far relied on.

Rai said: “Granularity of data will determine progressively lesser bias despite the weightage of negative or positive conversations.”

What Rai might mean is that if someone does an internet search for “how to vote”, and then views a video about Trump, this could lead the AI to decide that this person is likely to vote for Trump.

This may hold true irrespective of whether that same person had previously said in conversations that they preferred Clinton, for example.

Unlike some of the data analytics practised in the run-up to the elections by both the Republicans and in particular the Democrats, where individual voters are profiled – along with their names, addresses and all other available information in something called “micro-targeting” – MogIA claims to collect information anonymously.

Rai told CNBC: “If you look at the primaries, in the primaries, there were immense amount of negative conversations that happen with regards to Trump.

“However, when these conversations started picking up pace, in the final days, it meant a huge game opening for Trump and he won the primaries with a good margin.”

One thing that has been pointed out by some opinion pollsters and political commentators is that Clinton had not been doing well in some primaries against her Democratic opponent, Bernie Sanders, which may have provided an indication as to which way those states would vote.

However, maybe nobody could have foreseen how the general antipathy towards established or career politicians would translate into lifelong Democrats voting against Clinton, whose husband Bill was president for eight years between 1992 and 2000.

hillary-clinton

The general consensus seems to be against political dynasties, which the Clinton family may have become if Hillary had won.

On the other side, Trump beat at party selection stage his Republican rival Jeb Bush, whose brother George W was president from 2000 to 2008, and whose father George HW was president from 1988 to 1992, after being vice president alongside President Ronald Reagan and the director of the CIA in the 1970s.

Much like the mood in the UK which led to the British exit from the European Union, the electorate seems to have become discontented with politicians and is more willing to listen to people like Nigel Farage, of the UK Independence Party, and Trump, who is arguably only nominally of the Republican Party, many of whose leading members had refused to back him.

As well as MogIA, there were other AI systems which predicted a Trump win.

Among them was an AI developed by Nick Beauchamp, an assistant professor of political science at Northeastern University – also mentioned in the CNBC report.

Beauchamp’s AI system analysed more than 100 million tweets in the 2012 election and found that they closely mirrored the results in state-level polls.

“These results provide not just a tool for generating survey-like data, but also a method for investigating how what people say and think reflects, and perhaps even affects, their vote intentions,” Beauchamp says.

Another AI-based data analytics system has been developed by Helmut Norpoth, a professor of political science at Stoney Brook University. In results generated a couple of months ago, Norpoth’s statistical model had predicted a 97 percent to 99 percent chance that Trump would win the 2016 presidential election.

Norpoth’s model – which uses the candidate’s performance in the party’s primary polls as its statistical basis – is said to have correctly predicted the outcomes every general election dating back to 1912.

As quoted in the TheBlaze.com, Norpoth said before the election last night: “The bottom line is that the primary model, using also the cyclical movement, makes it almost certain that Donald Trump will be the next president.”

Norpoth had acknowledged, however, that “this is almost too much to believe”, but added that you could “take it to the bank”.

Between the election of a billionaire businessman reality TV star as president and the decision by voters in the state of California to legalise marijuana, times are certainly changing in the political world, and perhaps opinion pollsters which supply information and predictions to media outlets should change their data collection techniques to reflect the new reality we live in.

One other interesting fact which may be worth noting is that the Trump camp employed the services of Cambridge Analytica, the same data analytics company which is said to have masterminded Brexit.