AI has beaten human lawyers in a contest about law – and it wasn’t even close
Case Cruncher Alpha might cause some lawyers to look over their shoulder…
An artificial intelligence program has beaten human lawyers in a contest about law – and it won by a huge margin.
Overseen by independent judges, Case Cruncher Alpha took on 100 commercial lawyers. Both had to try to predict the outcomes of real PPI (payment protection insurance) mis-selling cases based on given evidence – pre-empting the decision of the Financial Ombudsman Service before it happened.
From more than 750 cases, and in an unsupervised environment, the program correctly predicted 86.6% – smashing the lawyers’ 66.3% accuracy.
“The main reason for the large winning margin seems to be that the network had a better grasp of the importance of non-legal factors than lawyers,” said the innovators – who will soon write a research paper on their approach to the challenge.
“Evaluating these results is tricky,” added the researchers. “These results do not mean that machines are generally better at predicting outcomes than human lawyers.
“These results show that if the question is defined precisely (such as – was this complaint about PPI mis-selling upheld or rejected), machines are able to compete with and sometimes outperform human lawyers.”
The team said technology such as theirs could solve “legal bottlenecks” at organisations “permanently and reliably”.
