In AI, Big Data May Be the Big Dodo
By Kripa Rajshekhar, the Founder of AI Pioneer, Metonymize.
Some AI experts are now acknowledging something that lawyers have always known: good laws need more than just many, many examples.
Reasoning about possible futures and interventions is crucial, if not essential to the practice of law. New computational tools are being built to offer these capabilities, represented by a movement called Third Wave AI.
The stakes are high: Big Data AI systems often repeat the mistakes of history and can be too complicated to explain or understand.
The time for the AI and Law community to shape the deployment of AI is now. This tale offers pointers on what AI can and cannot do today, and what it might be able to do in the future.
The White House, November 8, 2020
‘We’re getting fracked,’ said special agent Merlin. ‘Data’s the new oil, sir, and the fracking will not stop. You must sign this bill.’
The President looked distractedly at the vulture drones disemboweling the battery packs from the self-driving cars. Out in the distance, a dark smoke over the Atlantic shrouded the horror: the ocean foamed with self-replicating robot-plankton, transmitting energy wirelessly into an endless armada of pilotless ships. This Leviathan knew nothing, and sought to control everything, it fed on data and had snipped its power cord.
(Rodney’s Bookstore, MIT Campus, August 28, 2018)
Gargi mocked Yanis: ‘That’s really pretentious, even for you. How can you control something while knowing nothing?’
Gargi was a Law major, minoring in Computer Studies (CS). Yani was shooting for a double major in Finance and CS (a PhD of course) in time to graduate with the class of 2022. He had interned at Adam as a Sophomore (in High School). Adam was a super secret AI startup, built by a splinter team from a large tech firm that was going to ‘solve intelligence‘.
‘We are children of the Petabyte Age, G,’ Yanis replied. ‘I don’t like that book he wrote on Lisp, but I am with Peter Norvig when he said that ‘All models are wrong, and increasingly you can succeed without them.’’
‘With Big Data the scientific method looks really, really quaint: Hypothesize, model, test? Really!?’ he said, as they walked past Carlo Rovelli’s biography of Anaximander.
‘As TED’s Chris Anderson wrote in a prescient WIRED article over a decade ago: ‘There’s no reason to cling to our old ways. It’s time to ask: What can science learn from Google?,’ he concluded, dismissing a lithium trade alert on the holo-lens with a saccade of his left eye.
‘Did you read Judea Pearl’s July 11, 2018 paper? ,’ asked Gargi. ‘He would probably not agree with your assessment of the scientific method. In seven pages he offers seven constructive and actionable pieces of criticism for the dominant data-above-models approach in machine learning today. He also identified the model-based vs. model-blind dichotomy as the key to understanding the ancient rivalry between Babylonian and Greek science.’
Gargi picked up Douglas Hofstadter’s Godel Escher Bach and said: ‘It’s a dense read, but one that really resonated with both sides of my brain. It feels like we are replaying a saga from thousands of years earlier. You know, Pearl won the Turing Prize for his ground-breaking work in AI, we might want to take his ideas seriously.’
I couldn’t believe Artificial Lawyer had asked me to write another article after the 2,392 word tome that I sent back on May 5, 2017. Maybe I should have better understood the emphasis on: ‘Would you be happy to write a short piece setting out what an AI system can and cannot do today, and what it might be able to do in the future?’
Well, here we are. And this is the ‘short piece,’ covering three topics.
- What AI can do today
- What it cannot do today
- What it might do in the future
I have reproduced a table from Judea Pearl’s brilliant article (see link below). The table answers these three questions, literally, in that order.
Even if you just read the questions and examples very, very, slowly you will understand the space of possibilities for AI better than many of us who have AI plastered on our LinkedIn profiles.
A few points to help you make the connection:
- Most of what passes as AI today is doing ‘Association’, often badly,
- Most of what passes as AI today cannot model ‘Intervention’, not in a way that will pass any level of rigorous scrutiny. (Laws are interventions and often apply to interventions),
- If the Third Wave AI movement revives the place of models over data, we may in the future have AI systems that ‘Imagine and Retrospect’.
These new tools being developed will open up a new set of possibilities for deeper automated legal reasoning, at scale.
The current AI tool-kit is good at association, but we must remember that human reasoning in areas like the Law go far beyond this: banning cigarettes, adjudicating crimes, appealing misdiagnoses, and more. As Pearl’s examples reveal – AI must go beyond mere association to engage such areas.
I wish I had time to offer you more examples, but alas – the venture capitalists are beckoning and Elon Musk looks like he may be out of a job soon. If you made it this far so long, and thanks for all the fish!
- Weapons of Math Destruction, Cathy O’Neil
- The First Scientist: Anaximander and His Legacy, Carlo Rovelli
- THE END OF THEORY: THE DATA DELUGE MAKES THE SCIENTIFIC METHOD OBSOLETE, Chris Anderson, WIRED
- Theoretical Impediments to Machine Learning With Seven Sparks from the Causal Revolution, Judea Pearl
- Gödel, Escher, Bach: An Eternal Golden Braid, Douglas Hofstadter
- The Third Wave of Legal AI, Kripa Rajshekhar
- So Long, and Thanks for All the Fish (Hitchhiker’s Guide to the Galaxy Book 4), Douglas Adams
Kripa is the founder of Metonymize. With research backed by the National Science Foundation (NSF) and the Defense Advanced Research Projects Agency (DARPA) the Metonymize Explainable AI platform helps clients transcend the limits of current generation machine learning.