ILTACon in Nashville is now in full swing and there are several thousand people here, with plenty from all over the world. Here’s a couple of things Artificial Lawyer has seen so far.
The Keynote
To start the conference British keynote speaker Hannah Fry, Professor in the Mathematics of Cities, UCL, gave a masterclass on the risks of blindly trusting technology – which of course was ironic for a conference focused on the wonders of….er…technology. But still, some valuable lessons here.
Fry had many great examples and excellent anecdotes, all of which illustrated the challenges of trusting AI and algorithmic systems without understanding the wider real-world context in which they work.
In short, humans live in the hyper-complex physical and often emotional world. Software is just code, even if it’s quite clever it cannot truly behave like a human because it’s not living in the world as we do. It takes things literally. It misses key steps. It thinks getting an answer is the number one objective, even if sometimes that answer makes no sense to a person.
An example of the chasm between us and machines was illustrated by a study to find out how people chose words that would differentiate themselves from software. These tended to be very human motifs such as ‘love’ and ‘empathy’. But, the one that really stood out (see pic) was something very much in the realm of living things: poop.
It sounds absurd, but there is perhaps a philosophical point here. We want to anthropomorphise AI, but it can never be human, it can at best mimic narrow aspects of us, these living things. And some things it just can’t do.
Another key point Fry made was about ‘swapping’, by which she meant a program will tend to take the simplest path to get to the desired result. So, for example, a truck was using neural networks to figure out how to drive. However, the software saw through the human complexities of driving and decided that ‘keep the grass verge on the right and move forwards’ was its key purpose. This worked fine when it was on normal roads, but when it got to a bridge, and the grass verge was now perpendicular to the road because of the river, the truck went off the road and crashed, almost killing the people inside.
She also then gave several other examples, including one from the justice sector about automated sentencing software, but all had the same message: beware of assuming that an algorithm – or an AI system – can understand the wider context of what it is delivering. And in the light of genAI and hallucinations i.e. the system making answers to make you happy even if they are wrong, this was an important point to reiterate.
Overall, a useful lesson on staying grounded and not believing we can outsource our minds entirely to lines of code.
One last word and something that puts legal tech to its ultimate test was when Fry asked: if we had a choice between a human or an AI judging us in court, which would we choose? The vast majority of the audience voted for a human judge. And if a room full of legal technologists would prefer a human to evaluate them rather than an AI, what does that tell us…..?
(To be continued.)
The GenAI Wars
Thomson Reuters and LexisNexis may not publicly talk about the rivalry between them – but off record, man…it’s there for real. So, it was interesting to see both companies announce major genAI assistant developments on day one of ILTACon.
Later TR gave a talk about its new CoCounsel 2.0 (see story). David Wong, CPO, said that ‘this is the most meaningful work I have done at Thomson Reuters’. He then stressed the fact that they have all the three main food groups when it comes to AI tool development: ‘We have the data, the expertise, and the tech. Few have all three in such quantity and depth.’
And that’s a key point. If you look at some of the challengers out there, there are few that have all of that lovely, rich data. LexisNexis is one, so too is vLex. But, Wong is right, having genAI skills and having a smart team is great, but a ton of authoritative legal data is the cherry on the cake if you want to offer a really broad genAI platform. (Of course, some of that data could be brought to the party by a law firm’s own DMS, or it could be free to access legal data – but, nevertheless, there is data sitting at the centre of all things genAI.)
Then Jake Heller spoke, the previous CEO of Casetext – which TR bought – who is now Head of Product for CoCounsel. He made several key points:
- It’s only been one year since formally coming together as one company, i.e. ILTACon last year.
- Most M&A deals don’t go as well as hoped, but they have smashed expectations, he said.
- The state of the art in genAI is moving fast – and that is feeding into CoCounsel 2.0.
- The Number 1 request from clients is comparing documents, e.g. whether for contracts or testimony. This is very difficult to do, ‘but we have done it’. (P.S. readers may want to check out Hebbia and Draftable also, if doc comparison is of interest, among others.)
- The Number 2 request has been to take docs from a firm’s DMS and with its partnership with Synchly they can now do this. (Although, will all law firms want to create a direct DMS data pipe?)
- And finally, with the new CoCounsel High Throughput Beta facility they can ingest around 100,000 documents an hour….so that as Heller explained: ‘While you sleep AI reviewers are working for you.’ And that will be a powerful tool for due diligence and perhaps even aspects of eDiscovery. (And again, Harvey and RobinAI are companies you may want to look at also in this regard.)
- Oh…one last point. The talk ended with the point that CoCo 2.0 is now ‘Much more flexible and accurate’ – only one way to find out.
Overall, the talk by TR was really a talk about where legal tech is heading. Companies leveraging AI are building very broad, interconnected collections of what people are now calling ‘skills’, i.e. tasks that genAI can do. Most of this started in November 2022, and look how far things have come in terms of what’s on offer.
Reality Check in Small Law Land
One of the great things about ILTACon is the chance to meet a lot of IT folks from small and medium size firms, mostly from across the US. At lunch this site chatted to a table of people who are far removed from the world of BigLaw and huge investments in AI.
The key points they made were:
- None of them are using AI tools made by legal tech companies. But one was using ChatGPT across the firm, but the lawyers had orders not to put any client data into it. They were told to use made up names like Clark Kent and Lex Luther if they wanted to get legal responses involving different parties. (Guessing that one key reason they used raw ChatGPT – despite its limitations – is cost.)
- The main focus instead was on things like day-to-day security needs. AI legal tech tools would mostly come later, they said.
- Partners at firms wanted to use gen AI – but did not know why from a business perspective, but they’d seen stories about it and wanted to use it, especially for speeding up drafting documents.
- None of the firms had considered the economic aspects of AI, e.g. fixed fees vs billable hours in a world of increased efficiency. In this respect they are in good company with most of BigLaw.
- None of the clients had asked the firm to use AI. But, some said they had a lot of individual clients and small companies, so didn’t expect such requests.
- All of the firms said they would eventually use legal tech genAI tools, but wanted to put in place the right safety barriers first – although as noted ironically enough one of these firms was using ‘raw’ ChatGPT outputs. Plus, without engaging with legal tech companies how would the firms know what barriers and guardrails were needed and for what use cases?
Overall, there may be talk of a genAI revolution among some of the largest firms – and some smaller firms too (see Harvey’s expanded focus on the mid-market) – but in truth, many small and medium-size firms are as far from using legal genAI at scale and in a meaningful way as they were last year. But, then, things take time in law land.
On a more positive note: all of the smaller firms spoken to knew plenty about genAI and what it could do, and all accepted that eventually they would adopt it across the firm. They just did not know when, or what it meant economically for them.
—
Panel on GenAI, Tuesday 3.30PM at ILTACon with Harvey and Thomson Reuters
And if you are here in Nashville then maybe come long to this…..Artificial Lawyer’s founder, Richard Tromans, will be moderating a panel on ‘How today’s lawyers are enhancing their practice with AI’ with companies such as Harvey and Thomson Reuters taking part.
It’s at 3.30PM Presidential Ballroom South, Tuesday, August 13th.
That’s all for now. Plenty of sessions to attend!