Addleshaw Goddard is a UK-based law firm that was quick off the mark when it came to genAI. Artificial Lawyer spoke to Kerry Westland, head of the innovation group, Michael Kennedy, senior innovation manager, and Elliot White, director of innovation – known collectively here as Team AG – about how they’ve handled the new wave of legal AI at the firm and what they’ve learned so far.
Team AG said that they’d examined literally dozens of tools with a genAI aspect and had moved quickly to experiment and then bring aboard what they needed as a firm. They noted that they had started to look seriously at genAI in 2022 and that after talking to the management team of the firm got the go-ahead for a full-scale expedition.
And so began a great adventure, which is still ongoing. They have now ‘really got under the hood’ of genAI. They’ve also built their own ‘AGPT’ to have what they call ‘a safe space’ to see what an LLM can do.
As to AGPT, which perhaps we can call a ‘moderate thickness wrapper’ around a GPT-4 LLM, but which is not connected to the firm’s DMS, it ‘has been really useful’ and that around 1,800 people at the firm have now used it for things such as doc summary, interrogating contracts, and similar tasks that many lawyers have found an LLM can do well.
Overall, after exploring various aspects of LLM technology they have brought aboard in addition to their AGPT, Canada-based Spellbook, Thomson Reuters’ CoCounsel, and also Microsoft Copilot. And, of course, they’re still using earlier AI tools such as Kira, which is part of Litera.
With the LLM tools in particular they’ve found that the AI can infer things that were hard to spot before. One example they found involved parking places for shopping malls in real estate documents. Parking places are not usually designated by a specific clause in a contract and so are not immediately visible in a very large document, but they’re a key aspect of evaluating shopping malls. LLMs don’t always need specific clauses to decipher what ‘a contract means’ and so can pick up on things that might be missed otherwise, such as something as seemingly obscure as parking places.
Overall, each application has their strong points, and when it comes to a use case like contract review, different applications work best for specific needs or certain levels of volume involved in a matter. In short, there is no one-size-fits-all genAI approach (or AI approach in general), not even for something as focused as contract review work.
Moreover, they don’t want to be tied to any particular system. As with other law firms, the view is: ‘We don’t want to be under one vendor. We like independence.’
They add that via Azure, they can also build out use cases as they choose and are not married to any one LLM model either. That means they can look at use cases, costs, and which set-up makes sense. Overall, they want flexibility – so, for those trying to create one massive, all-seeing, all-doing genAI super product for lawyers, good luck….. Team AG at least probably won’t want to go all-in on that.
What Else Have They Found?
They have come to see that chunking is increasingly an issue, i.e. breaking down a document into far smaller parts to get better results before it’s handed over to an LLM. Even though ‘context windows’, i.e. the amount of text you can shovel into an LLM to examine, are growing, analysing small chunks of text tends to get the best results.
They’ve also spent a lot of time working on system prompts – the instructions that sit behind the interactions a user might have with an LLM via a chat interface, which steer and guide the responses without the user having to input these ‘back end’ instructions themselves each time.
And they’ve looked at ‘temperature’, i.e. the gradient of creativity the LLM uses when generating an answer. Team AG said that 0.55 is the best temperature. Not too hot and wild, not too cold and limited.
That said, they are clear that you cannot 100% remove hallucinations, but you can get to a point where they have the least chance to appear, or have an effect on the results.
They’ve also found that some prompts used on one LLM or product won’t work so well with others, and in fact can even clash with the system prompts already inserted into some, such as CoCounsel.
What Next?
As Team AG explains, this is just the start, even if they’ve already been working hard on genAI for some time now. This is really just the beginning of something that will grow and grow.
‘These are the worst LLMs we will ever see,’ they note.
And while some argue that generative AI tech is plateauing at present, or at least not showing the huge leaps expected compared to what was there a year ago, the reality is that what can be done is already impressive and over the medium-term it seems inevitable that what LLMs can do will improve.
At some point we will get to GPT-5, other LLMs will also improve. New ones may even come to market in the years ahead. Approaches to using LLMs will also evolve, just as ways of using the internet morphed slowly to support whole new types of applications that could not have been predicted at the dawn of the World Wide Web.
It would seem churlish indeed to think that we have ‘topped out’ already. What perhaps has changed is that the initial hype wave – and wow, was that a powerful wave – is starting to recede at last. And that’s a good thing (and Artificial Lawyer has to say this site is very glad it chose 2023 to take a sabbatical, because now we really are at the point of substantive change for lawyers who are using genAI tools for client work).
There is also perhaps no going back when it comes to being able to ask an LLM a question in natural language, for example about a contract, and then just see what it can come up with – something that would be very hard to do effectively with older forms of NLP / ML that usually needed specific training on equally very specific legal language.
Team AG certainly does not believe that we have reached a plateau already.
‘This will be the most transformative technology in the legal sector,’ they conclude.
Artificial Lawyer has to agree. But, as Westland, Kennedy, and White, have shown, leveraging genAI to make a difference to a law firm takes a lot effort. Addleshaw Goddard has put the time in to understand what’s out there, how it works, and how to make real use of it – while still keeping their regular legal tech projects going at the same time.
It’s clear then – at least at this stage – that to get the most out of genAI there is a learning curve and you need to put the time in to get where you need to be. In Addleshaw Goddard’s case, they have done that and learned a lot, and are now looking ahead to the continuing journey