Despite feedback from across the market that general genAI tools such as Microsoft Copilot and ChatGPT are not great for legal work, this year’s ILTA survey has shown they dominate when lawyers turn to LLMs for assistance.
That said, ILTA’s survey is always quite a peculiar collection of data, as it combines the views of many small law firms alongside the largest in the US. The total sample appears to be from 536 members of the legal tech organisation, and looking at the sample numbers in other questions in this early Executive Summary one can see that firms with over 700 lawyers are one of the smaller groupings.
So, please take all of this with a pinch of salt, unless you are at a small to medium size firm, in which case this data may be right up your street. Larger firms may have a very different view, and from what Artificial Lawyer has seen it’s clear that many BigLaw firms are miles ahead of the broader market – and in some cases have been so since last year.
That said, the data on genAI is still important, especially for those legal tech companies that want to build a broad revenue base by selling into market segments beyond the larger firms. Moreover, if we really want to see AI ‘transform the legal sector’, then all the smaller and medium size firms will have to join in with this evolution as well.
So, what does this limited look, with plenty of caveats, tell us? (N.B. The main survey will come out later this month, and this release is designed to be something of a teaser.)
Here, at least for this site, are the most important data points:
‘Is your firm using generative AI tools for business tasks?’
– Only 37% of respondents said ‘yes’.
– However that represents a 22-point increase over last year.
- Microsoft Copilot for Office 365 was most popular (at 55% of the above 37%), ‘but that response was a little deceptive’ ILTA explained and added: ‘We know this because we also asked, ‘What are your firm’s plans for Microsoft Copilot?’ and on that question, 58% of respondents chose Investigating. Only 3% are using it for timekeepers [i.e. lawyers using it right now], although 19% indicated a pilot was underway.
This shows that while Copilot dominates, presumably because it comes with most firms’ MS packages, and is thus relatively easy to get into because of this, its penetration is still quite low. Moreover, a very small number of lawyers are actually using it today, even if a firm has it on tap.
So, while it looks like Copilot is doing well, its hold on the legal world is clearly quite fragile and exists primarily, it would seem, because of its apparent convenience. It also shows that lawyers at the sample firms – which as noted with ILTA’s survey tend to be small to medium-size ones – are not really engaged with it that deeply.
Elsewhere in the genAI questions responses we can see that:
- Overall, ‘the top four responses for choice of genAI tools for business tasks were non-legal-specific’, such as ChatGPT and Copilot, as noted above. [ILTA doesn’t say what the two other ones were. Also, ‘business tasks’ could mean a lot of things that are not legal work, e.g. marketing emails.]
- When it came to the other genAI tools the following were the next most popular with the survey sample:
- ‘Westlaw AI-Assisted Research and CoCounsel (owned by Thomson Reuters) were tied at 22% in positions five and six on that list.
- Lexis + AI appears at 16%
- And, Harvey at 9%.’
- There were several other legal-specific platforms on the list as well, but they don’t give the info. In another chart related to the use of emerging tech, or awareness at least of new tech, other companies mentioned included: Spellbook, Draftwise, and Henchman (which is now part of LexisNexis).
And when in a separate question the sample was asked what they ‘believe genAI will be used for?’…..i.e. not using right now…..but what maybe, maybe in the future it may be used for, people said:
- Research (73%)
- Summarization (70%)
- Creating initial drafts of documents (69%).
So, what does this tell us?
Well, as with every ILTA annual survey the results are…..how can we put it…not exactly resonant of a legal tech revolution. Rather, the survey highlights the day-to-day reality of smaller firms, where the focus is much more on ‘IT’ than the innovation side of the legal tech spectrum. In fact, the executive summary spends plenty of time on what laptops firms are using and how the Cloud is faring these days.
But, if we get beyond the ineluctable modality of life within a medium-size law firm’s IT team and look at the bigger picture, what do we see?
While not exactly inspiring, the data shows that genAI – a technology that most law firms hadn’t even heard of back in mid-2022 – has become a small, but very real part of life for some in the sector. That in itself is a very positive move.
That so many in the sample were using (or given the info above, perhaps just investigating at this stage) general, non-legal sector genAI tools, is a bit depressing, to be honest, although not surprising. If you have paid $xxx as a firm to get your MS suite, and they bung in Copilot, then you may start there.
Of course, as noted, no-one this site has ever spoken to who has used it says it’s that good or useful for legal work. Every single legal innovation person Artificial Lawyer has engaged with since genAI arrived has highlighted that without refinement of responses and system prompting for legal use cases, as well as highly developed tooling in the application layer above the core LLM, then the ‘raw’ results from general LLMs are limited in use.
But, maybe if you just want a quick summary of a document that’s OK? But, to use Copilot on its own, in a totally ‘raw’ state for legal research…? For drafting contracts….? That’s clearly not the best choice. And in fact, drafting a whole contract from scratch with an LLM seems like a bad idea in itself – something this site picked up on back in late 2022, as it seemed obvious even then that making something so complex, so precise as an entire legal contract with a generalised tool was not a great plan.
Overall then some positives, and plenty of not so positives. The saving grace of this survey, as noted, is that it doesn’t really represent where the most innovative large law firms are. But, it’s a realistic snapshot of the wider market and will immediately sober up anyone who believes an ‘AI revolution’ is about to sweep the entire legal market.