Harvey Trains Open Source Models To Encode Law Firm Workflows

Harvey CEO, Winston Weinberg, has confirmed to Artificial Lawyer that they are now doing Proof of Concept studies with law firms to train open source LLMs to ‘encode’ their way of working on certain complex matters.

Weinberg underlined that the value was not just in encoding how complex work streams take place inside a law firm practice group, but there was an opportunity to include how that work is done with specific longstanding clients.

I.e. you encode the entire experience, from law firm to client, so that automation can be applied in a more precise and customised way to this recurring type of need.

The development comes as Kirkland & Ellis is working with Palantir – after its now famous ‘$500m AI investment’ announcement – which stressed that they were seeking to ‘bottle their secret sauce’ as it were, for certain areas of work. In fact, Kirkland made quite a lot out of the idea that what they were doing would be ‘unique’ and set them apart from what other law firms were capable of doing when working with legal AI platforms. And Artificial Lawyer has some thoughts on the whole secret sauce idea, more on that below.

After that announcement, AL found that Kirkland is hiring AI infrastructure experts with experience of working with GPU clusters – which strongly hinted at their own open source training strategy that will, presumably, work alongside their partnership with Palantir – but, we shall see.

Meanwhile, Thomson Reuters – see here – has also been working with open source LLMs to train them on the data giant’s huge store of legal information, which then will act as an additional AI-backed resource for their research offering.

In short, the idea of post-training open source is coming back. Previously, the strategy had been rejected on the basis that general models alone would become so good they would make such niche and customised approaches pointless. But, it seems the market is now moving back to this idea.

There are several reasons why this is so. One is a simple one: data security. A second one is that clearly legal AI experts believe that you can get better performance from specific training now. And third, that one of the triggering elements here may be agentic flows. I.e. this isn’t just about tapping general language understanding of an LLM, but putting in place more narrowly customised areas of training to support equally specific workflows, bringing together reference data, complex playbooks – i.e. digital twins of methodologies, along with the fine-tuned open source LLM that is pointed at specific legal work product for certain clients.

In short, this is all about customisation. And then through that more holistic customisation you can achieve a real improvement over the general models.

The Harvey move is part of a wider project to widen what can be achieved with legal AI. It was outlined by co-founder Gabe Pereyra last night on X.

He explained: ‘We are working on the first model in our legal foundation model series, inspired by Cursor’s Composer.

[There are] two goals:

1. Allow us to serve frontier intelligence across our product surface areas at an affordable price and a strong security posture.

• Create the foundations for law firms to build their own specialized models and own their own intelligence.’

Now, for AL the key phrase there is ‘own their own intelligence’ – and this goes to the heart of what Kirkland was getting at. One can see why this appeals to law firms, which understandably believe they have something unique in how they work, over and above just having a group of very smart lawyers.

As to the training, Pereya explained: ‘The model series will focus on complex client matters that span months and take dozens of associates.

‘The agentic system will learn to control legal tech tools, sub agents and ask for help from frontier models or human partners, much like a senior associate.’

I.e. this is not just some training, there is a whole orchestrated environment of tools, data and more as part of this. In short, it’s really a substantial project for each workflow, and for each law firm. But, this is what will be needed if we are to truly move to the high quality automation of complex legal work streams.

He continued: ‘We’ve open sourced benchmarks for evaluating our initial post training work that represents work done by associates and in-house lawyers. We are scaling these significantly using synthetic and human pipelines as well as building private evals for firms.

‘Open sourcing this data has allowed us to quickly validate the feasibility of post training open weight models for legal work. With our research partners we’ve already shown promising results post training open source models to approach frontier performance.’

Exciting stuff. Now, onto bottling secret sauce.

Does a law firm, in fact any law firm on the planet, really have so much special and unique capability in a certain area that no other firm has it? Do certain law firms have such incredible proprietary legal insights and ways of forming contracts for clients, that no other lawyer can do it that way?

AL has to say this site is sceptical about absolute exceptionalism here, at least on a grand scale. This site spoke to some experts on this and they largely agreed that aside from some very, very esoteric tax structures that truly are unique perhaps to the genius of one partner at a particular firm, that most final work product does not really have a moat.

Now, would a High Street lawyer understand a futures contract drafted by a New York elite firm? No. But, their rival on the other side of the road would. Documents get shared – that’s the nature of legal work. Rivals learn. Things get made public. It’s thus very hard to really bottle and keep secret ‘your sauce’.

But, your ways of working, especially for a particular client, may be a bit different to other firms. The way you assemble past data, the client’s playbooks, the orchestration of the whole process, may be special to you…..although, AL would again say there are limits to this uniqueness. Ford makes cars differently to Tesla, but not so wildly different that it feels like they are on different planets.

So, what can you really bottle that is so massively unique that no-one can copy it? It is probably just the relationship for one work stream with one client. That relationship is special to you. The client can do that work with another firm’s lawyers, but the experience will be different. In short, it all gets very human. I.e. what sets you apart from another firm is the fact that your lawyers have a personality, they have ‘character’, so does the GC. Combine those things and it’s rather like a marriage.

More about Harvey here.