Agents are the future of legal AI, but how will they really work at scale, and what is a law firm world model? Plus, how will this change the structure of legal businesses? Artificial Lawyer discussed these key topics with Harvey co-founder, Gabe Pereyra.
The interview was inspired by an article that Pereyra wrote about how Harvey is using agents internally and the framework around them, and then how this translates into applying an agentic approach at scale inside a law firm, or inhouse legal team. Part of this is developing a ‘world model’, which is the data infrastructure needed for the agents to operate across an entire business.
We also explore what happens when AI and agents enable the ‘intelligence’ layer of a legal business – i.e. the associates – to produce much higher volumes of work, but perhaps the ‘judgment’ layer – i.e. the partners / senior lawyers – doesn’t have the bandwidth to handle this output. This in turn connects to rethinking business systems over the long-term.
We discuss much more – but every aspect here goes to the heart of the future structure of legal businesses.
Please press Play to watch / listen inside the page, or you can also go to the AL TV Channel here.
—
There is an AI transcript of our conversation below. More about Harvey and its agents here.
—
Legal Innovators – California and Paris – June
You can now do express registration to join Legal Innovators California and Legal Innovators Europe in Paris this June.
A Legal Tech Conference For All of Europe
Legal Innovators Europe – Paris – June 24 and 25.
Express route to your ticket here.
And,
Express route to your Legal Innovators California June 10th and 11th ticket here.
Legal Innovators California, the landmark West Coast legal tech event, will take place on June 10 and 11, in the heart of the Bay Area, the home to many of the world’s leading AI businesses – and plenty of legal tech pioneers as well! More information and tickets here.
—
AI Transcript of Interview
Richard Tromans here again. Artificial Lawyer TV. Today with me is Gabe, as you’d probably recognize from Harvey, co-founder of Harvey. Gabe wrote a really interesting article a few days ago about world models and about agents and how this all connects to legal world. Gabe, thanks for being with us. Appreciate it.
Gabe Pereyra (00:30.766)
Thank you for having me.
Richard Tromans(00:33.422)
So let’s just recap with people. I’ll put a link in the next of the video so you can go back and read the original article. But Gabe, can you just give us a quick recap? So it started off with talking about Spectre, which is a program and an agent that works inside Harvey itself for your own company purposes. And then we kind of got onto the subjects of how that connects to something called a world model. And then in turn, how that connects to the legal world. You just talk us through those bits.
Gabe Pereyra (01:03.01)
Yeah. And I would think of Spectre less as an agent and more as the infrastructure you need for teams to collaborate with agents. And so maybe let me give a quick rundown of kind of the origin story, the motivation of why we built it. And then we can kind of go into, I think how it’s informing how we think about legal. what has been happening in software engineering about three years ago, four years ago, when we started Harvey, similar companies built kind of
what we built for lawyers, for engineers, kind of co-pilots. And what a lot of these companies like Cursor built was take your programming IDE and then put the models in there. But it was still local. So you were using your IDE, you had cloned your code, and you were working locally, and then you would push those changes. And as these models got better and they started getting more more autonomous, what happened is this moved increasingly from the IDE to now the command line.
which is basically just give these agents access to your terminal or your computer and have it write all the code, run all the tests, figure everything out, but still locally, and then push that to the cloud where you can merge this with the code that the rest of your team is writing. And as these models got more more autonomous, you run into a lot of problems if they’re tethered to your laptop. If you close your laptop and it’s running locally, then all those agents stop.
As your code base gets bigger, you need kind of infrastructure to run this remotely because there’s a bunch of performance reasons why you want to do this. There’s a bunch of security reasons, like you don’t want to be interacting with some of this code or data on your desktop. If you want to share that state with your other engineers, then there’s a bunch of reasons why you would want this to be remote. And Joey, who was kind of the engineer on our team that
originally built the first version of Spectre. His motivation was actually he’s a mobile engineer. And so he wanted a way from his phone to basically delegate these jobs to agents. And now you’ve seen a bunch of companies like Anthropic has a version of this. I think it’s called Delegate. And then all the coding companies are also building this. so this idea, like I’m not claiming we came up with this. think like a lot of the engineering companies are converging on these ideas. But
Gabe Pereyra (03:28.714)
What Spectre is becoming now is a way where engineers and non-engineers can delegate engineering tasks and increasingly non-engineering tasks to the system. And so you can go in Slack and you can say, Hey, Spectre, can you explain XYZ to me? Can you make this code change? Can you see if you can fix this bug? And then under the hood, you have the shared infrastructure where it has access to all of our company context or code base.
We’ve built a bunch of security on top of it and it lets our team collaborate with these agents in kind of a more centralized way rather than everyone on their desktop kind of doing their own thing and then trying to merge it at the end. And so when we talk about kind of like world model or company harness, that’s kind of the idea of putting all of your context, all of your company context into one place where it’s legible for all these models to use it.
Richard Tromans(04:23.138)
Gotcha, gotcha. And how does that then translate to the legal world? Because I mean, I think you picked up on this in your article and it’s interesting because I think people have been thinking about this too, although it doesn’t get picked up much, which is that we’re not just talking about how agents can be used more effectively and how you can build a world model with agents inside a law firm or a legal team, but that there’ll be agents inside the corporate.
or there might perhaps be agents for the law firm, not today, but perhaps at some point in the future will be responsible for, and they’ll be managing and coordinating all of these things. just talk us through that bit. So that’s the technical bit started off in engineering and it’s kind of opened down to broader areas. How do we get to the legal bit?
Gabe Pereyra (05:11.798)
Yeah, so I think there’s actually a very nice analogy between kind of software engineering workflows and legal workflows. And then there’s some differences that I think also make what we need to build for legal interesting and kind of unique to what’s being built in coding. so similarities, the way I would think about most engineering workflows is I have a code base that has all the code that I’m working on with my team and I will…
download that locally to my laptop, I will make a bunch of changes and then I will push those changes to the cloud. And if other people have made changes, then I need to go figure out, you know, which version we’re going to use. And so what.
Richard Tromans(05:49.956)
It sounds a lot like legal work.
Gabe Pereyra (05:52.166)
I was going to say the analogy there is if I’m working on a client matter, I go to I manage or net docs. I download all those documents. I do a bunch of research. I write a Word document. I need a version it and I need to merge it in with the changes that other folks have made. And so there is that very nice analogy. I think where software engineering for a bunch of reasons in terms of the scale, the type of work they have built very powerful tools for version controls.
for letting thousands of engineers all work within the same code base. And so there’s all this infrastructure that software engineers and the profession have built, which is part of the reason that I think agents are taking off so much in software engineering. And then I think there’s this very interesting analogy and how do you use a lot of that infrastructure but translate it in a way that is intuitive to lawyers. For example, in software engineering, you use Git for version control, which is
way more complicated than what you need in legal. In legal, you’re usually just versioning a doc and you always want to increment it. But there is a bunch of reasons why maybe under the hood, the agents would want to use Git and then the product would want to translate a simpler version to the user. I think the biggest difference between engineering and legal is when we work on our code base, that’s our code and there’s usually one large code base and you’re always adding to that.
Where law firms are very different is you can kind of think about them as having thousands of code bases. All of those code bases are temporary, but could be long-lived projects, right? Like an &A, a litigation, and all of that code or a decent amount of it belongs to their clients, right? And so the infrastructure you actually need to build for law firms is how do I manage all of these code bases or client matters of all of my different clients
and how do I deploy agents to all of them, but ensuring that none of the data gets mixed. And so this is a really good example of why you can’t use desktop products. And so one issue you see with these desktop products is the models will break these boundaries, right? If you ask the model, go figure out this litigation and it sees data that should be ethical walled on your desktop, like there’s a decent chance it go use that.
Gabe Pereyra (08:14.88)
And part of what we built for Spectre and what is increasingly important for these agent products is what’s called sandboxes. And so the way that you move your code or your legal workflows from your desktop into the cloud is you build a virtual machine. And so what Spectre does under the hood is when you select a code base and run a code agent on it, instead of downloading that to your desktop and giving the agent control of your desktop,
you download it to a virtual machine in the cloud that is isolated to just that task. It downloads the code base there and it operates over that, but it’s fully isolated and then produces the result. And so if you think of what you want for law firms, it’s the same thing. If I delegate an agent, I want to clone that one client matter that it’s working on. And I want to be able to show that, sorry. Do you want to go back a bit and I can?
Okay.
Richard Tromans(09:14.717)
We’ll keep it real.
Gabe Pereyra (09:16.206)
Yeah, no, my Winston is my only emergency contact, but I think it’s. Yeah, that’s all I’ll call him after it. And so backtracking. So what it does behind the hood with the sandboxes and what you would want for a client matter is we clone a specific client matter and then the agent only operates on that. And this is how you are going to be able to ensure ethical walls at scale. Because when you’re deploying.
Richard Tromans(09:19.808)
if you need to take a call.
Gabe Pereyra (09:43.894)
a lot of these agents at a law firm, what you’re going to need to be able to show is the auditability of every agent and be able to show that this agent only access data from this client matter on this run. And so that’s like a lot of the infrastructure that we’re building now, both for Spectre and like for our clients as well.
Richard Tromans(10:02.85)
Gotcha. Gotcha. If we have time to continue, if it’s okay to continue. mean, the and then you get further into the article and you talk about and it’s something that, you know, Sequoia and a couple other people have kind of alluded to, which is this idea of a split between intelligence and judgment. Yeah. Which is I find really interesting because, you know, people think of intelligence as in like,
very, very clever, but what they really mean by that is it’s complex work at scale. So legal work, even the simplest legal work is not easy work by any means. You need to have a high level of intuition and understanding to grasp it. But that can be codified, that can be crystallized, you can build in feedback loops to cover it. But then eventually it gets way too complicated, just too much judgment, too much complexity.
the feedback loops just become infinite, know? Then you get into real human judgment. And I guess, and then you use this analogy, which was about the partners that you get, was it limitless, limitless intelligence? And the partners have a bottleneck. And I thought also, I from artificial lawyers perspective, sort of economic perspective, that’s interesting, because I guess you can see that two ways, which is that the partners,
have now, or senior lawyers now, have this ability to do huge quantities of work, but they’re limited by their own normal human ability to be on top of making decisions around all this information that’s coming at them. It’s like, wow, I received 200 contracts today. OK, wonderful, but there’s no way I’m going to review more than five of them. No way. So it’s not going to help. But then there’s a second, which is like the bit that I’m always on, which is the economics bit in the
traditionally the partners have controlled access to the means of production. So traditionally and still is in almost all cases until very recently, it’s the associate body or the junior lawyer body or whatever it is. And so you’ve got it. it’s, particularly like that phrase because it had like a double meaning. And I guess the future to some degree will be then, you know,
Richard Tromans(12:23.34)
do the partners say, well, yes, it’s true. We do control the means of production, but half of that means of production, perhaps, will now be agents. And the other half will be junior humans who will maintain and operate, probably working alongside the agents. I just wonder how you saw that kind of panning out.
Gabe Pereyra (12:43.496)
Yeah, it’s a super complicated topic and I think everyone is figuring this out. I think one way to think about it and to your point of kind of part of the point that I think Jack and Roloff are making in the hierarchy to intelligence piece is and Winston and I have gone through this as you scale a company, a lot of what you start thinking about shifts from
What is the best product? What is the right idea or strategy to how do I build an organization that can coordinate a lot of people that the result is the end product and the strategy that you envision. But as we’ve scaled, you know, now we’re 800 people, so much job is thinking about what is the shape of the organization, the mechanisms, the operating system that lets all of these people work together to create a good outcome for our clients. Right.
And that way of thinking, like I think that is a good way to also articulate what these law firms do so well. Like I think a lot of people think about what lawyers, engineers, everyone does as writing the code or writing a contract. But as you get more and more senior, the work is that of how do you build the organization and what name partners, the senior partners, the partners that lead practice areas, what they actually do really well is how do I build a business? Right?
I run my &A practice in a way that I have a bunch of very talented partners that work for me and even more associates that can all work together on many deals to provide good outcomes at scale for
Richard Tromans(14:22.894)
Yeah, and the knowledge management, various people who feed into that, I mean, how is that accessible? And then you’ve got your business development team and your client, you know, partners who are involved in running projects and the partners who involved in maintaining client contacts and the training. It’s, you say, it’s all about the coordination.
Gabe Pereyra (14:25.942)
Yeah, and then there’s
Gabe Pereyra (14:45.742)
And exactly to your point, it’s not just the partners, it’s the CIOs and the knowledge managers and all the functions you need. And we have that same challenge at Harvey, right? It’s not just engineering, it’s product design, recruiting, sales, enablement support. And a lot of the work is how do you get all those pieces to fit together? And I think the really big challenge that is going to happen now is the old way of fitting those pieces together is going to change dramatically.
because these agents are going to change a lot of the variables that define how you create that hierarchy. Like law firm structure has been stable for a while because as much as, you know, some people complain about bill of bar and all these things, I actually think this is the optimal solution for the past two decades. Like this was the right
Richard Tromans(15:33.966)
If you’re using humans and humans are your engine, then yeah, the time it takes to do something. they’re not using petrol, they’re not using electrons. Yeah.
Gabe Pereyra (15:45.742)
And I think now what we’re seeing in our engineering org is same way there was kind of a roughly standard way to build a law firm. There was a roughly standard way to build an engineering work, right? Once you scale past a hundred engineers, if you’re trying to scale to a thousand engineers, there is well-defined, you need a platform team, you need a front end team, a backend team, right? There’s well-defined ways to start breaking up this work and these abstractions. And I think
Where it’s going to be interesting is a lot of the way that you break this up with humans actually translates to agents. And so there will be a lot of similarities. And so I think one way to look at it is if you think about we can give every engineer 10 employees, we need to start thinking about our infrastructure of that for a much larger company. And so there are similarities. But then there’s also differences, right? These models do work much faster.
but they make mistakes in different ways. And so the judgment and the way you coordinate them is different. And so it’s gonna be this really interesting problem of what is the new shape of organizations or I think pretty much every industry. And then where I think it will get more challenging is the technology will evolve. Like right now we’re personifying or anthropomorphizing the agents because that’s easy, but eventually they won’t look like that.
And will look like these hybrid human agent software organizations. And then where I think it gets really interesting for us and especially for law firms is then you also need to think about all of the training and the business model and everything that potentially will change on top of these models changing how your organizations function.
Richard Tromans(17:27.566)
Yeah, yeah. It’s, it was for so many different routes out of that. It’s very interesting. I mean, the point of, mean, it’s like at the moment, software is being taught to pretend to be human, but eventually once it’s encapsulated all the things that it’s expected to do in the human domain, it will kind of retreat back to being software. I don’t have to pretend anymore that I’m an associate. I look, I’m just, I’m just, am what I am.
Gabe Pereyra (17:51.604)
I mean, that’s such a great point because I think there’s kind of this meme of software is dead and all agents are going to go and do everything. And I think it feels like a lot of that comes from folks that haven’t run large organizations where I think people think that, once you get the agents, like that’s going to solve all the problems. And if you’ve ever managed a large group of people, it’s like, once you get all the people or all the agents, that’s when a lot of the problems begin.
And the thing that is so nice about software and especially vertical software is it solves your problem in the most direct way. And I think what’s going to happen in the next 10 years is there is going to be, I think this similar transition that happened with computers and the internet where you need to rethink a lot of these things and the shape is going to look very different. And I think enterprise SaaS is not going to look like enterprise SaaS today, but it’s also not going to look like every company has a million agents just running wild.
And a lot of what we’re figuring out within our organization is these agents are super powerful for engineering and all these other functions. It’s very clear that they are a net positive in terms of productivity, but then there’s all these challenges of security and coordination and they still make mistakes and those mistakes can be unintuitive. And I think law firms, in-house teams, every industry is going to go through this. And I think the interesting thing will be there will be a lot that is shared.
across every industry, there’ll be a lot of shared infrastructure. And then for domains, especially regulated domains, there will be a lot that is unique. And I think where legal is potentially the most interesting, especially for in-house and law firms, is I think they will set the tone for how you govern these systems within companies, how you regulate them. Like if you’re at a bank or a private equity firm or any of these regulated industries, how do you
show regulators, show your clients that you are using this in a responsible way. And I think that to me is a place where the legal industry can really lead on how do you think about that. And I think that’s an interesting problem.
Richard Tromans(19:56.418)
Yeah, yeah, no, totally. mean, the whole way the lawyers will begin to police agents that are not operating inside their legal domain, that’s going to be really fascinating. We’ll have to do another podcast. And the whole hierarchy thing, again, I think we could do a whole thing on that as well. But just to wrap up, just very simply because we kind of zip through a whole bunch of things, just so that people understand, when we say a world model, and if, let’s say, a legal or a law firm world model, what would that be?
what are the components in this world model?
Gabe Pereyra (20:30.662)
I would think of for the law firms, there is kind of a hierarchy as we think of building these systems. There is at the client matter level where this, and we’ve talked about this for years now and what we’ve been building towards of the first step you want is I’m an associate, I’m a partner, I’m working on the client matter and I can go in there and delegate anything to an agent that has all of the context that I need for the client matter. And even that is complex where
there is just the client matter and you could think of that as kind of the data room or the discovery corpus plus the document management system. But then you also need to think about case law and other data sources and all the precedent within the firm. So you start thinking about, okay, I want all of the client matters. I want to understand how they’re connected. I want to understand all of my clients in the context I have them on them. And then you also want to think about more of the firm
operations, because I think a lot of what we’re starting to think about internally is not just how do you get all the context, but how do you start encoding all the processes so these agents can work with them and change them? And like a lot of what determines your shipping velocity at scale in an engineering org is not just you have talented engineers and you have kind of your code base arranged in a good way. It’s what is your development process?
And so I think increasingly for law firms, the world model will be how do you encode the process for every practice area into these systems, so the agents and the humans can follow them? How do you encode the way that you’re billing, the way that you’re showing these bills to your clients so you can start optimizing these systems end to end? But I would think of the goal for law firms is how do we encode our firm into a way that any human or any agent can operate on it with
the right permissions and all of the context and kind of business context that they need to do their job. And that’s how I would describe what we’re building with Spectre first for engineering and then our entire product organization and then eventually the company. And I think that’s informing how we do that for law firms and how we do that for in-house teams.
Richard Tromans(22:44.206)
Fantastic, amazing. So many different points there, but thank you very much. Thanks, Gabe. Really appreciate it.
Discover more from Artificial Lawyer
Subscribe to get the latest posts sent to your email.