Why General AI Alone Is Not Enough for Legal Work

By Senne Mennes, LawVu.

Few terms in legal technology have become as overused as ‘AI-powered.’ Not to mention over-hyped.

Today, virtually every legal tech vendor claims that they have AI solutions capable of handling complex legal work, while general-purpose AI providers themselves also tout legal capabilities. This creates confusion and a crowded market where buyers struggle to distinguish between a fit-for-purpose AI solution and a chatbot dressed up in legal clothing.

Underpinning this confusion is a general lack of knowledge about what AI can do for counsel, and what it can’t. We thought it was time to change that.

General-purpose AI has unquestionably changed the legal landscape by creating serious efficiency gains for both legal teams and law firms. AI can summarize and analyze repositories of contracts, redline documents, and accelerate research in ways that would have seemed impossible just a few years ago.

There’s no denying that general AI is incredibly capable, but it’s important to recognize that capability alone is not enough.

General-purpose AI vs purpose-built tools

General AI models are trained to be broadly useful across an enormous range of tasks; think of it as a jack-of-all-trades and master of none. They are trained on vast collections of publicly available information and are optimized to generate plausible, contextually relevant responses.

A lawyer reviewing a supplier contract might ask an AI system to identify unusual provisions. While a general model can compare the document against patterns it has observed in training data, it cannot inherently understand how that supplier relationship fits within the company’s procurement strategy, if certain deviations can be approved, or which contractual risks are commercially acceptable in that specific context. In other words: AI is limited in its capacity to make judgment calls unless guidance is provided – not unlike an extremely intelligent, somewhat overzealous intern.

The limitations of general AI

In order to get optimal performance out of the incredible capabilities of General AI, it’s important to be aware of its limitations. Some chief limitations for contract-related work are:

• Inconsistency: try asking the same question to a general AI tool twice and you’ll see that it always provides a slightly different answer.
• Lack of context: general AI tools can only fall back on the patterns they know, and they don’t know yours unless you teach it.
• Lost-in-the-middle problem: AI can only ingest a finite amount of text in one sitting, much like our short-term memory has its limits. Even in situations where those limits are respected, AI may still gloss over or ‘forget’ certain text that has been submitted to it. This is a strangely human quality where it pays more attention at the start and end of a text, but its attention flags in the middle.
• Layout-awareness: drafting, redlining, and reviewing all still take place within Microsoft Word for most lawyers. General AI tools rarely allow you to make changes in a document while respecting its existing layout, terminology, and formatting, meaning you’re still spending a lot of time doing menial things like fixing cross-references in AI-generated clauses.

Consider these limitations in a few simple use cases:

• Drafting a contract from scratch: the Sales team needs an NDA and we don’t have a good template. You could boot up ChatGPT or a similar tool and receive an NDA with a single prompt. The inconsistency problem means you need to review this from scratch every single time you go through this exercise, though – a great way to burn through the time you saved by not using a standardized template. 
• Drafting a clause from scratch: you could ask AI to draft a payments and invoicing clause for a general commercial contract and it will make a few judgment calls on what needs to be in there. You likely won’t fully agree with all of them. So you try again. You clarify that the clause must contain a 30 day payment term, that one party is responsible for all taxes, a dispute resolution mechanism for complaints, a late payment interest, etc. By the time you finish writing out what kind of clause you need, you’ve effectively written it yourself.  
• Reviewing and redlining a contract: asking AI to review a contract will again yield different results every time. Without further guidance, AI is lost on what your risk appetite is, what your industry standards are, what your hard stops are, etc.
• Reviewing a lengthy contract: even if AI can ingest a full contract for review, it may gloss over details in the middle. This can be mitigated by orchestrating how the AI absorbs the text, which is something general AI tools do with varying degrees of success at best.  
• Redlining a third-party clause: general AI tools can suggest language to help tweak a third-party clause. Rarely do they allow you to then cleanly insert this language in the style of the document since that is a hard problem to solve.   

Understanding the importance of organizational knowledge

Despite limitations, general AI tools can serve a legal department exceptionally well. The caveat is that they won’t deliver the results you want on their own; and that they require lawyers to be cognizant of these limitations and know how to navigate them. Both problems can be solved by feeding general AI with organizational knowledge.

Organizational knowledge is the intangible information regarding your company or clients’ risk appetite, standard and fallback clauses, base template material, industry-specific drafting customs, negotiation history, and more.

General-purpose AI does not automatically know your organizational knowledge as it is generally stored across templates, precedents, playbooks, clause banks, document management systems, and emails.  

Organizational knowledge is not something an LLM inherently understands. Yet in contracts-related work, it is often the factor that distinguishes a generic answer from a useful one. That’s why it’s so important for legal teams in the market for AI tooling to assess how organizational knowledge can be leveraged. Without it, AI can at best provide the occasional useful suggestion. 

One such assessment is surprisingly simple: can the system operate according to your organization’s standards, or is it merely generating answers in a way that ChatGPT could also do for you?

Purpose-built AI tools can draw from organizational knowledge in multiple parts of the drafting and reviewing process. A general-purpose tool cannot.

Questions to ask vendors when assessing an AI solution

Most general AI chatbots struggle to satisfy four basic requirements by default. Not because they are poorly designed, but because they were never intended to function as complete legal systems.

When assessing vendors, ask these four questions:

1. Can the tool leverage your organization’s preferred drafting style and positions? Or does it only rely on a default style?
2. Does it clearly identify uncertainty and knowledge gaps, or does it confidently generate answers regardless of its ability to do so?
3. How is the AI orchestrated to ensure mitigation of the inconsistency and lost-in-the-middle problems?
4. Does it improve through interaction with your team’s knowledge, documents, and processes, or does every new task effectively begin from scratch?

Purpose-built legal AI tools should be able to provide a satisfactory answer to all the above. Why? Because they aren’t designed to solve every possible task; they focus on specific workflows and embed legal context directly into the process.

Precision tools close the gap

While general AI is excellent at drafting, summarizing, and helping lawyers ideate solutions to complex legal issues, this isn’t the only tool they need. In that sense, you can compare general AI tools to a Swiss army knife, great at doing lots of things, but hardly useful when you need to drive a nail into the wall.

Rather than look at the technology and try to find a use case for it, lawyers are much better off looking at the use case and finding the technology that helps solve it best. Generative AI tools are often the answer, but fall short in areas like document comparison, proofreading, clause assembly, formatting, and reference checking.

The strongest purpose-built AI tools, such as LawVu Draft, take a hybrid approach, combining general AI with precision tools, rather than relying on prompts alone. The AI provides speed and language generation, while precision tools ensure auditable outputs that are consistent and accurate.

While general-purpose models are a powerful foundation layer for legal technology, they remain only one component of a complete solution.

The most important question for legal teams evaluating AI is not, ‘Which model does it use?’ It is, ‘What is the model paired with?’

General AI alone will continue to produce impressive demonstrations. General AI combined with purpose-built legal infrastructure is what ultimately changes how legal work gets done.

—

About the author:

Senne started his professional career as a lawyer at an international law firm. After a few years in private practice, he co-founded ClauseBase where he was responsible for helping legal teams improve their drafting and reviewing work and experience value from the software. He continues that same responsibility as Customer Experience Lead for LawVu Draft after ClauseBase was acquired by LawVu in 2025

—

[ This is a sponsored thought leadership article by LawVu for Artificial Lawyer. ]