Not All Automation Tools Are Created Equal: Top-Down vs. Bottom-Up

By Dan Broderick, legal tech company co-founder.

The legal technology space is a crowded place these days; it’s a full-time job keeping track of the hundreds of emerging legal technologies and the problems they aim to solve.

Contract management has evolved drastically in just the past two years, with several players in both the pre- and post-execution spaces. Certainly, COVID-19 has accelerated the rate at which companies and firms are evaluating ways in which they can streamline workflows and save money. There is not one company or law firm out there that isn’t concerned about their bottom line.

Deciding which technologies that align with achieving those goals can take significant time and effort. What problem are you trying to solve? What should an organisation know before choosing a solution? What makes one product different from another? How much hands-on time is required to implement and train the product?

As you embark on any buying decision, it helps to distinguish between two software design approaches – top-down and bottom-up. Most legal technology providers create tools that rely on a single ‘industry standard’ set of rules (top down), rather than learning from the client’s own data (bottom-up). While a standard set of rules might be easier to implement in the short term, a bottom-up learning approach results in a more viable, adaptable, efficient, and effective output.

Differences Between Machine Learning Methods

With a top-down approach there is little, if any, customisation of the product. It is a one-size-fits-most scenario. A top-down tool cannot infer client specific rules due to its reliance on a broad set of generalised data rather than client data. As a result, top-down tools work on a shallower class of problem than bottom-up tools. For example, indicating a distance from an industry standard document or adding static boilerplate text when a contract section is missing. This approach fails to replicate the contract markup process and how individual organisations treat risk.

Software built with a bottom-up approach, on the other hand, combs through a client’s historical contract data to build a customised product that constantly learns and adapts to the client’s specific needs. Bottom-up learning refers to learning implicit knowledge first and then learning explicit knowledge on that basis.

The data itself identifies the problem and the software becomes smarter over time. The machine learning algorithm infers rules from the data without ever committing them to code. This provides for customisation based on the substantial historical context and an individual company’s playbook, with far less hands-on training time from attorneys.

Most Contracts Require Company-Specific Language and Other Nuances

In contract review, there are some scenarios where a top-down approach may be useful, such as finding the term of agreement or the governing law clause in each contract. These provisions, which are extensively used and infrequently changed, are relatively simple and easy to create prescriptive rules around.

But many other aspects of contracts are not as cookie cutter. It is more difficult to create prescriptive rules for items requiring nuanced editing. For example, inserting a company-specific sentence about compliance with a specific policy. The top-down approach doesn’t work in this case because those clauses are unique to each organisation. A one-size-fits-all approach will not generate the work product required by the law department or law firm.

Additionally, a top-down approach, once built, is limited to the problems it was built to solve. Adding new use cases means finding a new, large, generic dataset that demonstrates the problem and building new rulesets around that data. This requires significant time and resources and is akin to starting application development anew for each new problem.   

Bottom-Up Learning Continues to Evolve

Bottom-up learning tools accommodate more customisation and better results because they constantly learn from the source material and changing the rules only requires changes to the data. There is little up-front human involvement because the algorithm teaches itself.

However, attorneys still need to oversee the learning process.

For a bottom-up model to work properly, it must also have access to a quality dataset. This requires good gatekeepers to maintain and manage the data being analysed. It also requires human controls to enable users to tweak how the system learns and what it discards. Without this step, the bad edits will get into the model and could cascade.

The biggest advantage of a bottom-up learning approach is its potential for evolving rules over time without human intervention. Ultimately, this means the system can grow with a company and increase its ROI without significant work on their part. Simply fixing a document, changing a term or a governing law clause informs the system that you have a new rule. Rules are customised based on the dataset of each company providing results tailored to situation, industry, or company preference.

With a top-down approach there is limited customisation. In effect, it’s like repurposing the technology to solve an adjacent problem. Customisation is possible, but only within a predefined set of parameters. It is impossible to encode a single set of rules that would work for every situation.

Technology with a strong return on investment will likely be based on a bottom-up approach. While an off-the-shelf product may seem appealing, the reality is every company is unique and requires the customisation that only a bottom-up approach can provide.

---

About the Author:

Dan Broderick is the CEO and co-founder of BlackBoiler, a contract review company that uses proprietary deep learning and natural language processing to speed up the process of high-volume contract editing for Fortune 500 companies and AmLaw 100 law firms.