Pew: How Stanford + Suffolk Law Schools Are Improving NLP for A2J

When looking for answers to legal questions, people increasingly start their searches online. But what they find isn’t always very useful—prompting the law schools at Stanford University and Suffolk University to team up to harness artificial intelligence (AI) to help people identify their specific legal issues.

Historically, machines have struggled to understand context in human speech. For example, if someone says, “I’m getting kicked out of my house,” most people understand that the person is not being physically kicked but is rather being removed from his or her home—or, to use the legal term, evicted. But machines typically can’t understand “kicked out of my house” as “evicted” without being trained through a large number of similar questions.

The Stanford and Suffolk teams, with funding and support from The Pew Charitable Trusts, have collected thousands of online questions about possible legal issues to start developing a data set that can serve to train a natural language processor (NLP)—a subset of AI focused on understanding context in speech. An NLP could recognize that people who seek information online about getting “kicked out” of their rental property, without using the legal term “eviction,” need insight into eviction law.

NLPs work by examining data sets of questions and looking for patterns in words and phrases that match up to a legal term. For example, if hundreds of questions in a database used the phrase “I’m being kicked out” and those questions were all identified by experts as dealing with the legal issue of eviction, then the NLPs would learn that if a person says “I’m getting kicked out,” there is a good chance he or she is being evicted.

One challenge in the creation of an NLP is getting enough questions in a data set, tagged correctly by humans, so that the NLP can start recognizing patterns. So teams at the two law schools have created an online game to increase the data available to help build NLPs; players match questions to a legal issue. They call the game Learned Hands, a reference to the famed American jurist Learned Hand (1872-1961).

The game displays questions asked online about a potential legal issue. The players then read through each question and say whether it references a specific area of law. Once a statistical proportion of players agrees on a legal term for a particular question, that question and its legal term go into the data set—at which point an NLP starts to analyze additional data and spot patterns

To help realize the vision of a good legal NLP, the Learned Hands developers need as many people as possible to play the game. While a legal background can be useful in answering the game’s questions, players from a variety of backgrounds can help make the needed connections between common phrasing and legal issues.

Those interested in learning more about this effort can read this blog by David Colarusso, director of Suffolk University Law School’s Legal Innovation and Technology Lab. And because NLPs are only as good as the data they are supplied, organizations with their own data sets of questions about potential legal issues can share their data with the Stanford and Suffolk teams.

Guest Post by Erika Rickard, a senior officer, and Lester Bird, a principal associate, with The Pew Charitable Trusts’ civil legal system modernisation initiative.