Non-disclosure agreements (NDAs) are often seen as the most standard of all legal documents, the perfect test case to prove the benefits of natural language processing (NLP) for review purposes in the legal world. But, the truth is not so simple.
The reality is that NDAs can be highly complex documents and some large corporates can keep dozens of different NDA templates on hand in readiness for further changes and adaption to each individual use case.
As one senior inhouse lawyer told Artificial Lawyer recently, despite appearances when it comes to NDAs the permutations are almost infinite, at least they are at their company.
Yet some legal technologists interested in standardisation and using NLP to derive efficiency and find insights in documents remain focused on NDAs. This is because, despite all that variation, NDAs are used so often in business and share so many common principles it is still one of the best use cases for NLP-driven review.
One such innovator trying their hand is Holland-based Arnoud Engelfriet, who has created NDA Lynn, with his colleague Steven Ras.
Both are partners at ICTRecht, a legal advisory firm specialised in IT law and also founders of JuriBlox, a legal tech platform.
So, first of all, given that NDAs are not quite as uniform as we all might hope, how has NDA Lynn performed when it comes to reviewing such documents?
The team reports that accuracy is 87.21%, meaning in 87.21% of cases it identifies a clause correctly. The precision (the number of true positives over the total number of positive predictions) is 88.24%, and its recall (the number of true positives over the number of positive instances) is 82.85%.
And if you want to understand what ‘true positives over the number of positive instances’ means then check this out on Wikipedia, which might help.
In any case, it means it’s accurate compared to most lawyers, especially tired junior associates on a Friday evening, hunched over their desks and dreaming of going home. Moreover, the main issue with NDA review is not to find what you would normally expect, but to find the bits that are missing, or seem strangely worded, or contain what look like unnecessary clauses and demands.
But, let’s hear some more first from its creators.
‘We launched in 2017 and have now had a chance to review over 900 different NDAs,’ says Engelfriet.
This has allowed the team to adapt their models and learn from the variations. In short, to become more accurate.
‘The system sees what is unusual, it will tell you if it’s standard or not, and it also gives you a legal definition of the clauses that appear in the NDA so you can understand [as a lay person] what they mean,’ he adds.
This can also help reviewers decide if an NDA meets their company play book.
Engelfriet says that what you do next may vary, but one clear choice is to send the document back with suggested changes based on the issues Lynn has flagged up.
He notes that although the system could in theory miscatergorise something that was OK as being not OK, that it has never yet given approval for a clause that was clearly unusual and outside the norms already developed.
Engelfriet explains that the system at present is free, but an enterprise version is being developed. Also, for €99 a human lawyer, (AKA Engelfriet himself), will do a review of the NDA and help polish off any remaining problems a client of Lynn may have.
The only condition for using Lynn is that you agree to let the system use the data to learn. Engelfriet says 90% of people agree to this. If not then it costs €45.
The enterprise version is just now launching and around 2,000 people have used Lynn so far.
All in all a success and perhaps proof that despite the lack of perfect NDAs and 100% perfect NLP accuracy, legal AI tools can be developed that add value. Good luck to the team.