Artificial Lawyer’s great Guest Bloggers at ILTACon 2017 move into Day Three of the legal tech epic in Las Vegas.
Guest Bloggers, Alicia Ryan, who is a Knowledge Management Attorney at US law firm Fenwick & West; and Matthew Golab, who is legal Informatics Manager at Australian law firm Gilbert + Tobin, (pictured above together) report on what happened and share some excellent insights into certain legal AI tools.
Signals in the AI Noise
By Alicia Ryan
Greetings, Artificial Lawyers! Conferencing is exhausting. I haven’t been to one in a while, and never one quite so large as this. I’d forgotten what a constant furor it can be.
And, judging by how much sharper the elbows were around the coffee urns today, it’s not just me feeling the strain.
Fun + Games
On the plus side, I finally made it outside the hotel. I hadn’t been out of doors since the opening reception on Sunday night, so I took advantage of a brief lull in the programming to sit poolside and feel the sun on my face.
Also, I finally managed to meet up with my co-guest blogger, Matt, and he’s a super nice guy with some great insights on various AI tools.
To cap off the day, Kira Systems invited me to their user reception at the swanky Skyfall Lounge far atop of the Delano Hotel. After recovering from the slightly harrowing elevator ride (up the outside of the building), I really enjoyed getting to talk more with the Kira team and meet some of my counterparts at other law firms who are making use of the Kira platform.
Which leads me to the substantive topic of this post: Kira Systems.
I was on the Kira ‘Solutions Spotlight’ panel on Monday to talk about implementation tips. I had not intended to write more about them – nor even to mention them by name – but I reconsidered after experiencing first-hand the value of hearing other users talk about their own experiences.
I’ve heard the term ‘AI’ more times than I can count over the last three days. But which AI products will deliver tangible benefits to users today?
Obviously, I can’t speak to all of them. Here is what I can speak to this: I am personally serving as the project lead for the Kira implementation at my firm. We are about nine months into it. All the attorneys in our initial group of users are using the tool in practice and are finding value in it. Value that outweighs the learning curve, value that speeds up the work, value that can be passed on to clients.
What Does Kira Do?
Kira is a multi-featured platform. At it’s simplest, it can be used to easily compare several documents or compare a set of documents against a form. (So you can quickly verify, for instance, if a target reps that all its invention assignment agreements are on a single template.)
Kira’s key functionality is its ability to extract provisions from documents. Out of the box, Kira has over 200 provisions that come standard. These include all the common provisions an attorney would look for in the M&A due diligence context.
Finally, Kira has a feature called ‘Quick Study’ that enables you to teach it new provisions. In my own work, I am using this feature to conduct surveys of different types of agreements. I drag my document set into Kira, do the review I would otherwise do anyway, highlight the provisions I would otherwise highlight, and then I do one additional final step: I click the “Learn” button. This tells Kira I want it to review all my tagging, apply its algorithmic secret sauce, and learn from what I have done.
It does this remarkably well. This feature, in particular, continues to impress me. For documents on similar forms, Kira starts getting the review right after ingesting 20-30 examples.
Almost as an aside, Kira has a nice workflow interface. Any number of people can indicate they have reviewed a document. Each document can be assigned out, and progress (number of documents reviewed vs. unreviewed) is shown in chart form on the project dashboard.
Day Three – Neota Logic and More
By Matt Golab
I attended two sessions relating to AI today (Neota Logic and Liberty Mutual) and then part 3 of the AI series – From Theory to Practice.
Beyond the Hype: Putting AI to Work at Liberty Mutual by Ryan McLeod (Neota Logic) and Jeff Marple (Director of Innovation at Liberty Mutual Insurance Company)
This was a vendor sponsored session which meant that the first 15 mins were Ryan taking us through the fascinating world of what Neota Logic is and can do.
A quick recap is that Neota is a form of AI that used to be called an expert system, and is a logic/reasoning/inferencing AI system.
A key factor about expert systems is that they are fully auditable and you can easily see how decisions are made – which is a key differentiator to the other kinds of NLP and Neural Net forms of AI where in some of the systems there is a level of opacity, however you don’t necessarily understand how the system arrived at a decision.
Most Neota systems are web apps where you are filling out a form/questionnaire.
Jeff mentioned that using AI enables you to do higher quality work, and that Neota is a great first step into using AI. He also made the comment that the biggest challenge in using AI is getting your data ready and that this is 90% of the work. Liberty Mutual was in dialogue with Neota for about 3 years before they had a pilot and then they were able to run their pilot and develop a tool within about 2-3 weeks with a team of technologists and some lawyers (who contributed bursts of time).
Their scope was NDAs. Liberty had about 25-30 ‘standard’ NDAs that they reduced down to 2-3 standard forms and then automated in Neota. The process to get a signed NDA used to take at least 1 working day, and now it is reduced to about 15 minutes and is quite often completed over the phone.
Jeff mentioned that Neota is a business facing development environment where you don’t need developers or to write code and that he has a team made up of business analysts, 1 developer and 4 lawyers working 1-2 hrs a week on projects.
They then ran through a live demo of the NDA process, and it is quite an efficient and effective way of getting standard NDAs executed, and also queries the external party to ensure that they have the authorisation to sign and execute the NDA. A key consideration with this system is that it only works with standard clauses and doesn’t currently have the facility to handle clause negotiation.
There was then some discussion about future legal roles as legal technician (Jeff) or legal engineer (Ryan).
Ryan mentioned that Neota should be wholly web-based in spring 2018 – the product is currently run on desktops in a Java environment.
The next session was Part 3 of the AI series – From Theory to Practice. This panel was made up of:
- Anna Moca (White & Case)
- Amy Monaghan (Perkins Coie LLP)
- Jonathan Talbot (DLA Piper LLP)
- Justin Tsisin (Google)
- Andrew Arruda (Ross) was the moderator.
Jonathan opened and mentioned that he comes from a data science background and that his focus at DLA is process improvement and automation.
DLA formed a team to focus on optimising M&A/Due diligence and this was made up of the head of M&A and three associates. They initially focused on process mapping and checklists, which achieved a 10-15% improvement, and then they selected Kira for M&A.
Julian from Google then mentioned that clients don’t care whether you are using AI, what they want to know is that you are being efficient and effectively using technology to help them.
Amy then mentioned that they are piloting Kira as well as piloting Neota.
In terms of the pros and cons of using AI, Amy said:
Cons – constantly educating and having to correct misconceptions over what AI can do and that it isn’t a magic bullet.
Pros – you can get creative in using AI tools and that these can solve 90% of a problem. Amy also mentioned that if you create a good relationship with a vendor that they can then help you. [this is an area that we emphasise at Gilbert +Tobin in that we are working with our vendors rather than only using their products].
With building business cases for AI, it is better to have pilots that have a narrow scope and that everyone can clearly see the benefits as otherwise you are bound for failure and the pilot may pre-emptively fail. It is also easy to underestimate the power that you have with AI and to take it for granted.
At DLA they mentioned that now that they have Kira, that if you complete a due diligence exercise and the client then changes scope, you are able to respond in hours rather than days.
At Google they have the philosophy of being an AI first company and rather than focussing on tools they focus on business problems. They still have concerns over accuracy levels of ML.
They had a project where they built a TensorFlow powered ML/AI system to focus on patents. They found that the ML was close to human lawyers on precision, however a lot higher than humans on recall [precision and recall are statistical terms that are the holy grail of search, you want to aim for 100% precision and recall – ie that the documents that you return are highly accurate].
Also mentioned that implementing an AI system is the same as implementing any technology, and that we need to calm down the hype around AI.
In addition: you should carefully think through your problem and process and translate business concepts into ML concepts before talking to a vendor as you will then get a higher quality result.
Justin then mentioned LexPredict and that they have open sourced ContraxSuite.
At DLA they have the philosophy of buying what you can and then building what you must.
At White & Case they develop scorecards for lawyers to evaluate AI systems, as the perspective of the end user is quite important.
Some challenges identified:
You need clean data.
Google mentioned that in their patent tool, they realised they needed clean data, so they arranged for a law firm to categorise a set of patents. [An interesting way in which a law firm is helping a client where the law firm is being used to train the system, and then presumably the law firm isn’t needed for these patent tasks].
Also discussed was that you have to have a skilled team available to support a pilot as it will fail if you don’t have adequate support.
This was the third and final part to an excellent series of AI sessions.
In the afternoon, I visited Ayfie and got some fascinating insights into their AI engine. I also visited Doxly (not AI but related in that it is a system that is focussed on supporting deals through checklists and completion). While visiting Doxly, I had the unique opportunity of being in a triumvirate as the CEO of Doxly (Haley Altman) was there as well as Michael Mills (Neota Logic) and so we were able to chat about the ways in which Doxly could [in theory] be supercharged by Neota and its ways of turning information for decisions/questionnaires into documents.
I also visited the Luminance booth and chatted about the latest and greatest.
At lunch I had the distinct pleasure of sitting at the same table as Judge Peck (of Da Silva Moore predictive coding fame) and Craig Ball (computer forensics and ediscovery all-round guru).
Another satisfying and fulfilling day. 7.5km travelled today – although I did have 1-2 too many wines afterwards!
2 Trackbacks / Pingbacks
Comments are closed.