GenAI Benchmarking Project: Virtual Workshop + Next Steps

After a successful meeting over the summer, the project to develop thinking around legal genAI accuracy is moving to the next stage, with a virtual meeting planned, along with appointments to help drive things forward.

To explain more, here is an update from John Craske, head of innovation at CMS, and who with LITIG, has been organising things along with the support of Artificial Lawyer and Legal IT Insider.

Update from John Craske, CMS

‘To help give the initiative some momentum, we have recently appointed Jo Owen (who was CIO at Cripps until recently) to help us with the organisation, project management and getting stuff done. The project team has also been joined by David Wood (David is also on the Litig Board, as well as having a leadership role at Simmons Wavelength) to help drive things forward, and Gideon Green who is an associate at CMS (some of you might recognise Gideon from the first workshop).

The workshop

We were fortunate to have 40 participants, including representatives from law firms, legal tech providers, journalists, conference organisers and others, who came together to discuss how we can make this vision a reality.  It was a fantastic starting point, and we owe a big thank you to everyone involved, including those who helped organise and facilitate the event and a name check for Richard Tromans from Artificial Lawyer who helped set the scene so well.

What’s next?

To build on this momentum we will be hosting a virtual workshop session at 4pm (UK time) on the 24 October 2024.

This virtual workshop is open to all (subject to a sensible limit that still allows for a workshop style discussion) but is really aimed at those of you who couldn’t attend the in-person workshop.  We are likely to only take one participant from each organisation, so if there are a few of you on the mailing list, please agree who is going to come along.  Once we know how many of you are interested, we will either invite everyone or just a carefully selected group of the people interested!  Please don’t email me … register using the link below.

In this second workshop we will test the outputs from the session in August and aim to build a set of key considerations for the next stage, which will be a working group to help create our benchmark(s). Of course, even if you don’t come along, we will also share the outputs from the virtual session with the whole group.  Based on engagement and feedback so far, we have seen considerable interest in this initiative, it is quite likely that we also set up special interest sub-groups who help direct and inform the working group.

The form will be open until 5pm UK time Friday 18 October. 

[ NOTE: Places will be limited – and those who signed up early on to the project are likely to be considered first. ]

In parallel, the project team are engaging with a number of organisations and groups across the legal industry to learn and join up where we can, so we don’t duplicate any effort. If you are working on or are aware of any similar initiatives, we would love to hear about it.

If you would prefer to simply stay involved and follow the progress, you don’t need to do anything: you are on the list. However, if you do not wish to receive any further information about this initiative, just let us know.

We are looking forward to continuing this exciting journey with you!’