Shoosmiths’ Pavlovian AI Experiment Succeeds, What Now?

You get what you reward – and Shoosmiths’ Pavlovian experiment to reward staff for using Microsoft Copilot has seen them hit one million prompts, triggering the addition of £1m to the firmwide bonus pot. But, what has been achieved?

First some background. In April the national firm, which has about 1,000 lawyers, announced it would dish out an extra £1m in bonuses if the firm hit a certain level of prompts. It was, as you’d expect, announced with plenty of fanfare and statements about the firm’s ‘commitment to continuous improvement [that] reinforces its position at the forefront of legal innovation’.

They have now reached that one million prompts point already, and thus everyone gets a nice extra bonus. Plus, they will do the same next year.

So, aside from proving that if you give people money for doing stuff then they will do it – regardless of what good it does – what has Shoosmiths shown us here?

They said that the use of Copilot has ‘boosted efficiency and enhanced client delivery’ – which is positive. The bigger question is: by what degree?

The firm notes that this genAI tool, which cost billions of dollars to produce, is:

• ‘particularly effective in checking emails for quality and clarity prior to sending, and helps with
• summarising,
• ideating,
• analysing,
• researching
• meeting management’.

However, they point out that as Copilot ‘is not legally trained’ they don’t use it for any tasks that need ‘legal expertise’.

David Jackson, CEO at Shoosmiths, commented: ‘Shoosmiths is a leader when it comes to innovation, setting the tone for what impactful and effective AI adoption looks like across the industry.’

OK, I’ll take a breather now and a sip of tea before plunging onwards, while also trying to be positive.

It’s commendable that they’ve adopted AI tools to help with very basic tasks that soak up time. This is a good thing, so kudos for that. But, what is presented here is really just the fringe benefits of using any LLM, it’s the early adopter version of what legal AI looks like.

Now, that’s not a bad thing, you have to start somewhere. But, using AI for tweaking emails that you’ve already written normally is not going to move the dial much, nor is using AI for planning meetings, nor doing a bit of ‘ChatGPT, help me with this marketing plan’. Plus, the emphasis that AI is not used on the more important aspects of legal work underlines that this is really not that deep in terms of impact.

There is no discussion about taking key workflows and applying AI and other forms of automation to them. There is no mention of how legal data can be leveraged to create new value – and shared with clients in a collaborative way. There is no mention of productisation. There are no economic data points supplied either. It’s just: we have lots and lots of prompts going on.

So, what has changed? It seems that these are primarily ‘secretarial’ type tasks that have been given over to AI. Is that a bad thing? No, this is good. Congrats for that. But, if this is the cutting edge of legal innovation then we are all in deep doo-doo.

Now, about Pavlov.

Pavlov empirically showed the world that you can change canine behaviour with a mix of stimuli and reward. Of course, most people knew that already, but he made a more scientific endeavour out of it all.

In this case, we have a firm where every time you insert a prompt, no matter how minor in terms of outcome, it helps to provide you with more money, i.e. it contributes to a share of the extra firmwide bonus. Lawyers and other staff now have a big incentive to keep asking Copilot to check those emails, or provide a meeting plan.

In fact, has the stimulus and reward combo actually led to some staff spending more time than is needed on inserting low value prompts – as they are now conditioned to do this in order to make more money?

I.e. you get what you reward.

Conclusion

AL wants to be positive whenever this site hears about a law firm bringing in AI tools, but….well, if you’ve got this far then you’ve read the above.

But, it’s a start. Maybe just drop the prompt targets and focus on tangible benefits for the firm, and equally tangible benefits for the clients.

—

More about the firm here.