1 of 10

Slow wisdom. Fast execution

Developing AI in Law Firms and Legal Tech Companies

Laura Hodgson

Principal Product Manager, Clio

1

2 of 10

2

What is the best way to design AI systems for lawyers?

Insights from two perspectives - law firm and legal tech

3 of 10

Multiple different approaches, with benefits and disadvantages depending on the approach
Is there any kind of synthesis of approach and any consensus on best-practice yet?
Where can tech firms and law firms collaborate to get better results for all lawyers?

3

Multiple different approaches with pros and cons on either side

But I think what is yet to happen is anything like a consensus or a collaboration between firms and tech that will really deliver what lawyers need. There are some great examples in pockets but I feel the legal industry as a whole is struggling: struggling with the opportunity gap between smaller and larger firms; struggling with cost; struggling to understand the law on the one side and the technology on the other.

This is my call today - for both sides to learn from the other and to collaborate a lot more than is currently happening

I have worked in both legal tech and spent many years working in law

Summary:�- Laws firms have a depth of institutional knowledge and practice experience that has yet to be replaced by much of the technology out there

- The technology company side has taught me a lot about pace and discipline.

We have a dichotomy between - knowledge and instinct and speed: lawyer in law firms have a vast depth of practice experience and intuition about how to undertake tasks and talk to clients and the speed of delivery of legal technology companies.

4 of 10

How do these differences play out?
We are looking at expertise vs pace
The gap between those two worlds is too often a source of mutual misunderstanding
Different motivations

4

So what is the nature of this dichotomy? What is different in terms of legal technology firms and law firms in terms of how they approach the development of AI within law.

Point about context - the stats

2024 LexisNexis survey of over 1200 legal professionals for work and observed that the use of AI had nearly quadrupled in just over a year.

According to Clio’s Legal Trends Report, 96% of UK law firms now integrate AI into at least one of their operations

The professional is adopting AI tolls faster than it is developing the structures for adoption.

In many cases, law firms are under huge pressure to be seen to use AI and are sometimes neglecting effective adoption - with exceptions where firms combine purchase of AI tools with rigorous adoption and training programmes. Meanwhile, technology companies often don’t fully understand the adoption challenges that firms have

5 of 10

AI development and adoption within law firms - observations

Delay

Bias

Leadership interest

Decisions are almost always made by committee
The space between ‘good idea’ and ‘go’ is measured in months, not weeks
By the time a law firm has approved a product, its features have already been superseded

Practice-group bias - the loudest voices innovate first
These interests do not necessarily align with the firm’s interest

Direction shifts not because the evidence changed, but because the person at the top did.

5

Decisions are often made by committee - the partnership model introduces two persistent problems:

There is a significant gap between a decision to do something and that thing being complete. Projects are measured in months, not weeks

By the time the outcome is achieved, the technology and the legal market has moved on

Bias. Within law firms, bias plays a significant role in how technology is developed. The partners who carry the most weight in these deliberations are frequently those with the strongest existing interests, and those interests do not always align with where the technology is going or the firmwide interests.
There is also a third dynamic: senior leaders can change strategic direction without shared decision-making. Direction shifts not because the evidence changed, but because the person at the top did.

6 of 10

AI development in legal technology companies - observations

Product-centric

Too fast?

Lacking depth

Decisions are made by product managers
Product managers have breadth of responsibility and velocity of delivery
Failure is part of the process

Products released before they are fully ready
Fix afterwards mindset
Lack of engagement

Focus on the big ticket issues of document reviews - corporate and litigation
Leaves some areas of legal practice under served

6

In a legal technology company, decisions are made by product managers with a portfolio brief - along the lines of: keep your product relevant and iterate towards greater usage or greater sales.

This produces something law firms struggle to generate: nimbleness, intuition and velocity of delivery.

Speed has its failures - products released that are not fully tested

Focus on the most lucrative types of work - large document reviews

The vast market opportunity and appetite for products is arguably moving too fast.

Many product developers have no legal experience and have a shadow appreciation of what working as a lawyer is like

Lack of deep knowledge of law or a subject matter expertise

Creates gaps in coverage that don’t help industry lawyers or specialists

This leads to tools that will do 60% of legal work but 40% of work in a firm is left un-catered for.

This gap gets even worse for company legal departments who need that specialist subject matter expertise

7 of 10

Need to learn from one another

Law firms needs to allow greater speed and flexibility amongst the teams building AI solutions
Technology companies need earlier and deeper engagement with the people who will actually use the products
Both sides need to look at the skills of those working on legal technology and share more
There could be a better way…

7

8 of 10

A skills gap

Need for improved understanding of technology within law - not just in the largest firms
Legal technology firms need far greater practice experience - especially beyond corporate and litigation work

8

The most obvious way to improve how things are working is to bridge the skills gap:�Greater clarity within firms about the roles in which lawyers can help develop technology - still some confusion about the status and function of innovation in law firms and how these roles interact with knowledge etc.

Very little technological knowledge in most firms - machine learning specialists are very rare in law firms; non-existent outside the very largest firms.

Firms going it alone and building their own AI capabilities - I believe - is very challenging - in terms of the number of engineers you need, the understanding between how law firms work and data utilisation, etc.

Tech companies need to do more to appreciate the real challenges in firms and the breadth of work that needs AI to support - not just document review.

Sometimes the detail matters - take it slowly and focus on depth.

Evaluation - then the biggest shared problem. Much more needs to be done within the legal industry to improve the evaluation of AI.

9 of 10

Where both worlds must collaborate

Evaluation. Both law firms and technology companies struggle with evaluation

Identifying hallucinations is critical and should involve both firms and tech
This actually requires human effort and thus takes time!

Secondments - each world must learn from the other (especially engineers)

Law firms should ask for more support
Co-design is ideal

Build greater trust

9

10 of 10

Thank you

10