"Messy" and high end law firms - some reflections
I’ve recently finished reading Tim Harford’s new book, Messy. I was attracted to it because I’ve long enjoyed his writing for the FT and, more concretely, because the book addresses issues relating to collaboration, planning vs. improvisation, automation and algorithms of relevance to what I’ve been thinking about and working on in recent years in the context of lawyers and law firms. In this post, I’ll discuss what Harford has to say in the four areas just mentioned with particular reference to high end law firms.
Harford’s overall theme concerns the importance of drawing “magic out of the unique challenges and atmosphere of a particular moment, stepping away from structure and control but gaining an undefinable energy as a result.”
The book is wide ranging and I won’t try to cover everything, but here’s my attempt to summarise some points of interest in the four areas mentioned above:
- Collaboration. Harford emphasises the desirability of working with strangers, and people from different backgrounds if you want to do more effective work in areas in which creativity matters. Cognitive diversity is important. Being required to think about a problem from uncomfortable, seemingly irrelevant, angles can help avoid groupthink. It’s desirable to find ways to expose yourself to different approaches in an almost random way, given the human tendency to retreat to comfort given half a chance. When it comes to collaboration in complex projects, in which success requires effective input from teams with different perspectives, Harford emphasises that the tension and stress resulting from the teams’ interactions can be powerful and shouldn’t be ducked. That said, the tight-bonding of a given team (which, as Harford acknowledges, is useful for increasing performance in very focused tasks), may lead to conflict and other problems when it interacts with other teams. For that reason, Harford emphasises the importance of people who can effectively bridge between two tightly-bonded teams by virtue of being trusted by both based on close prior working experience with each (“a tough and undervalued role”).
- Planning vs. improvisation. Harford makes the point that giving people freedom to improvise, respond intuitively to complex situations and go for fleeting opportunities, without being constrained by detailed rules, processes and scripts, can be a good way to deal with novel and challenging situations. This part of the book starts with examples from contexts such as musical performance, political speeches and customer service in challenging consumer contexts. He then moves on to discuss various military examples, a business one (Bezos/Amazon) and a political one (Trump, who at the time the book was written had recently secured the Republican Party nomination).
- Automation. Harford observes that seeking to maintain human “supervision” of a substantially automated task is challenging, since automation which is generally reliable is likely to cause the human to stop paying attention, place too much reliance on the machine’s outputs, fail to acquire relevant skills and allow existing skills to atrophy. All this naturally reduces the human ability to act effectively in the very situations which are the purpose of retaining a human supervisory role in the first place. Examples are given of this in contexts where quick reactions are key – in relation to aircraft, drones and cars: “When something goes wrong in such situations, it is hard to snap to attention and deal with a situation that is likely to be bewildering.” Harford then discusses the broader tendency towards passivity and over-reliance upon generally reliable systems even in situations where there is no great time pressure, illustrating the point with various examples of over-reliance on GPS, and an example of a governmental refusal to recognise a data error reminiscent of the film Brazil. Harford suggests experimenting with techniques such as artificially seeking human input more often than strictly required by a semi-automated system, as a way of keeping people engaged and skilled. He illustrates his point by reference to a Dutch traffic expert who emphasised the importance of designing in some vulnerability and danger into intersections as a way of getting drivers to pay attention.
- Algorithmic assessment and matching of people. Harford suspects that the placebo effect may be much more significant than generally recognised in attempts to match people algorithmically to each other, or to roles. He mainly discusses this by reference to the online dating context, focusing on the well-known McKinlay / OK Cupid case, though he also touches upon the use of personality tests in recruitment (Exhibit A: The Rev. Paul Flowers). Harford’s interpretation (“the algorithm itself was worth very little”) of the experiment discussed in point 3 of a July 2014 blogpost by OKC’s founder, Christian Rudder is different from Rudder’s own summary (“suggestion is about half the story, and whatever we think your compatibility is about half the story. The best thing by far, roughly twice as powerful, is to both be told that you’re good and actually be good.”) Harford is sceptical of the typical “big data” sales pitch, remarking that “When it comes to breeding horses, it’s reasonable to think we can arrange the qualities we’re looking for into a tidy list: temperament, strength, speed. When it comes to finding someone who can make us happy, or manage not to nearly bankrupt one of our venerable financial institutions, the qualities we want are altogether harder to define.”
Reflecting on these ideas in the context of big high-end law firms:
- Collaboration. The nature of much high-end legal work is that there are opportunities to work with uncomfortable counterparties, though of course individuals vary in their willingness to take those opportunities. Diversity in all senses is still work in progress in most high-end law firms, particularly at the senior end. Some aspects (e.g. gender, race, LGBT, social/economic background) are least being recognised and starting to be addressed these days. Cognitive diversity in the sense of deeper involvement of people from backgrounds other than law – is perhaps even more challenging, given the existing gaps which exist in most firms between the lawyers and everyone else (e.g. IT, finance, marketing, HR). This is a really tough issue to address effectively within existing large firms: I suspect that doing so may be an important factor differentiating those who prosper and those who do not over the next decade or two.
- Planning vs. improvisation. Responding in a nuanced way to situations is the name of the game in high end law. Having spent virtually all of my working life in environments with a high level of autonomy, I appreciate the value of a bit of mess. The really difficult thing, in my view, is to strike a good balance between the requirements of creativity (e.g. mess) and efficiency (e.g. process) and, in particular, how best to avoid impeding the former while striving for more of the latter. This sounds pretty banal when expressed like that, but is, I think, as important as it seems to be challenging to achieve in practice. More on this below.
- Automation. Aside from legal research and the ever-more-rapid generation of text, automation has so far made relatively little progress in the traditional “front offices” of most high end law firms. It’s increasingly a different story in their administrative functions and in the captive fee-earning operations which have over the last few years been established by many firms to take a more organised approach to executing certain types of information-intensive work (e.g. disclosure/discovery, due diligence, contract management). In the latter context, it’s noticeable that real success has involved leadership by people who really get the importance of the human factor and who are acutely aware of the limitations and dangers, as well as the benefits, of automation. As the potential increases for automating aspects of legal work currently done by lawyers, the questions which Harford raises about how to keep the human role meaningful are likely to become ever more significant. The “solution” we seem to be trending towards in the case of cars and aircrafts (i.e. full automation) isn’t likely to be achievable in law, since the interpretation and application of legally-significant artefacts seems likely, I think, to continue to require significant human involvement. Even as the nature of evidence changes (e.g. greater digital records of where people were and what they did or said) and as fundamental legal artefacts (e.g. contracts and statutes) are increasingly generated in digital formats, I suspect that the human factor will still be significant. This assumes that the future of law continues to involve legal rules which have a status different from administrative guidelines, coupled with a credible institutional machinery for enforcing them, and a commitment to a detailed and objective engagement with messy reality, including complex human motivations and ambiguous behaviour. On that assumption, I think law firms have a future, but there is a lot of hard work to be done in working through the organisational and day-to-day working implications of the sorts of issues that Harford raises.
- Algorithmic assessment and matching of people. Algorithms have, in effect, been used in law firm recruitment for years, for example in the use of University name and degree results as a “good enough” mechanism for filtering junior lawyer job applications down to a manageable volume – a method nowadays increasingly seen as too simplistic. Over time, more detailed scoring methods have developed but always, so far as I’m aware, with positive hiring decisions being essentially human decisions, based at least on multiple interviews but, more likely these days, supplemented by other methods (e.g. vacation work schemes). There are some ongoing experiments in some firms to find ways to reduce the risk of unconscious bias by exploring “blind” work allocation and holding interviews without providing interviewers with details of the candidates’ background (obviously not a watertight approach but maybe more important as a signal to everyone involved as to what really matters). There has also been well-publicised use of other tools (e.g. assessment centres, psychometric tests, even a video game), though the precise weight that they play in hiring decisions is hard to pin down. It’s noticeable that marketplace-type apps for lawyers, such as this recruitment one, focus on fairly simple definitions of whether an applicant is qualified (e.g. area of expertise, location, years of experience), perhaps with limited other evidence being available (e.g. reviews/ratings in the case of a “legal services” app) but with the “purchaser” (prospective employer or client) ultimately being left to consider the more complex questions of suitability. See here for a short review of the potential benefits and risks of more ambitiously algorithmic methods being tried in recruitment, though not specific to law firms. A big question is what will happen if attempts are made to take a more “data-driven” approach to questions of internal work allocation and reward in firms. Clearly, there are some serious risks to consider. It’s interesting in this context to note the increasing reaction against the supposedly “objective” (but in practice very costly yet still often perceived as political/unfair) performance review systems developed by most large enterprises in recent decades, though a question is whether the trend will now be towards more data-driven approaches or more human-centred approaches. Perhaps both, but much will depend on context. For example, at junior lawyer levels in high end law firms, I am doubtful whether it makes much sense to take algorithmic methods of work allocation beyond the basics, i.e. flagging people who have capacity and some experience (possibly self-certified) in a relevant area, but leaving people to make a human judgment as to whether to involve someone in a project. If one tries to take things further, risks of gaming and other unhappy outcomes may arise. In so far as the problems being tackled are ones of bias, it remains to be seen whether algorithms are going to make a significant contribution. My own intuition is that Harford is right to warn against over-hyping, though it seems worth undertaking some (very) carefully designed and objectively assessed experiments in this area.
Stepping back from these specific points, a broader reflection is that, while high-performing law firms benefit from a lot of good, productive mess (to an extent which, I think, is often underestimated from outside), there’s also a strong element of wasteful mess (often underestimated, I think, from inside).
A really big question now being considered, under various guises, in most high-end law firms is how much falls into each category and how to address the problematic bits without harming the good bits. I think that the dangers which Harford points out are real ones to watch out for in untangling them.
I believe that real progress can be made, but it’s certainly not easy in the kinds of law firms where creativity is a significant part of the value they provide. It involves a sensitive understanding of how people with different skills and backgrounds can effectively collaborate and communicate; a careful, highly context-specific consideration of the multiple causes of waste and inefficiency in a law firm; an astute assessment of how to encourage more efficiency in a way which is empowering rather than deadening, with close attention being paid to the possibility of unintended consequences; and tools which support such efforts.
Switching to the area I now work in (i.e. software, after all those years in legal practice), it seems to me that there is quite a lot in common between Harford’s ideas and those which led to notions such as Agile and Scrum, which have become influential in modern software development – partly as a reaction to the problems of over-planning and excessive tidiness which Harford identifies.
While these notions aren’t expressly discussed by Harford (one can’t cover everything!), I thought it was interesting that a famous example which he does discuss in some detail is one which also appeals to Jeff Sutherland, one of the leading lights behind Scrum, and is discussed in the latter’s 2014 book – namely, the case of Major John Boyd, a famous US pilot and tactician, and his analysis of (among other things) what makes a good fighter aircraft (clue: ready visibility of what’s happening outside the cockpit is quite important).
An approach like Scrum does involve a process supported by tools, but it’s “process” in the sense of a set of disciplines to ensure that there are opportunities to discuss important things in a focused way; process with the aim of opening up discussion rather than closing it down. This is a notion which I think law firms can also learn from, and which will become all the more important as automation (with its dangers) becomes more significant. That is to be distinguished from “process” in the sense which I think most lawyers bring to mind at present when the “p-word” is mentioned, i.e. reducing the way of conducting a particular task or project type into a more-or-less defined workflow. Both are important.
On a slightly different note, a contrast, I think, between Scrum and Harford’s ideas is the question of whether team collaboration on complex projects is best advanced by bridging between highly focused small subject-focused teams (one of Harford’s ideas) or by promoting “cross-functionality” within small teams, then co-ordinating those teams (a notion important to, e.g., Scrum). There may not be a universal answer to that question: among other considerations, one usually has to take an organisational starting point as one finds it. For example, if you’re starting up a new type of business within a large law firm you may be able to experiment with cross-functionality, but if you’re trying to link up IT, finance and lawyers more effectively then bridging may be more important. Much depends on sensitivity to context, which is of course Harford’s larger point.
A further illustration
Just before I finished this post, I had a look at my inbox and saw an item which is illustrative, at least to me, of why Harford’s themes are relevant in high end legal practice.
It’s a piece by Steve Blank on 31 October 2016 which articulates the problem that really successful businesses face when those responsible for its success depart.
The tendency, Blank observes, is to appoint
“the operating executives who reported to them [and who] believe it’s their turn to run the company… Once in charge, one of the first things these operations/execution CEOs do is to get rid of the chaos and turbulence in the organization. Execution CEOs value stability, process and repeatable execution. On one hand that’s great for predictability, but it often starts a creative death spiral – creative people start to leave, and other executors (without the innovation talent of the old leader) are put into more senior roles – hiring more process people, which in turn forces out the remaining creative talent. This culture shift ripples down from the top and what once felt like a company on a mission to change the world now feels like another job.”
Without wanting to exaggerate the parallels between visionary software companies (Blank’s example) and successful law firms, I think there is a broader theme worth thinking about as law firms seek to get on top of mess: distinguishing between different kinds of process, and between good and bad mess.
One can, of course, find in such a book whatever one wants to find.
I suspect that “Messy” may be a comfortable read for many “old school” lawyers as it could be read as validating a lot of that instinctive lawyerly scepticism about automation, algorithms and systems.
That’s not what I would take from it, though.
In my view, it’s important not to stop experimenting with technology and systems in the law, but Harford’s book contains some useful reminders to think hard about how to make progress without causing unintended harm to the good aspects of older ways of working.
In fact, I would go further – I think that a thoughtful approach to technology and process can not only help to make repetitive tasks more efficient but also help promote the sorts of good mess that Harford emphasises, by helping to tackle a lot of the distractions, lack of effective linkages and communication failures which currently get in the way. But that’s another story.
In summary, with my thoughts focused on high end legal practice (and, to some extent, software development), I found Harford’s book a great stimulus. It’s full of interesting thoughts and speculations, loosely pulled together by an overall theme. Messy, in a good way.
Graeme Johnston is the CEO of Juralio Ltd, formerly a partner in a global law firm.