Agile and UX design

A Twitter rant

First, a capture and refinement of a Twitter thread from September 2020:

Doug Collins asks:

Do #UX and #Agile really mix? Why or why not?

Nick Finck responds:

Agile is a way of thinking that was turned into a process. If you read the manifesto you’d think “sure this works with design” but somehow the execution (process) was misunderstood as “shipping fast” thus compromising quality & effectiveness of good research & design.

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

I think there is a lot of implicit hostility to UX design in the Agile Manifesto.

“Responding to change rather than following a plan” accepts — commits to! — a world in which there is no way to out-think the tendency of “customers” to have “changing requirements”.

I see Agile as a response to software development teams getting jerked around by a failure of leadership to deliver clear, consistent product / service direction. “Fine. We will release attachment, and just get good at thinking short-term and responding to capricious direction.”

A lot of software developers have had their fingers burned by half-baked attempts at UX design. They tried it with the wrong people, incentives, or timeline, and reasonably (but incorrectly) concluded that the coherent vision UXD promises is simply impossible. The Agile Manifesto rests on this assumption that this clarity of product vision is impossible, or at least doomed. Most folks in the industry simply cannot conceive of what good UXDs can do given time and opportunity.

But a dev org still must decide what the product / service / process will be & do. Agile culture locates this in a “product owner”. The PO is expected to deliver clarity & direction ... which is the thing that is still missing from most projects and organizations.

I believe in the product owner role: it is what product management should do, properly understood. Good product management is, in my humble opinion, the most important thing you can do to get products with good UX design.

Even with all those hesitations, as a UX designer I love a lot of things about the Spirit Of Agile. I want to do UXD with a spirit of experimentation. I want to solve the countless onesey-twosey challenges in conversation with devs instead of through a formalized process with heavy documentation.

But I also know how powerful it is to step back with one or two colleagues and spend time — weeks, months — to sink deep into wrestling fundamental product design questions to the ground and pre-visualizing them well enough that everyone is clear on what we want to make.

You can make a film Mike Leigh style, workshopping with actors with a camera running until it turns into something, and get magical results that way. But most movies start with a nerd or three writing a screenplay, and maybe drawing storyboards, before the expensive production.

Agile reflects a pragmatic adaptation to dev org leadership not merely abdicating their responsibility, but in many cases being actively hostile to anyone trying to take it on. UXD seeks to deliver an instrument which enables responsible leadership giving clear direction.

To my mind, the core tragedy of Agile is that it simply accepts the abdication of responsibility by organizational leadership. There is a lot to admire about Agile methods, but there is only so much one can do for the Titanic from inside the engine room. Someone needs to stand on the bridge with a map.

Resources

Criticism of Agile

Agile As Trauma

My favorite single commentary on the subject: it says a lot of things which I had long tried to say better than I have ever managed, plus a lot more.

The Agile Manifesto is an immune response on the part of programmers to bad management. The document is an expression of trauma, and its intellectual descendants continue to carry this baggage. While the Agile era has brought about remarkable advancements in project management techniques and development tools, it remains a tactical, technical, and ultimately reactionary movement. As long as Agile remains in this position it will be liable to backfire, vulnerable to the very depredations of bad management it had initially evolved to counter.

#AgileKillsKittens (or Agile In Their Own Words: The Problem With Agile & Scrum)

An index of numerous critiques.

Agile and the Long Crisis of Software

An analysis which covers a lot of ground.

What I discovered was a long-running wrestling match between what managers want software development to be and what it really is, as practiced by the workers who write the code. Time and again, organizations have sought to contain software’s most troublesome tendencies — its habit of sprawling beyond timelines and measurable goals — by introducing new management styles. And for a time, it looked as though companies had found in Agile the solution to keeping developers happily on task while also working at a feverish pace.

Why Agile Is Losing Steam

A tidy overview of core problems which critics like me foresaw ages ago.

As UX desperately tried to counsel Agile (and failed): 1) The name of the game is achieving outcomes that create bidirectional value for users and the business; 2) outcomes are achieved by making smart design decisions; 3) impactful design decisions are typically made well above the level of product teams; and 4) real design work and having an impact are therefore inherently political in nature. Agile’s response, sadly, was to scoff at this, double down on the error of tailism, and insist that the solution is always “more software”.

Agile vs Waterfall And Other Obfuscations

A word about how agility is of course good … while Agile-in-practice tends to eschew the well-considered-plans-held-lightly which enable agility. I am honored to be quoted there, in very heavy company.

Executive Responsibility, UX Design, and Product Management

A commentary of mine which picks up on the problem identified by Agile and offers the integration of a strong UXD practice into software development organizations as an alternative solution.

From Agilists

The History Of Agile And Its Manifesto

An index of resources which I found by way of Christina Wodke, who pointedly observes that the folks indexed there are all men for some reason.

Why Agile Needs To Take Over Management Itself

An instructive comment about how orgs have resisted any of the deep cultural, role, and process changes they need. But. It contains not one word about users or designers. It complains that “Agile” has had its meaning corrupted, but offers no explanation of what the correct meaning is. Terrifying to a UX designer like me.

The larger the initiative, the more PM will reside above the level of dev teams. To raise the agility ceiling, then, UXD must be similarly raised. And yes, this higher-level UXD work will largely take the form of research. There is a bizarre irony in rejecting such upfront research as being somehow anti-Agile “BDUF” (Big Design Up Front).

Project negotiation

A Twitter thread from Alan Holub which reflects the best of Agile’s spirit, which I genuinely admire … and shows how Agile reflects an adaptation to management’s fundamental distrust of developers and disrespect for what they do and how they do it. I tend to think of Agile as primarily an adaptation to challenges in development projects by corporate IT, but this says some interesting things about the problems it addresses for consulting / contracting development teams.

A traditional “agency” contract is adversarial to its core. You argue about price, you argue about scope, and as the work begins, you argue about changes (and argue more about scope and deadlines and cost). The agency is worried about going broke. The client wants work that they don’t pay for and gripes about paying too much for work they can't see. The negotiation is constant, unpleasant, and an expensive time sink. The risk is very high. It’s a war.

Compare that to an Agile contract, which is about collaboration. We collaborate on what to do first. We build it and collaborate on the details as we build. We then collaborate on what to do next. We deliver frequently so the client knows exactly what they’re paying for. We get paid for the work we do, and the client doesn’t pay for things they don’t need. It's a fundamentally social arrangement. There is very little risk.

When Agile Manifesto is talking about collaboration over negotiation, this is what it’s talking about.

An agilist on UXD

Rob Donoghue responds to “I also know how powerful it is to step back with one or two colleagues and spend time — weeks, months — to sink deep into wrestling fundamental product design questions to the ground”. (I wish I had a link to the source! My page is now the only version I can find.)

So, turning this one over, because when I think about the list of challenges to working with UXD, there are a ton of antipatterns I have to go through first before I could even judge the interaction with agility.

Yes I remain optimistic.

So, here’s the question I’m dwelling on: Is UXD substantially different than other fields which benefit from deep expertise, time and focused work before picking up the tools? How does it compare to architecure, or other fields of design in this way?

And to be clear, I don’t really have an answer - they seem similar to me, but that's from the outside, and I’m sure there are nuances I miss.

On the design front, I am much more likely to encounter situations where design resources are stretched so thin that all the good intentions in the world wouldn’t get us the kind of deep thought it would deserve. But architecture is something we can do. Sometimes.

And agile hat on, it kills me when we don’t do that work or take the time to do it. It makes the work feel like walking on thin ice, and it’s not the push for working in sprints that keeps it from happening.

Now that gets me thinking: One of the reasons we at least carve out a space for architecture is that it’s already in the wheelhouse of the devs, at least in theory. Other areas of deep work don't get the same attention, and some of that may be an agile problem.

Not because of any intrinsic resistance, but because of the emphasis on cross functional teams. As much as we love to talk about stacks, the reality is that deep domains also tend to be specialty domains, which are hard to fit into a cross functional team.

This shows up a lot with designers. It might be great to embed a designer in the team, but it’s only cost effective in certain situations, so we build weird half measures to try to include design, but it's usually flawed.

Same is true of almost any specialist.

What you sometimes see is that in attempt to be cross functional, you get people who might know a little about a given discipline. Sometimes that works out ok, but in some fields it's arguably worse than nothing.

Now given all that?

I’m honestly not 100% sure what agile + UXD should look like, because my experience is with only half the formula. But that experience tells me that the main blockers are just knowledge and intent.

(But I’m also optimistic).