Executive responsibility, UX design, and product management

This bit from Jim Highsmith’s essay 18 Years Of Agile Manifesto For Software Development #1: History provoked this post:

This type of situation goes on every day — marketing, or management, or external customers, internal customers, and, yes, even developers — don’t want to make hard trade-off decisions, so they impose irrational demands through the imposition of corporate power structures. This isn’t merely a software development problem, it runs throughout Dilbertesque organizations.

Absence of clarity

Highsmith is not the only person to read Agile as an adaptation to software development teams getting jerked around. “If we cannot get clear direction then we have to work in a way that makes the churn less painful.”

When we talk about “hard trade-off decisions” which software development orgs avoid, we typically think of stuff like feature prioritization and allocation of time & resources. But the tendency to look away from those tactical choices flows from vagueness at a more strategic scale. An absence of clarity about product intent (what should this thing be?) and development plans (how shall we create & deliver it?) produces tactical churn. Organizations grounded in strategic clarity suffer less churn, so they do not need the radical flexibility which gives Agile development its name.

Fred Brooks put his finger on this in The Mythical Man-Month half a century ago, and what he said then remains true: software development requires thinking very clearly about what we want and need a system to do before we try to build it.

I submit that software managers & executives have generally abdicated their responsibility in failing to produce and maintain that clarity. This happens because, as Brooks describes, clarity is hard.

The industry generally operates as if clarity is not merely hard but impossible. Resulting norms reïnforce churn & confusion rather than resolve them. Agile accepts these consequences as inescapable … then in actual Agile practice, the appetite for clarity re-surfaces anyway, as we see in Scrum’s dream of a Product Owner role.

The parable of the architect

Agile reaches for clarity by building; the “waterfall” processes Agile derides often do much the same thing in practice.

Software development does not much resemble building a house, but we can get away with a loose analogy:

Imagine a world which builds houses like we build software.

A person who wants a house would go to a building contractor who would ask for a feature list. How many doors and windows? How many kitchen cabinets? How many stairs?

Then the contractor would get to work: pouring concrete, cutting wood and steel, et cetera.

At the point when the house is half-framed, the contractor would bring the future occupant in for a first look.

“Why is the bathtub in the kitchen?”

“Well it stands to reason with the plumbing. Is that a problem?”

After the resulting heated exchange, the contractor would walk away shaking their head thinking, “There you have it again. Customers always change their minds about their requirements because they just cannot name their needs until they see something mostly built. You have to expect these disruptions and roll with them as well as you can.”

Every house would end up looking like it was drawn by Dr. Seuss.

An architect from our universe would try to help, but would seem crazy to the people in this world. “You want to draw up ‘plans’ first, before we start construction? You are dangerously naïve! Who can afford that extra time and expense? Don’t you know how customers’ ideas constantly change?”

The software industry needs a practice comparable to architecture.

One may protest that we already have roles not-coïncidentally known as “software architects” or “system architects”, but these do not address the bathtub-in-the-kitchen problem. To solve that we need a visualization of the user experience of a system, analagous to a building’s blueprints, before we even start to build it.

Skepticism about UXD

Why don’t software development organizations make some kind of user experience “blueprint” first?

We do not reject doing it. Like the people in the world of Dr. Seuss houses, it does not even occur to us as a possibility to reject.

If one had never encountered someone with perfect pitch, one would never depend on having someone identify musical notes. Visualizing the user experience of a system which does not yet exist depends on a comparably surprising knack.

The few people in the industry who have seen attempts at making such a “blueprint” know that most fail. Doing deep UX design requires not just the knack but subtle skills & processes which differ greatly from other practices in the industry. Without all of that lined up, a UX design project fails before it even starts.

Plus people in key roles have their own hesitations.

Programmers

The common expression “UI/UX” reflects an assumption that “design” means wrapping an “interface” around a system which programmers define. Programmers therefore tend to read proposals to do deep UX design as intruding into the the work they do on the system. Even if they do appreciate deep UXD, hard experience teaches them to doubt that it will protect them from churn as the organization calls for changes.

Programmers feel such responsibility for building a good system in the face of the usual clumsy, shifting, ill-conceived direction that they are notoriously stubborn in resisting plans which they fear will compromise the quality of their product. They have every reason to expect designers to deliver yet more bad direction just as Marketing and executives too often do. The word “designer” often reads to them as meaning an artsy fool dangerously naïve about what is easy, hard, and impossible to implement … which proves true all too often.

In my long experience as a UX design consultant, once I deliver deep UXD it thrills programmers. At last someone offers the clarity they have always needed to plan and work effectively! But that opens another challenge. When programmers with deep UX design in hand say to marketing, management, and executives, “when you give me direction it should come in this form”, it creates political complications ….

Executives & managers

Even if one can persuade executives & managers that deep UXD is possible, its methods are sound, and it supports them in making decisions, they hesitate over the scary changes to organizational structures & processes which it requires to leverage properly.

UX design takes time to prepare before setting programmers to work coding. This feels like an expensive delay in an industry which conceives of “making software” as programming. Though UXD saves time by avoiding bathtub-in-the-kitchen problems, skipping unneeded functionality, and easing other frictions from unclear direction, these advantages are hard to imagine if one has never seen it happen. And even after a deep UX design project, one cannot see the problems which did not come up because the design preëmpted them!

Creating a clear UXD plan does not just enable many of the strategic & tactical decisions which Highsmith describes software organizations refusing to make, it also requires making those decisions. This can feel awkward to software managers & executives who have learned to thrive in existing environments. Surfacing those decisions makes responsibility for them — and the risk of getting them wrong — a new burden.

Executives who understand Vision as their job can take designers articulating product vision as a threat. That is not a fault from ego; a person with the temperament a good executive needs can have a hard time registering designers’ characteristic disinterest in glory, misreading designers as disrupting the executive’s necessary authority.

Committing to the changes necessary to properly weave deep UXD into an organization’s processes presents a risk that observers will blame any failure for any reason on those changes. As Archibald Putt teaches us, the tech industry forgives or even rewards executive or management failure if one fails for conventional reasons. Why take the career risk of an unconventional failure?

UX designers’ skill or other constituencies’ enthusiasm cannot address these challenges. To leverage design effectively, an organization needs a strong product management practice.

Product management

Product managers shape their product to meet business goals set by executives. To fulfil this responsibility, PMs must own every decision about their product and every decision about the process for making it.

Technical opportunities & constraints identified by engineering, customer drives identified by marketing, user needs identified by designers, and more all inform those decisions. When necessary tensions arise between those constituencies’ concerns, they must turn to the PM to adjudicate them.

This may sound like a PM either having to do everything or having to throw their weight around giving orders. But in practice, good product management is overwhelmingly collaborative, with the PM communicating, translating, and facilitating discussion.

“Hey, Mr. Designer, Marketing says we need an answer for customers who ask for Feature X.”

“Cool, Ms. PM, we should address that with UX Y.”

“Hey, Ms. Programmer, if UX Y proves hard to do, I don’t want you to grind on it. I will put you in a room with Ms. Designer to hash out a solution almost as good which you can deliver more easily.”

This understanding of product management underlines the mistake UXDs make if we claim we should Own The User Experience. It sounds natural, and reflects frustration at the way the industry dismisses our professional expertise, but since the UX is the product in users’ & customers’ eyes, and since concerns about marketing, business goals, and technical development therefore also inform the shape of the user experience, PMs must own it. Designers own a narrower space of design judgment about what serves users best.

This does not mean that PMs author the UX design plan. Designers propose, refine, and communicate the UXD plan because they have the relevant skills; PMs exercise their ownership of the design through editorial control of this work by designers.

This relationship plus PMs mediating the concerns of other constituencies which must inform the design creates a container supporting designers doing our best work. Mediocre designers supported with a strong product management practice will deliver better product UX than great designers subject to weak product management.

This exemplifies how product mangement is management, ideally making decisions and sharing information without making stuff. Wherever one finds PMs doing “real work”, they are filling a gap the organization should patch with whatever resource is missing — usually the attention of an expert in that domain.

Facing upward in the organization, PMs enable executives to focus where they should, on business goals. Executives need not examine the UX design or other elements of the product plan; the PM handles those, in pursuit of the goals the executives set. The PM role compels executives to sharpen goals where they are unclear. “We have talked about growth, and Marketing & UXD have discussed some different approaches. X should get us the most users, Y should get us the most sales revenue, and Z should get us the most profit. Which kind of growth do we want?”

Thus PM + UXD equips executives to make informed decisions which provide the clarity absent from most software development.

To people accustomed to the process chaos Highsmith describes, this can sound like silo’ing. But as with the example of PM mediating between Marketing and Design, good product management and clear role distinctions actually enable vigorous and graceful collaboration between different constituencies, including executives.

---

This post captures and refines a Twitter rant from December 2022; I significantly updated it with a host of edits in April 2024