Design and conceptual integrity

This post captures lively Twitter discussion from January 2022 featuring some of my favorite people from Twitter.

---

John Cutler asks:

UXR [user experience researchers]...

have you ever done skilled, moderated usability testing of a design ... exposed major problems in the design ... had the fixes shot down (maybe someone wanted to be *bold*, or “just ship it”)

...shipped it as is

...and it actually turned out fine?

Andy Budd replies:

I guess it depends what you mean by “fine”.

I’ve definitely seen teams ignore the results of usability tests. The issues they identified still happened. However the bulk of users managed to struggle through. They ended up slightly more annoyed, but task completion remained high.

Tom Kerwin says:

But it’s a fair point: you can almost always find usability issues, and you’re almost always going to have to launch with some, and it’s not always obvious which are truly critical.

On top of that, some issues are very time consuming or complex to fully solve.

Experience helps.

Every discipline has their own lens.

E.g. Folks working in fraud tend to assume every failed conversion was a fraudster who got cold feet.

It’s blind men feeling an elephant.

As Venkatesh Rao put it, all are “right” but one of the blind men is the HiPPO - their truth runs the show.

Budd replied:

True. Also “Shot down” is highly emotive language. An alternate framing is “weighed up the potential problems against the opportunity cost, and decided that even if there are inherent usability issues, it made business sense to launch anyway”

Designers seem to spend an awful lot of time framing business decision in a way that paints them as the victim and everybody else as uncaring idiots.

Kerwin replied:

So true!

It’s classic Fundamental Attribution Error

First: if you saw the facts, you’d agree with me.

Then: oh you see the facts but don’t agree? You must be too stupid to understand

Then: oh you understand fully but don’t agree? You must be evil.

Pure tribal thinking

---

And a lot of design rhetoric goes towards, “how can I persuade others to see it my way?”

Much less goes towards, “hang on, what if it’s me that needs to update my beliefs?”

Ironically, it’s when you come in truly open to you being wrong that you start to be able to influence

Venkatesh Rao replied:

Jiminy Cricket syndrome. I think designers often see themselves as the conscience of a project. Sometimes the rest of us go against what they want simply because nobody likes being constantly sermonized at. Maintain enough profanity in the process to keep designers in check 😆

I responded to Kerwin’s comment:

There are two things entangled here which too many designers have a hard time disentangling:

Is the problem that colleagues fail to prioritize UX issues as highly as we do, or that they refuse to accept our assessment that the issues are there?

When designers gripe that our colleagues refuse to prioritize UX issues as highly as we think they should, we need to update our beliefs.

When they refuse to accept our assessment that the issues exist at all, they are burdening us with persuasion we should not have to do.

Kerwin replied:

This is an interesting one to unpick.

I think it goes beyond designers. All makers, all craftspeople tend to assume that the issues we see are critical, and get frustrated that others don’t seem to care.

With engineers, it’s technical debt, code quality.

Often we forget to step outside our own technical language and explain why it might matter to others — whether that’s coherence, techical debt, infosec. The best in each craft domain tend to be able to frame the issues as business trade-offs : the choice plus its consequences. I’m not sure that’s persuasion we shouldn’t have to do

I’ve seen it worse with engineers than designers, to be frank. Exec: “oh those engineers are always complaining about technical debt. I just nod and then get them to dash off the MVP anyway. It’s been fine so far!”

This issue must be pretty timeless, as it’s baked into old stories: Chicken Licken, The Boy Who Cried Wolf ... and there’s no easy fix. Effectively translating deep, subtle craft concerns into the comparitively crude language of business takes patience, empathy, humility.

It’s always easier to throw up your hands and grumble about “the idiots over there”

I replied —

I think you are missing the distinction I was trying to make.

Yes, people in almost every profession find that the concerns of their domain are not prioritized as they think they should be. And they are often at least half right, as in the example of technical debt, but while engineers may say something like “we should do X server-side; though it will less responsive for users it will be easier to support” and have leadership choose client-side because they do not prioritize future support challenges highly enough they do not face leadership routinely saying, “we will do X client-side because we think engineering are wrong, it will actually be easier to support”, in the way that other constituencies often tell design “no, you are wrong, users will be better served by something else”. So the whole question of “persuasion” is screwy here. I am hesitant to think that designers should be doing persuasion at all internally. On the one hand, as a designer I may have opinions about strategic product/service decisions which prioritize certain UX elements more than Eng or Mktg consider important, but I should not be trying to persuade, I should be giving PM clarity in weighing the decision and on the other hand, when providing that clarity, when I say that A is much better UX than B, and C is just a bit better than D but worth doing if it is easy, I should not have to persuade my colleagues that I am right in that assessment.

Now in practice, designers have propagandize the value of good UX and the dev practices which support it, strategically. But that is different from having to persuade colleagues of our expertise in our own domain in the tactical day-to-day. Solving UXD problems and communicating UXD solutions is hard enough. If UX designers also must persuade colleagues that those solutions are good, it wastes everyone’s time & energy, and feeds our temptation to overstate the importance of specific solutions.

Kerwin replied:

In design and writing, our work is both more visible and more subjective than, say, engineering. Our work is much easier for everyone to have An Opinion about.

I replied:

Exactly! Engineers face their own distinctive challenges in being respected, but the opacity of the material protects them from the routine substitution of others’ judgment in their domain which designers often face.

Kerwin replied:

I find it’s better to put the time and energy into persuading people that most ideas and solutions are totally fungible ... and that requisite variety can be more valuable than consistency. The more you’ve had to fight for something, the harder it is to let it go.

I replied:

Oh yeah, this is a good point about the sunk-cost-ish psychology of sandbagging the importance of a UXD solution (or anything by any constituency) which one had to fight for.

Rao joined in:

Not sure why I’m tagged here but I’ve nearly always been on the opposite side of this issue and in my experience it’s usually fine to ignore designers and ship because there are usually bigger risks/uncertainties baked into the release that overwhelm design considerations.

In my experience designers tend to resist thinking in terms of probabilities and risks if they are sensitive to them at all. Design is a binary to most of you — it either has integrity or is somehow philosophically compromised.

PMs and engineers more naturally think in terms of probability that something will matter and the costs if it does. Designers and infosec people tend to be on the other side. Everything is either perfect or a showstopper.

PMs own risk so typically have their way.

The best designers I’ve met tend to act like lawyers. They lay out risk comps rather than design comps, and understand that they don’t own the risk.

Dorian Taylor chimes in:

infosec people too for that matter

Cutler responds:

IME, everyone -- PMs and eng included -- fail to consider non-linear dynamics, the oddities of time, our propensity to fool ourselves, overestimate their resolve to “fix things later”, and underestimate the oddly exponential value of a job well done

I replied:

Well said.

One thing that drives UXDs bats is that dev orgs tend not to register the importance of design coherence. Often any one of a hundred compromises to the UX is trivial by itself but if you screw up a dozen of them the whole thing falls apart.

Kerwin replied:

So much this. And once you’ve got to incoherence, it’s very hard to “fix” incrementally.

I’ve noticed that a lot of designers smell this kind of problem but reach for consistency - which doesn’t really fix the problem but does create a lot of headaches.

Lots of designers are trapped starting with “ideal” screens and then those get hacked up as scope is cut to meet deadlines. Unclear when coherence is lost.

Better to start with a coherent “skeleton” and then add flesh iteratively.

Taylor replies:

unless you can’t even make it to “coherent skeleton” before that gets hacked up as scope is cut to meet deadlines

Kerwin replies:

LOL – yes, but then it was all doomed anyway.

A coherent skeleton concept model can be a dozen sticky notes and connecting lines.

Taylor replies:

yeah but the work to create a coherent skeleton (brooks called it “conceptual integrity”) looks like “nothing” to onlookers until you have one, so even that is a risky period

I replied:

Yes. And so few people have ever seen it done that when designers talk about doing it, it sounds absurd.

Taylor replied:

the thing that seems to be perennially hard for designers to communicate is what Venkatesh Rao called “philosophical compromise”, and like absolutely yes it is philosophical compromise, with real material consequences.

like it isn’t just some aesthetic preference; it affects outcomes.

I replied:

Exactly. And few people understand how brittle that coherence is: it can absorb a few small compromises, but not many, and one big hit can shatter it.

Taylor replied:

put it like this: say you’re an engineer working on a bridge and the client says take out half the suspension cables — no engineer would put their stamp on that

now what if the client says to the architect: “we’ll build the bridge, but not all the way to the other side”

Kerwin replied:

I was an engineering student once upon a time

We had to build a cantilever

My maths was good

My measuring – not so much

My cantilever buckled under almost no load

I learned quite a lot

Taylor replied:

you would have gone to prison if you were the engineer of record

Kerwin replied:

That was one of the things I learned!

Also that you can go to prison for other people’s mistakes

I replied: two generations from now people will be baffled that software dev was not professionalized like structural engineering.

Taylor replied:

the engineer who rigged the volkswagen dyno hack went to prison but his boss didn’t

also for the record i do not support licensure for designers per se

I replied:

In the long run I would like a guild model of some kind, but hard licensure requirements like with medicine is too much. I’m not sure exactly what that looks like.

Taylor also replied:

the protestations of designers being dismissable as the whinging over the personal aesthetic preferences of an effete class of trust fund children is not helped by the fact that the design field is in fact populated by an effete class of trust fund children

i should clarify: not exclusively populated as such; perhaps not even a majority, but enough to run into them often enough to look like a majority if not a totality

To which I replied: I do notice that designers who are not craftspeople but rather would-be artists have a strong tendency to have more expensive shoes than their pay can account for.

Rao replied:

This is more of a western pattern, especially North American. In my experience, in Europe and Asia, designers seem to be more… normal. In North America, I think “design” probably got tied up in “critical design practice” French theory crap and wrapped up in esoteric political concerns yo some degree.

Taylor replied:

imo d-school (and architecture school) is where the offspring of the professional class go when they’re “creative” and don’t want to go get an mba or become a doctor or lawyer and their parents will shun them if they get a fine arts degree; they come out wearing black turtlenecks

Taylor said:

i remember seeing some article about social networks that a minority can look like a majority if they are in the right places (eg central or spanning) in the network; i probably have it in zotero but don’t feel like scouring for it

Rao replied:

They do tend to dominate social media about design

Taylor replied:

yeah because they’re charismatic and know a lot of obscure names and facts

I replied:

Lots of trust fund folks throughout the startup world, of course, which is under-discussed.

Taylor replied:

i am not a trust fund kid but at 18 i wanted to go to art center in pasadena to learn how to design cars (because i wanted to self-express) and for a brief moment my extended family was willing to pool their resources to send me; that would have been a terrible, terrible waste

i mean if for no other reason than the harsh reality of automotive design is that it is an even more extreme situation than architecture—practically nobody who enters the field ever gets to do anything cool, lol

(that and by 21 i had completely stopped giving a toss about cars)