I did an interview with InfoTrust on UX and analytics a couple of months back, and the interview is now up on their website! Check it out if you have a chance.
I don’t know how we got there, but use of “user testing” instead of “usability testing” is pretty widespread and I don’t understand it at all. We’re not testing users. We’re testing the usability of the design.
Language matters. We construct reality out of language. Language shapes how we see the worldâ€”it limits our perceptions, it tells us what kinds of things exist or don’t exist in the world. The power to name a thing lets us shape what that thing is for and who can use it.
So what we call usability testing matters. However, nothing convinces others like experts, so please do check out this blog post by Dana Chisnell from 2008:
Every now and then, I hear people say that “user testing” is a method that only generates qualitative data. There are two problems with this statement.
First, we are not testing users. We are testing the usability of the product in question. It is usability testing, not user testing. In an age where “user experience” has superseded “usability,” we may want an even better name for this method, but it is most definitely not “user testing.”
Second, usability testing is a perfectly good way to gather quantitative data. It just so happens to also be a very good way to get qualitative data.
When you conduct a usability test, you can quantify the number of errors that happen during tasks (and categorize them for even cooler measurements), completion time, whether or not participants completed the test, and you can collect task-level and test-level survey data.
Usability testing generally takes place with small sample sizes (on the order of 6 people or so), and this isn’t a barrier to descriptive statistics. You just have some big confidence intervals around your averages, but you can just acknowledge that in your reporting and decision making and get on with it.
I do understand thinking that quantitative data gathered from usability testing isn’t so helpful if you’re taking some kind of lean approach to product development. You’re moving quickly and treating usability testing as a more generative activity. Maybe it would be worthwhile to draw a distinction between these activities.
I came across a couple of articles recently exploring the connection between user experience and product management.
First, Phil Dahnke over at UserZoom, they talk about how UX is everyoneâ€™s responsibility, particularly product management, in Can you be a successful product manager without UX research? One paragraph in particular stands out to me:
A successful Product Manager will champion the user and strive for a shared vision amongst the UX, UI and design teams based on their customer and user research. Sometimes this means changing direction and deviating from previous plans based on user feedback. Simply relying on the UX team to know whatâ€™s best for the product users can result in efficient products users might not like.
Or, to put it another way, the concerns of a UX specialist are a subset of the product managerâ€™s.
Hereâ€™s another article: Is UX Part of Product Management, byÂ Adrienne at Brainmates. Rather unsurprisingly, Adrienne comes down on the side of â€œyes.”
These articles are in the same vein as Eric Riesâ€™ Lean Startup, which positions product management as the key role in understanding and acting on information about users. Or, to put it another way, to do the thing that the UX field has proposed as its main value proposition.
Itâ€™s food for thought about what sort of career trajectory you should take if youâ€™re in UX and very interested in taking on more responsibility for building products.
Iâ€™ve been pondering lately where there will even be formal UX roles in 10, 20, or 30 yearsâ€™ time.
Certainly, I donâ€™t think the concepts of caring about users and designing based on researching those users will go away. And I certainly hope Iâ€™m not jobless and starving in a few decades.
It feels like weâ€™re in the middle of a race to see who can own UX the most. Everyone wants to talk about the user experience (to the point where that term has even displaced the more accurate â€œuser interfaceâ€ in a lot of cases). Developers build user experiences, quality assurance people assure the quality of the user experience, product managers want to swallow the UX skillset whole and incorporate it into their own jobs.
Meanwhile, thereâ€™s a constant drumbeat of â€œyou donâ€™t need UX specialists.â€ All you need to do is read this list of 5 tips on how to conduct interviews, and youâ€™re instantly an expert on user research! What kind of fool worries about getting bad data from bad research? Any research is, after all, better than none. (Protip: No, it isnâ€™t)
All that said, though, I do want everyone to think about how their work affects users. Disseminating these skills is going to make life better for everyone. And when we live in a world where practically everybody has had at least a class on UX and concerned with getting real data about usersâ€¦ what do we need specialists for?
I suppose weâ€™d need specialists for highly specialized problems. In those cases, though, I canâ€™t imagine weâ€™d see the kind of â€œUX Unicornâ€ model, where we pretend that a UX expert can also be an expert at visual design and coding. High specialized problems will probably need people that can solve tricky research problems, or who spend all their time thinking about complex design problems. How many specialists could the world possibly need, though?
Iâ€™m less worried than curious. I suspect a lot of UX people will end up migrating to product management.
Letâ€™s start by getting right to the point: When someone is rejected from a job theyâ€™ve applied to, they deserve to know as soon as possible. If they ask for feedback, it behooves you go give it.
Iâ€™ve searched for a job often enough myself, and watched plenty of people do it for years at a time, and it is hard. Itâ€™s a hard, hard process and it takes all of your energy just to keep up your morale. And as hard as it is to be on the side of the table thatâ€™s trying to hire someone, itâ€™s far harder to be the one whoâ€™s looking for a job. After all, your livelihood is at stake.
Coming at it from that perspective, I really donâ€™t understand why companies simply stop talking to candidates rather than rejecting them, and why they refuse to offer feedback when requested. A little empathy is in order. And if not empathy, some appreciation for the possibility that one day you may be on the other side of the table. Wouldnâ€™t you want to know that you were rejected? Wouldnâ€™t you want to know what you did wrong or what skills you need to improve?
We got a copy of Baxter, Courage, and Caineâ€™s Understanding Your UsersÂ at work recently. I was one of the reviewers for this book, which was a super-interesting thing to do. Besides getting listed in the Acknowledgments (yay!), I also got a mention as suggested further reading for web analytics. When you start getting recommended by other books is how you know youâ€™ve arrived.
If you are interested in developing your skills and raising your profile in the UX community, you may have thought about giving talks at conferences. If youâ€™ve already tried submitting proposals to conferences, youâ€™re probably feeling pretty discouraged at this point. Well, I have a foolproof guide to getting your proposal accepted:
Be someone that conferences will invite to speak.
It turns out that filling out a conference proposal is, in fact, a foolâ€™s game. UX conferences get hundreds of proposals. The only guaranteed way to speak at conferences is to be one of the handful of UX celebrities that gets invited to speak at every conference.
Unfortunately, this option is not open to most people. Fortunately, there is a second foolproof way to get your talk accepted:
Be incredibly lucky.
The foremost thing to understand about the conference review process is that it all comes down to the opinions of the reviewers. These opinions are incredibly subjective and idiosyncratic. Some conferences actually share the reviewersâ€™ feedback with you (which is a great thing for them to do). Every time, Iâ€™ve found that reviewers will give opposite and mutually exclusive feedback. Iâ€™ve had the same proposal described as:
- Too detailed and not detailed enough
- Such common knowledge that no one would want to hear this talk and an important emergent topic that everyone would want to hear
- Too basic and too advanced
So, while I appreciate getting the feedback, I donâ€™t even know what to do with it. As it turns out, getting a talk accepted is just like playing a slot machine. You pull the lever and hope you get three cherries.
Some of the team at Michigan State Universityâ€™s Experience Architecture program wrote an article for User Experience Magazine and they do a great job of describing whatâ€™s so exciting about their program.
I agree completely with a program rooted in the humanities, for the reasons they described. Human values are necessarily the heart of what we do in user experience, and something that, while I donâ€™t believe the University of Michigan School of Information was against, it wasnâ€™t really something that entered the conversation.
Itâ€™s interesting to see programs that are created when itâ€™s taken as given that there is a user experience field, and it is a field where people find jobs. When my own school, UMSI, reorganized to incorporate Human-Computer Interaction, it was the early days of usability in industry, and I donâ€™t think there was as clear of a career path.
However, I do have reservations about efforts to push more people into the user experience field. Michigan is not a hot market for user experience (or any job, really). I worry that weâ€™re giving young people the skills they need to evacuate the state, or to engage in a frustrating job search.
But much more importantly, seeing an undergraduate program in UX spring up (and this is in addition to UMSI starting its own undergraduate program) makes me wonder about the future of our field. Does it hasten the day when there are no distinct â€œuser experienceâ€ job titles, and instead the skills are taken as a given in a variety of different jobs?