Category Archives: UX Profession

Visual Analytics

The following is a letter I sent in to User Experience Magazine after a recent article on Visual Analytics. Sadly, I don’t think they publish letters to the editor anymore.

I am writing to offer feedback on Bartosz Mozyrko’s recent article, “Visual Analytics: Uncovering the Why in Your Data.”  I am pleased to see the magazine covering data analysis topics.  As the author of Practical Web Analytics for User Experience, I also have a great interest in the subject. I’d like to offer a few comments and concerns.

Mozyrko argues that visualizing web analytics data answers the question of “why” users do what they do, in a way that either plain numbers and/or page-to-page path data do not. I must disagree: Neither kind of data actually reveal the “why” of user behavior. Rather, things like heatmaps and session recordings are just representations of plain, why-less data.

Mozyrko starts by describing how traditional web analytics tools like Google Analytics are insufficient for the task of answering “why,” which is accurate. However, he then argues that visualizations help people digest large amounts of data. While true, understanding large amounts of data still doesn’t solve the problem of “why,” as Mozyrko suggests.

Later, he states that visual analytics focus on what happens within pages, whereas traditional web analytics tools focus on how people move between pages. This statement is an oversimplification. Robust tools like Google Analytics and Adobe Analytics capture a rich amount of data on in-page and between-page behavior. In fact, having these two kinds of data together in one tool offers amazing opportunities for analysis, such as the ability to segment data about in-page behavior based on something the user did elsewhere during their session.

The rest of the article discusses types of visualizations. However, click tracking heatmaps, session replays, and form analyses still do not tell you why users did what they did. Mozyrko never answers the question of how to understand the “why.” He writes “armed with your visual analytics, you now have a complete picture of how users engage with your website or mobile application.” However, the picture is not complete—a complete picture would contain the answer to “why” and Mozyrko does not discuss how to answer that question.

A complete picture of user behavior comes from triangulating different kinds of data. Web analytics data work best when combined with data from things like surveys, usability testing, interviews, field studies, and so on. Interaction with users and putting together quantitative and qualitative data sources are the ways that we get closer to answers the “why” question.

Lastly, Mozyrko has a link in his article to a case study on his company’s website. He is the CEO of a company that makes the kind of tools he discusses in this article. While it makes sense that he would be highly knowledgeable about data visualization tools, this connection along with the inaccuracies of this article make it seem more like a piece of advertising than an article that seriously engages with the question of how to better understand user behavior.

I remain committed to maintaining the highest possible ethical standards for UXPA, and felt obligated to share my concern.  Thank you for your attention, and for your efforts managing this vital publication.

Michael Beasley

Ignite UX Michigan 2015 was Great!

Ignite UX Michigan is over for another year. We had this year’s event on Tuesday at Live in Ann Arbor. We counted over 160 people this year – more than the official count for last year (140). Every year, I have this feeling like we’ve managed to catch lightning in a bottle. It’s amazing to see such a crowd, and all these excellent speakers.

And after a good night’s sleep, it’s time to get up and get back to work – there is a whole bunch of wrap-up to do after this year’s event, and then we have a short turn-around before 2016. We’re permanently moving the event to March, starting with the next event, which means there isn’t a lot of time to rest up. It’ll be all right. It’ll be better than all right, in fact! I’m really looking forward to what’s next.

Accessibility Summit 2015

I went to the Environments for Humans Accessibility Summit 2015 this week. For the sixth year in a row, the University of Michigan hosted a screening of this virtual conference, giving a bunch of people from the community the chance to see it for free.
The best part is the good company – it’s always good to reconnect with people!
Other than that, the first day was all right. Outside of the talk on integrating accessibility with agile, the talks weren’t squarely in my realm on interest. What really grabbed me about the agile talk was that ultimately, it’s all just people figuring out better processes. Which really is in keeping with the spirit of agile.
On the second day, the talks looked a lot more like they were to my tastes. Whitney Quesenbery started off the day with a talk on usability testing with people with disabilities. However, that was followed by another talk covering similar ground. And though I was looking forward to the talk on accessibility tools and unpacking WCAG, neither really got into the kind of material I wanted to hear about.
All in all, I’m kind of worried that what I need at this point is material that goes into more depth than you can really cover in an hour.
If I could offer one bit of constructive feedback, it would be to encourage speakers to practice more. Some of them were very disorganized, and when that’s combined with slides that aren’t well-organized either, and all that’s combined with being a virtual conference, it makes it really hard to follow a talk. That happened a few times.

“What to Build” vs. “How to Build It”

I’ve been thinking lately about how UX is usually brought in to answer “how will we build it?” questions, but not “what will we build?” questions.

“What will we build?” questions deal with the matter of finding out what users’ needs are and connecting those needs to something the business can actually do. Getting the right answer to this question means that the organization doesn’t spend a bunch of time building the wrong thing—building something that, no matter how well designed, doesn’t meet anyone’s needs. The user experience field offers some great tools for learning about users and answering the question of “what will we build?”

Of course, this question is usually answered long before UX is involved.

Instead, we’re usually involved at the point of “how will we build it?” This question assumes the matter of what to build is already settled, and instead all that’s left to do is figure out what the design should be. UX can offer value here, obviously. The problem is that we’re usually pigeonholed into answering only that question, even when it turns out that we’re working on a feature that should never be built because the design problem was framed wrong from the beginning.

The effort to push UX earlier into the design process is an effort to get closer to the “what will we build?” question, but part of the problem we face is that there are already people who “own” that question, and our effort to provide more value can look like a challenge to them.

I’m not sure what to do about this, yet, though. Just pondering it.

Keurig: Bad, Wasteful, Unethical Design

Shortly after I started working at my current job, we replaced the old school drip coffee machines with Keurig machines. For those unfamiliar with them, Keurig makes coffee machines that let you insert a little sealed plastic cup into the machine, press a couple of buttons, and have coffee emerge. Cleanup is as simple as throwing a little gob of plastic into the trash.

The Keurig people have built the skill of making a cup of coffee into the device itself, making it easy for people to get consistent results. Not good results, mind you, but consistent results, which is something that we know people care about. Plus, you get coffee that seems fresh, made just for you! And you get to exert choice over what kind of coffee you put into the machine. You get custom made coffee and the ability to choose (within the constraints of the system, of course). Just make sure you only buy Keurig brand coffee cups. They frown upon you using off-brand coffee.

Of course, Keurig, besides making terrible coffee, is super wasteful. Granted, you waste less coffee using this system. Generally speaking, people have no idea how much coffee to use when they’re brewing it (this is the skill that Keurig builds into the artifact). But you are using a bit of plastic every time you make a cup of coffee and tossing it in the garbage afterward (these cups can’t even be recycled). Keurig coffee is not exactly an ethical choice.

Should a UX person participate in making a product like the Keurig machine? Should we make products that devour natural resources and create unnecessary waste, and terrible coffee to boot? In a perfect world: No. We don’t live in a perfect world, though, and living in society means choosing which compromises we’re going to make.

The discussion of sustainability in UX is thought-provoking but, I would say, not really relevant to most people’s lives. UX people don’t often get to participate in discussions of what to build; we’re usually brought in to help figure out how to build it. Without the ability to influence what gets built, we’re left with two choices: Participate or starve.

So while I’d love to live in a world where Keurig doesn’t exist, I wouldn’t blame any UX person that was involved in building it. If it’s any consolation, though, the design of the Keurig machine is so bad that I can only conclude that there isn’t.

Improving the Ignite UX Michigan Speaker Selection Process

For the third year in a row, we’ve made it through the speaker selection process for Ignite UX Michigan. As before, it’s kind of a heartbreaking process. Having been rejected from every conference I’ve pitched a talk to for the last 2 years, and having been accepted by two out of dozens, I can sympathize with the people that we can’t fit into our lineup. Being involved in the selection process also really drives home how good all of these proposals are.

We’ve used the same basic selection process three times. It’s designed to be as fair as possible, but the concept of fairness, unfortunately, depends a lot on how you define it. Lately, I’ve spent a lot of time thinking about how we can make it more fair next time, and how to further the goals of the event.

What Are We Trying to Accomplish?

The two highest priorities for Ignite UX Michigan are:

  • Being a free event
  • Having diverse speakers

The speaker selection process doesn’t have a lot to do with the first priority, freeness, but it obviously is highly relevant to the goal of speaker diversity. Diversity pretty much has the dimensions that you’d expect—gender and race. Closely related and of great importance to me is diversity in terms of “do you see this person speaking at conferences all the time?” The UX field sure has an in crowd. On the plus side, if you’re disappointed to miss one of the headline speakers at a conference, you can just catch them at another conference later on in the year. You always see the same people speaking. On the minus side there may be some other voices in our field that would be really good to hear from.

Of course, there are more aspects of diversity than what I’ve just discussed, and they’re important ones, too. Something dear to my heart that we can’t address in Ignite UX Michigan is class diversity

The scariest part of taking a stand and saying that diversity is important is that, once you start worrying about it, you see the ways that you can and do fail. The work is ongoing. I’d love to talk about what we can do to get better.

The Current Selection Process

Well, how can our process let us down when it comes to diversity? To get people to propose talks, we make public announcements about the submission process, and we try to encourage specific people to propose talks, but it’s obviously very likely that there are plenty of people that just never get the message in time. That’s already a limiting factor—we’re more likely to get proposals from people that are plugged into the community. To a large extent, we get what we get, and the most practical idea I can think of for increasing diversity at this stage would be to just keep engaging with the community to try to spread the word about our event.

Then, we feed all those proposals into the selection process. I would characterize the process that we have used for the past three years as not specifically promoting or detracting from diversity. The process is:

  1. The 12 highest-rated talks are selected, with the following adjustments:
  2. We average the ratings for each talk
  3. The reviewers fill out the survey. For each proposal, they rate it from 1 to 10, with 10 being the strongest vote in favor of a talk. Each proposal is rated separately (as opposed to ranked against each other). The reviewers are volunteers who do not know who proposed talks, insofar as trust people to be honorable about this.
  4. The facilitator creates a survey that lists the titles and descriptions of all of the talks. This survey doesn’t include the names of the people that proposed talks and, in a perfect world, has no way of identifying who submitted the proposal
    1. Although people can submit more than one proposal, we only take one talk per person
    2. In 2014, a talk I proposed was selected, but I didn’t want to take the space of another speaker, so we chose the top 13 instead
    3. In cases of a tie, the proposal with the lowest variance wins

Why choose this approach? Well, we want a diverse set of speakers, true, but we also don’t want to have a speaker selection process where we use our judgment to hand-pick our lineup. Using judgment risks us choosing who we know, and that’s how you end up with the same people talking at every conference. So we use the blind-review-with-math approach, and hope that 1) our pool of proposals is diverse to begin with and 2) that an impartial selection process will produce acceptable results.

Possible Improvements

I’m interested in improving the process, I’ve thought about it a lot, and I’m not sure exactly what the best approach would be. Some ideas that have come up are:

  • Apply weighting to proposals:
    • Like, if you spoke at the previous year’s event, your proposals get a not-insurmountable penalty—the logic being that good proposals will still get accepted, but it’ll be a bit less likely.
    • Apply a bonus to people who have never spoken at a professional event before (or never spoken at a national conference)
  • I am strongly reluctant to start asking people things about themselves, like gender, but we could:
    • Set aside a certain number of speaking slots for students
    • Have people classify their talk into one of X categories, and then have a certain number of slots for each category
    • Of course, neither of these things necessarily help with diversity, but the student thing is perhaps promising
  • Instead of using low variance as a tie-breaker, instead use high variance—this means that we’d favor the talks that people disagreed on
    • Along those lines, we could fundamentally reimagine the review process so that diverging opinions is a primary factor

This is tricky but important stuff. When we started Ignite UX Michigan, we didn’t even know if people would submit talks. Our process was something we came up with after we got more proposals than room for speakers and an attempt to come up with something relatively fair. How can the selection process promote the goal of diversity? I’d love to get more ideas and, perhaps even more importantly, find people that are interested in helping us get better.

The Myth of the Shortage of UX People

There’s this pernicious idea out there that there’s a glut of UX job openings and a shortage of UX people to fill those roles. To pick just one article on the subject, let’s take a look at UXmatters’ “Why Is It So Hard to Find Good UX People“. It has such gems as:

“It is so hard to find good UX people because they’re scarce!” exclaims Tobias. “There just aren’t a lot of people out there who are actually UX people. There are not that many people coming out of the right college programs—for example, CMU, Bentley, or Clemson—and those few get hired right away.”

Or about about this:

“The first problem that I’ve observed is with the supply to demand ratio: there is so much UX work out there that those with UX experience get snapped up very quickly,” replies Cory. “Because of the supply and demand issue, I’ve seen employers who end up lowering their required number of years of experience. It’s simply too hard to find UX professionals who are super experienced. However, employers do not always recalibrate their expectations to match a UX person’s years of experience.”

I fully agree that there is frequently a mismatch between what employers are looking for (or think they want) and what they can realistically get, but that’s not exactly a UX-specific problem. What gets me is this idea of a “supply and demand issue.” I’d love to see some numbers on this, because from where I’m sitting, there isn’t any such thing.

Instead, I see an environment where UX people looking for full time work have to scour the landscape, looking for those openings. I see recruiters and hiring managers so swamped with applicants that they don’t even have time to reject the ones that they interview. And if you’re going to treat people in the community with such disrespect, it’s clear that you’re not really concerned about your ability to find more UX people. I see that searching for job openings in a single area produces just a couple of jobs at any given time, and the chances of one of those jobs being full time is slim.

Maybe this is a Michigan thing, and in other places there really is a shortage of UX people. I guess if that’s the case, I would encourage some of these companies that are hurting for UX people to come check out Michigan.

ConveyUX in February 2016

It looks like I’ll be going to Seattle in February 2016!

I’ll be teaching a workshop on Web Analytics for User Experience at ConveyUX. It’ll be an iteration on what I taught at UX Lisbon. Who knows what an extra year of thought will produce?

I really value these opportunities to share what I know about the topic of using web analytics for user research. It’s an important tool for us to have in our toolkits, but it’s outside the comfort zone for a lot of folks in the user experience field. However, it’s nothing we can’t handle!

This will be my first trip to Seattle, so that’s a bonus. I’ve often wondered if the Pacific Northwest is for me, so this trip will help answer that question.

Thoughts on UX Lisbon 2015

A few weeks ago, I went to Portugal for the first time to attend UX Lisbon. It was a good conference, and not just because I gave two workshops there!

The conference had three days—the first and the third consisted solely of half day workshops, and the second day was a continuous series of talks. I gave my analytics workshop on the first and third day. It was exhausting but worth it. You get things out of teaching that you just don’t get any other way. I had the chance to do a lot of thinking about how to present the material and that’s going to pay off soon.

The lineup of speakers on the second day was great. I find that lately, I want to see short talks that will provoke thought rather than practically useful talks, because there just isn’t that much room to impart practical information in 30-60 minute timeslots. Lisa Welchman and Josh Seiden kicked off the day with two solid talks. I wasn’t familiar with Lisa before the conference and information governance is at a nice intersection of being super interesting and super important. The downside of a talk like this, and other ones that call for us to engage in changing the world, is that they assume a level of power that you rarely see in the world of UX.

I had the chance to see a short talk by someone from Facebook, talking about how they managed the transition of forcing people to download Messenger, and how disabling a feature in the Facebook app was actually an improvement for users. I politely listened—there’s no sense in heckling someone when they’re giving a talk.

Why Ignite UX Michigan?

Why have Ignite UX Michigan? Because diversity is important.

When you look at UX conferences, you see a lot of the same people headlining, over and over. There are more people that are worth hearing from in our community, and we’re not going to hear from them unless we make an environment where it’s possible.

Making an environment with more voices means extending opportunities to speak to more people, obviously. While Ignite UX Michigan doesn’t have that far of a reach, what we can offer is an opportunity to gain experience speaking, to try out one’s ideas on an actual audience. I would love to see Ignite UX Michigan as an early stop on someone’s speaking career.