The following is a letter I sent in to User Experience Magazine after a recent article on Visual Analytics. Sadly, I don’t think they publish letters to the editor anymore.
I am writing to offer feedback on Bartosz Mozyrko’s recent article, “Visual Analytics: Uncovering the Why in Your Data.” I am pleased to see the magazine covering data analysis topics. As the author of Practical Web Analytics for User Experience, I also have a great interest in the subject. I’d like to offer a few comments and concerns.
Mozyrko argues that visualizing web analytics data answers the question of “why” users do what they do, in a way that either plain numbers and/or page-to-page path data do not. I must disagree: Neither kind of data actually reveal the “why” of user behavior. Rather, things like heatmaps and session recordings are just representations of plain, why-less data.
Mozyrko starts by describing how traditional web analytics tools like Google Analytics are insufficient for the task of answering “why,” which is accurate. However, he then argues that visualizations help people digest large amounts of data. While true, understanding large amounts of data still doesn’t solve the problem of “why,” as Mozyrko suggests.
Later, he states that visual analytics focus on what happens within pages, whereas traditional web analytics tools focus on how people move between pages. This statement is an oversimplification. Robust tools like Google Analytics and Adobe Analytics capture a rich amount of data on in-page and between-page behavior. In fact, having these two kinds of data together in one tool offers amazing opportunities for analysis, such as the ability to segment data about in-page behavior based on something the user did elsewhere during their session.
The rest of the article discusses types of visualizations. However, click tracking heatmaps, session replays, and form analyses still do not tell you why users did what they did. Mozyrko never answers the question of how to understand the “why.” He writes “armed with your visual analytics, you now have a complete picture of how users engage with your website or mobile application.” However, the picture is not complete—a complete picture would contain the answer to “why” and Mozyrko does not discuss how to answer that question.
A complete picture of user behavior comes from triangulating different kinds of data. Web analytics data work best when combined with data from things like surveys, usability testing, interviews, field studies, and so on. Interaction with users and putting together quantitative and qualitative data sources are the ways that we get closer to answers the “why” question.
Lastly, Mozyrko has a link in his article to a case study on his company’s website. He is the CEO of a company that makes the kind of tools he discusses in this article. While it makes sense that he would be highly knowledgeable about data visualization tools, this connection along with the inaccuracies of this article make it seem more like a piece of advertising than an article that seriously engages with the question of how to better understand user behavior.
I remain committed to maintaining the highest possible ethical standards for UXPA, and felt obligated to share my concern. Thank you for your attention, and for your efforts managing this vital publication.
Ignite UX Michigan is over for another year. We had this year’s event on Tuesday at Live in Ann Arbor. We counted over 160 people this year – more than the official count for last year (140). Every year, I have this feeling like we’ve managed to catch lightning in a bottle. It’s amazing to see such a crowd, and all these excellent speakers.
And after a good night’s sleep, it’s time to get up and get back to work – there is a whole bunch of wrap-up to do after this year’s event, and then we have a short turn-around before 2016. We’re permanently moving the event to March, starting with the next event, which means there isn’t a lot of time to rest up. It’ll be all right. It’ll be better than all right, in fact! I’m really looking forward to what’s next.
I went to the Environments for Humans Accessibility Summit 2015
this week. For the sixth year in a row, the University of Michigan hosted a screening of this virtual conference, giving a bunch of people from the community the chance to see it for free.
The best part is the good company – it’s always good to reconnect with people!
Other than that, the first day was all right. Outside of the talk on integrating accessibility with agile, the talks weren’t squarely in my realm on interest. What really grabbed me about the agile talk was that ultimately, it’s all just people figuring out better processes. Which really is in keeping with the spirit of agile.
On the second day, the talks looked a lot more like they were to my tastes. Whitney Quesenbery started off the day with a talk on usability testing with people with disabilities. However, that was followed by another talk covering similar ground. And though I was looking forward to the talk on accessibility tools and unpacking WCAG, neither really got into the kind of material I wanted to hear about.
All in all, I’m kind of worried that what I need at this point is material that goes into more depth than you can really cover in an hour.
If I could offer one bit of constructive feedback, it would be to encourage speakers to practice more. Some of them were very disorganized, and when that’s combined with slides that aren’t well-organized either, and all that’s combined with being a virtual conference, it makes it really hard to follow a talk. That happened a few times.
I’ve been thinking lately about how UX is usually brought in to answer “how will we build it?” questions, but not “what will we build?” questions.
“What will we build?” questions deal with the matter of finding out what users’ needs are and connecting those needs to something the business can actually do. Getting the right answer to this question means that the organization doesn’t spend a bunch of time building the wrong thing—building something that, no matter how well designed, doesn’t meet anyone’s needs. The user experience field offers some great tools for learning about users and answering the question of “what will we build?”
Of course, this question is usually answered long before UX is involved.
Instead, we’re usually involved at the point of “how will we build it?” This question assumes the matter of what to build is already settled, and instead all that’s left to do is figure out what the design should be. UX can offer value here, obviously. The problem is that we’re usually pigeonholed into answering only that question, even when it turns out that we’re working on a feature that should never be built because the design problem was framed wrong from the beginning.
The effort to push UX earlier into the design process is an effort to get closer to the “what will we build?” question, but part of the problem we face is that there are already people who “own” that question, and our effort to provide more value can look like a challenge to them.
I’m not sure what to do about this, yet, though. Just pondering it.