The following is a letter I sent in to User Experience Magazine after a recent article on Visual Analytics. Sadly, I don’t think they publish letters to the editor anymore.
I am writing to offer feedback on Bartosz Mozyrko’s recent article, “Visual Analytics: Uncovering the Why in Your Data.† I am pleased to see the magazine covering data analysis topics.  As the author of Practical Web Analytics for User Experience, I also have a great interest in the subject. I’d like to offer a few comments and concerns.
Mozyrko argues that visualizing web analytics data answers the question of “why†users do what they do, in a way that either plain numbers and/or page-to-page path data do not. I must disagree: Neither kind of data actually reveal the “why†of user behavior. Rather, things like heatmaps and session recordings are just representations of plain, why-less data.
Mozyrko starts by describing how traditional web analytics tools like Google Analytics are insufficient for the task of answering “why,†which is accurate. However, he then argues that visualizations help people digest large amounts of data. While true, understanding large amounts of data still doesn’t solve the problem of “why,†as Mozyrko suggests.
Later, he states that visual analytics focus on what happens within pages, whereas traditional web analytics tools focus on how people move between pages. This statement is an oversimplification. Robust tools like Google Analytics and Adobe Analytics capture a rich amount of data on in-page and between-page behavior. In fact, having these two kinds of data together in one tool offers amazing opportunities for analysis, such as the ability to segment data about in-page behavior based on something the user did elsewhere during their session.
The rest of the article discusses types of visualizations. However, click tracking heatmaps, session replays, and form analyses still do not tell you why users did what they did. Mozyrko never answers the question of how to understand the “why.†He writes “armed with your visual analytics, you now have a complete picture of how users engage with your website or mobile application.†However, the picture is not complete—a complete picture would contain the answer to “why†and Mozyrko does not discuss how to answer that question.
A complete picture of user behavior comes from triangulating different kinds of data. Web analytics data work best when combined with data from things like surveys, usability testing, interviews, field studies, and so on. Interaction with users and putting together quantitative and qualitative data sources are the ways that we get closer to answers the “why†question.
Lastly, Mozyrko has a link in his article to a case study on his company’s website. He is the CEO of a company that makes the kind of tools he discusses in this article. While it makes sense that he would be highly knowledgeable about data visualization tools, this connection along with the inaccuracies of this article make it seem more like a piece of advertising than an article that seriously engages with the question of how to better understand user behavior.
I remain committed to maintaining the highest possible ethical standards for UXPA, and felt obligated to share my concern. Â Thank you for your attention, and for your efforts managing this vital publication.
Michael Beasley