We set out to improve the page people see when they find an article in a search engine, land on JSTOR, and then find out they don’t have access to the article because they haven’t logged in yet.
The original design that served as the first control.
To investigate this problem, I gathered data from previous user research on the page and installed CrazyEgg to measure what people actually click on with the current design, including parts of the page that weren’t active features. This low-cost method gave us ideas about possible problems, including the theory that a disabled button was distracting people from the task of getting access.
Heatmap data for the original page.
First Round of Testing
I advocated A/B testing as the approach to improving this page. Although it seemed more promising to replace the disabled button with a useful call to action, we were able to set up an immediate test where we simply removed the disabled button entirely.
The results of this test were mostly positive—the conversion rates for directly purchasing articles and using the free reading program increased 10.63% and 2.15% respectively. However, people using their university login decreased 16.59%. We interpreted these results to mean that we were on the right path.
Second Round of Testing
For our next test, we actually replaced the disabled button with useful calls to action. The rationale for this design was that we were not only removing the distracting control, but using that prime real estate to give users options that would actually be helpful.
For this test, ecommerce conversion rate decreased 9.57% but the free reading program improved 25.18%. Meanwhile, conversion rate for people getting to their university access improved 1.40%. While this improvement was small, it affected a vastly greater number of people than the other access methods. Based on these findings, we chose to operationalize this new design because the outcome lined up with my team’s goals.
The final design, for now.