We mess with Jenkins configs. We struggle with failing automated tests. We finally get Kubernetes doing what we want it to do. We wake up at 2am to restart a failing API gateway. Why? So the user gets the best experience possible. How do we know if our efforts are paying off?
Who is having a good experience? Who is having a bad experience? Under what conditions? What is a good user experience? What is a bad experience?
Before we can tackle these questions, we must first have basic data collection in place. We must be able to measure a user’s experience. So, what are the elements that make up the user’s experience. Broadly, I tend to think of measuring experience in a couple of different respects: transactional and journey. By “transactional,” I mean the click-by-click (touch-by-touch, swipe-by-swipe) operations that your users perform in order to navigate your website or app. The “journey” aspect of a user’s experience is how effectively your business delivers on the user’s desire to achieve some goal. For example, if you are Amazon, you want to focus on how happy the user is with each of the actions taken to get to a purchase. This might include the using search, reading reviews, the decision, going through the purchase and the receipt of the product.
The “journey” and “transactional” approaches each have their place. Typically, user experience and product management folks will be interested in the “journey” while developers and folks with more operations focus will be interested in “transactional” monitoring. Journey monitoring focuses on things like user feelings and effort at various points during the journey. On the other hand, transactional monitoring tends to focus on much more tangible metrics. This video is a bit dated but does a pretty good job of detailing the nooks and crannies of the tech in this space.
Normally, transactional monitoring can yield more urgent action. As SREs, we’re much more likely to leap out of bed for a bad purchase transaction experience than to improve the overall journey across the website.
I’ll talk about my experience working with user experience monitoring in upcoming posts. Do you monitor your user experience? Transactional? Journey? How do you do it and what tangible actions come from that monitoring?