Today, let’s dive into transactional user experience monitoring. First, I want to start with a bit of philosophy. The reason we want our systems to be reliable is to provide value to our businesses. Business value means the site remains available and performant for your customers. As long as we continue to maintain a quality user experience, do you really need a chatty alert every time CPU exceeds 95%? That way lies madness – well, at least, alert fatigue. Managing all abnormal events as priority alerts effectively means that nothing is treated as an urgent problem. We should alert on bad user experience and alert on conditions which we know will lead to bad user experience. Everything else gets shunted to a backlog that gets prioritized and addressed like regular, managed work – not as urgent PagerDuty notifications at 2am.
In my last blog, I began discussing two broad categories of user experience, transactional and journey. To recap, transactional user experience refers to experience that a user receives when interacting with a system on a step-by-step (click-by-click or touch-by-touch) basis. Typically, the metrics that matter in transactional user experience are response time as experienced by the user and system errors that the user encountered. In contrast, journey experience is about the overall flow that a user has when she first touches the site through to achieving some business result. Today, I’ll focus on transactional user experience.
We can monitor user transactions through two approaches, synthetic and real user monitoring (RUM). A synthetic monitoring tool fires requests to the system and compares the results to expectations. For example, we might script a request for the catalog page and we might expect to see at least one <div> element with a particular CSS class. Real user monitoring, on the other hand, uses a variety of different technical approaches to inject an agent into the user’s client. Once injected, the agent reports response time and error information on each user interaction.
Real user monitoring is significantly more complete than a synthetic approach since RUM endeavors to captures every real user interaction while synthetics are only cover a subset of the actions that are possible on your site. But, synthetics have a few advantages of their own. First, synthetics can run 24 hours/day to alert you of problems even if there are no real users on your system. Second, and perhaps more importantly, synthetics require zero application environment changes whereas injecting a RUM agent involves changing the application container, therefore, synthetics can be easier to set up. If you’re just getting started with user experience monitoring, start with synthetics.
In a future post, I’ll talk about my experience incorporating user experience monitoring as part of an incident management workflow and how it can serve as the cornerstone for effectively managing incidents.
What are you doing to monitor user experience? Are you using synthetics? What about real user monitoring? Did you find it easier to get RUM going in your organization compared to the synthetic tool?