Case Study: Reconsidering Decisions
The primary product I work on at WebMD is a medical simulation called MedSims. The aim of MedSims is to educate physicians by allowing them to go through a simulated patient visit and providing clinical feedback on each of their decisions. Users run tests, diagnose the patient, and order drugs.
Note: You need a Medscape account to use Medsims, but it’s free to join!
Alternatively, you can check out the demo video.
One of the key moments within MedSims involves showing the user the decisions they missed or may want to reconsider. This is a critical point in the educational model, as it provides the user with the opportunity to receive guidance and make adjustments to the case voluntarily.
Our existing screen for this interaction was working, but there was room for improvement. As an organization we wanted to see more doctors capitalizing on this chance to adjust their diagnoses and orders.
I wanted to be sure to understand the user's perspective when approaching this section of the simulation, so I set up a number of usability testing sessions.
I moderated 10 sessions (which in hindsight was more than we needed!) and found that 8 out of 10 users were uncertain of what was being asked of them in this section. We interviewed the users to unpack a little more, and realized a few things:
The sheer volume of information in the section was overwhelming
Users could not easily tell the difference between guidance for a decision they made and guidance for a decision they omitted
The transition from the prior sections where users work on the case to a section based around reviewing was a bit abrupt.
Research + Brainstorm
As I started to put together potential solutions, I started to think through what parallels the review section might have in another application. Looking at the problem from an abstract view, the MedSims simulation could be viewed as a multi-step form that includes a final check before submitting.
I found that in Turbotax, there was a similar interaction where the system provides final checks before you submit your tax return. An intermittent screen in shown to delineate the moment of the system review, and then you’re shown the items that need to be addressed. I thought we could leverage a similar flow for MedSims.
I added a modal to interrupt the flow of the simulation and indicate the moment between the main simulation and the review.
For clarity, I divided the section into two buckets - decisions the user omitted and decisions the user made - since both might require action.
This change was manageable enough for engineering to implement it quickly on a branch. With this in place, we set up a quick A/B test on an imminent case to see which version of the review section would perform better.
We saw a significant uptick in physicians choosing to act within the review section with the new design - enough so that the branch was merged and pushed to production the next sprint. We also validated this with later user testing, where the feelings of being overwhelmed or uncertain were no longer present.