Most EMR reviews fixate on the note. That is understandable, since documentation is where clinicians feel the most friction, but it misses the two workflows that touch every encounter and carry the highest safety stakes: entering orders and reviewing results. A system that makes a lab order take eleven clicks, or that buries a critical potassium in a queue with two hundred normal results, will wear a practice down long after the note templates have been tuned. This protocol gives you a repeatable way to review those two workflows, whether you are evaluating a new system, comparing two finalists, or auditing the one you already own.
Why order entry and results are the right place to look
Order entry and results review are where the EMR stops being a typewriter and starts being a clinical system. They involve decision support, interfaces to labs and imaging, routing between team members, and patient communication. They are also where usability problems translate directly into safety events: wrong-patient orders, duplicate orders, missed abnormal results, and delayed follow-up. The SAFER Guides published by the Office of the National Coordinator devote entire self-assessments to test results reporting and computerized provider order entry for exactly this reason. Reviewing these workflows tells you more about how a system will perform in daily practice than any feature checklist.
Setting up a fair review
A review is only useful if it is comparable across systems and across reviewers. Establish the ground rules before anyone touches a keyboard.
- Use a realistic test patient with a full chart: active problems, a medication list with at least one interaction risk, an allergy, prior labs with a trend, and a pending referral.
- Recruit three to five reviewers with different roles: a physician or advanced practice clinician, a nurse or medical assistant, and someone from the front or back office who handles results routing.
- Give every reviewer the same scripted scenarios and the same rubric, and have them work independently before comparing notes.
- Record the session or count clicks and screen changes with a simple tally sheet. Time each scenario.
- Review the system as configured for your specialty, not the vendor's polished demo environment. If the vendor will not let you into a realistic configuration, note that as a finding.
Order entry scenarios to script
Script scenarios that reflect what your clinicians do dozens of times a day, and include a few that are designed to trip the system.
- Routine lab panel. Order a comprehensive metabolic panel and a lipid panel for a follow-up visit, associate the correct diagnoses, and route to the patient's preferred lab. Count clicks, note whether diagnosis association is automatic or manual, and check whether the order set remembers the preferred lab.
- Order set with modification. Use a diabetes follow-up order set, then remove one item and add a microalbumin. Does modification break the set or require starting over?
- Medication with an interaction. Prescribe a drug that interacts with a current medication. Note whether the alert is specific, whether it fires at the right moment, whether it can be overridden with a reason, and whether the override is recorded.
- Duplicate order. Attempt to order a test that was completed three days ago. The system should surface the prior result and ask whether to proceed.
- Imaging with appropriate-use criteria. Order advanced imaging and see whether the system supports the consultation and captures the required information without leaving the ordering screen.
- Wrong-patient guard. Open two patient charts, then attempt to order in the second. Does the system display the patient banner clearly, and does it require any confirmation of identity?
- Pended and signed by someone else. Have a nurse pend an order under protocol and the clinician sign it. Evaluate how the queue presents pended orders and how many steps signing takes.
Results review scenarios to script
- Inbox triage. Load the results inbox with a mix of normal, abnormal, and critical results. Can the reviewer see severity at a glance? Can they sort and filter without opening each result?
- Trending. Open a hemoglobin A1c result and view the last five values as a graph and as a table. Note how many steps it takes and whether the trend view is available from within the result.
- Acknowledge and act. Acknowledge a result, add a comment, and route it to a nurse with an instruction to call the patient. Then check whether the routed task is visible to the nurse and whether the clinician can see it was completed.
- Patient release. Release the result to the patient portal with a note. Check whether the system supports the immediate release expected under the information blocking rules and whether there is a way to attach context.
- Closing the loop on an outstanding order. Find every order placed more than thirty days ago that has no result. This is the most important safety scenario, and many systems make it surprisingly hard.
- Covering provider. Have a second clinician cover the first clinician's inbox. How is coverage assigned, and are covered results distinguishable from the covering clinician's own?
The scoring rubric
Score each scenario on five dimensions, each from 1 (poor) to 5 (excellent), and record the raw click count and time alongside the scores.
| Dimension | What a 5 looks like | What a 1 looks like |
|---|---|---|
| Efficiency | Task completed in the fewest reasonable steps with no re-entry of known data | Repeated data entry, many screens, no defaults |
| Clarity | Patient, context, and status are always visible; no ambiguity about what will happen | Patient identity or order status unclear at any point |
| Safety support | Alerts are specific, well timed, and overridable with a recorded reason | Alerts absent, generic, or so frequent they are ignored |
| Team workflow | Routing, delegation, and coverage are natural and visible to all roles | Work disappears into personal queues; no visibility for others |
| Recoverability | Errors can be caught and corrected before signing, and after signing with a clear audit trail | No undo; corrections require vendor help or leave no trail |
Average the scores per scenario, then per workflow, and keep the raw click and time data as a tiebreaker. Do not let a single spectacular feature pull a low-scoring workflow up; the daily grind is what your staff will live with.
Failure patterns we see repeatedly
A few problems appear across many systems. Diagnosis association is manual and repetitive, so clinicians pick the first code that fits rather than the right one. Interaction alerts fire at the end of the prescribing flow, after the clinician has committed, instead of when the drug is selected. The results inbox treats every item the same, so critical values do not stand out. Outstanding-order reports exist but require a report writer to run. And coverage arrangements are handled by sharing passwords because the system's delegation model is too rigid, which is both a safety and a security failure.
Ask the vendor to fix, not to explain. When a scenario scores poorly, ask what configuration would improve it and whether that configuration is available to a practice your size. Some problems are settings; some are architecture.
Turning scores into a decision
Present the results as a one-page grid: scenarios down the side, systems across the top, average score in each cell, with click counts and times in a second grid. Add a short narrative of the three findings most likely to affect daily practice, and a list of configuration changes the vendor committed to. If you are reviewing your current system, the same output becomes an optimization backlog with the vendor and a training plan for staff.
The value of a protocol is that you can run it again. Repeat the review a year after go-live and compare. Systems and configurations drift, and a protocol you already own is the cheapest way to notice.
Common questions
How long does this review protocol take to run?
Plan on about two hours per reviewer per system: thirty minutes of orientation, roughly an hour of scripted scenarios, and thirty minutes to score and comment. Comparing notes across reviewers adds another hour.
Should we let the vendor drive the demo for these scenarios?
No. The reviewers should have hands on the keyboard. A vendor-driven demo hides click counts and hesitation points, which are exactly what you are trying to measure.
What if our EMR cannot produce a report of outstanding orders with no result?
Treat it as a significant safety finding. Ask the vendor for a workaround such as a saved report or worklist, document it, and consider a manual reconciliation process until the gap is closed.
Can this protocol be used on our existing system rather than for a purchase?
Yes. Running it on your live system, with a test patient, produces an optimization backlog and a training plan, and it establishes a baseline you can measure against after changes.