In a classic Dilbert cartoon, the “client” proudly claims to Dogbert the consultant that “every three months, an existing customer acquires another product.” Dogbert replies, “what if you don’t count warranty replacements?” Client: “Oh, then we don’t look so good.” This exchange is one example that can represent a real problem in how businesses define success. Other signals sound like this: “That’s the wrong process,” “That’s not what I meant,” or “That’s not how it’s done.”

What all these have in common is a lack of clear operational definitions. And just to demonstrate how serious this can be, consider feedback from a federal judge that was asked why it was taking so long to support President Obama’s pledge to close the prison at Guantanamo, a process intended to quickly triage dangerous inmates from those that could be deported:

“It would have helped if Congress had given us a definition [of an enemy combatant],” said Judge Royce Lamberth. “The Bush administration gave us four different definitions; the Obama administration gave us another definition; each of our courts is deciding for themselves the proper definition.”

More recently, a change in how one Chinese province tracked cases of coronavirus created a nine-fold increase in reported cases overnight.

The purpose of operational definitions is not only to ensure everyone is on the same page with how to measure performance, it identifies all elements necessary to produce reliable data. For example, the OR is the most expensive space in a hospital to operate—measuring productivity is important to financial viability. Do you start measuring the cycle time of procedures when the patient first enters, when the patient is on the operating table, or when the surgeon makes the “first cut?” Does a computer, clock, or watch measure the start and end time? These considerations matter and it should be obvious how the operational definition itself might give rise to unintended variation. One urban legend has it that a surgeon was known to nick a patient in the OR before getting himself a cup of coffee (can you guess how they measured start time?).

The challenge of writing an unambiguous definition can be made powerfully clear by giving each of two teams a box of animal crackers. Challenge each team to write a clear definition of a “good cookie,” and then have them count the number of passing units in the box assigned to them. Next, switch both the box and each team’s definition (keeping the number of passing units secret). Challenge each team to apply the definition they’ve inherited to the new box of crackers they received from the other team. I’ve run this exercise hundreds of times, and the percentage of teams arriving at the same result has consistently been in single digits.

Here is a protocol for establishing clear and unambiguous operational definitions:

• Identify the characteristic of interest. Are we concerned with counting positive (acceptable) or negative effects (defects)?
• Select the measurement instrument. Will this be a gage, or are we relying solely on visual inspection? If visual, how will you control for conditions so they are not affecting perception?
• Describe the test method. How is the measure to be taken, and to what degree? If we are looking at cycle time, when is the start and when is the finish? Are we looking for hours, minutes, or seconds?
• State the decision criteria. Under what conditions does the measurement represent pass or fail? Consider the benefit of images here to eradicate ambiguity.
• Test and then document the operational definition. The operational definition should make the task clear and easy to perform. The best way to test a definition is to ask different people to complete the test on several items by following it. Adapt your definition as necessary to eliminate any remaining ambiguity.

Counting animal crackers is a great place to start if you wish to challenge your staff to take the matter of writing clear definitions seriously, and will create an accepting environment to follow the protocol provided. True, a good animal cracker is not a matter of life or death, but what if that damaged animal cracker was one of your products, deemed acceptable by the definition, and shipped to a customer? What if that animal cracker with a missing right paw was a post-surgical patient?  This is not the time you want to be heard saying, “That’s not what I meant.”