From broad complaint to a testable problem

From broad complaint to a testable problem

A broad complaint rarely gives a team enough direction to test a cause. Define what counts as the problem, over what period and out of how many opportunities. Break comparable observations down by useful factors such as request class, location or shift. Distinguish large counts from high rates. Then observe where the concentrated gap occurs and choose an analysis method suited to its mechanism. A ranking can guide attention but cannot establish causation. Keep reopened work and exceptions within the declared boundary so that an apparent improvement does not simply move the problem out of the measure.

Follow the method

1. Team A
2. Team B
3. Before interpreting

Keep volume, error rate and next question together

Fictional request-processing review: team A reports 20 errors in 1,000 cases; team B reports 10 in 100. Manager Elif initially assigns all improvement attention to A because its error count is larger. The shared error definition and case mix still need checking.

Follow the method

1. A error-containing cases
2. B error-containing cases
3. Combined
4. Next breakdown

The smaller error count hides the larger rate

Convert counts into comparable rates, define a focused problem and choose the next breakdown without treating a difference as proof of cause.

Fictional request-processing review: team A reports 20 errors in 1,000 cases; team B reports 10 in 100. Manager Elif initially assigns all improvement attention to A because its error count is larger. The shared error definition and case mix still need checking.

Role: Process analyst with both team representatives.

Normal condition

Counts and denominators refer to the same period and definition; comparisons state their scope.

The gap

Raw counts suggest one priority while rates suggest another.

• Treat each count as cases with at least one defined error for this exercise, not unrestricted defects per case.
• Different case mix, detection or periods can invalidate a direct team comparison.

1. Define what is being counted

   Elif confirms that numerator and denominator refer to completed cases in the same week and asks whether both teams apply the same error rule.

   Why: Twenty defects and ten defective cases are not comparable numerators.

   Evidence: A written definition and period are attached before interpreting.

2. Normalize the comparison

   Calculate A as 20 / 1,000 × 100 = 2% and B as 10 / 100 × 100 = 10%. Keep counts alongside rates.

   Why: The denominator exposes different opportunity volume, while the counts show the size of the observed burden.

   Evidence: Rate bars use one percent scale and retain both sample sizes.

3. Break down toward the actual work

   Request routine/exception case counts and error counts for each team, then examine where and when the specific error occurs. Avoid collecting every available category without a question.

   Why: A useful stratum narrows the gap or tests a comparability concern; it does not simply make another chart.

   Evidence: The next collection plan names case type, event definition and handoff point.

4. Write a focused, provisional problem

   State that B has 10 error-containing cases among 100 in the supplied week, with case mix and detection comparability unresolved. Assign the next observation before choosing a cause.

   Why: A higher observed rate does not establish worse effort or a causal mechanism.

   Evidence: Problem statement includes denominator, time, uncertainty and next owner.

The apparent team effect is mixed with case type

Further fictional records show B handles mainly exceptions and A mainly routine cases, but type-specific denominators have not yet been reconciled.

Do not rank team capability from the pooled rates. Obtain comparable within-type counts and examine the work conditions.

Case mix can change aggregate rates even when a named team is not the cause.

Retain original totals and reconcile each stratum back to them before interpretation.

Run a practice session

Materials

• Count/denominator cards
• Blank comparison sheet
• Calculator

1. Show counts before denominators · 4 minutes

   What changes when volume is revealed?

2. Calculate and reconcile · 7 minutes

   Why is the combined rate not 6%?

3. Practice equal-rate case · 8 minutes

   What claim can you support?

4. Choose next breakdown · 5 minutes

   Which question will the new category answer?

Debrief

• Check definitions before correcting arithmetic.
• Ask what would make a comparison unfair.

Calculate the combined rate from summed counts, then write one cautious problem statement.

Transfer into the work

Owner: Process analyst and representatives who apply the event definition

Record: Stratified counts/rates with data definitions

Review: After collection and before causal analysis or prioritization

Evidence: Reconciled denominators, comparable scope and point-of-occurrence observations

Repair definitions or collect missing denominators rather than rank people from incomplete counts.