Maturity Model part 8
Take a look around where you work. Pay attention to what is happening.
What are people doing right now?
Go ahead. I’ll wait.
What did you see? What did you notice? Of the things you noticed, how many will be formally measured?
Toby is filling his water bottle. Is that measured? Coleen is getting people together for morning tea. Are we measuring that?
What about the conversation while we waited for Duncan to join the meeting online? It connected work Betsy had started in communications with a campaign HR ran three years ago, and it ended up saving a whole heap of rework. And we cannot forget Amy, who saw a mess and cleaned it up before anyone else noticed. Just initiative: something needed doing, so she did it.
Essentially, what I am saying is that the old standby, “What gets measured gets done,” is one of those sayings that is both true and not entirely helpful. A bit like, “It is always in the last place you look.” Of course it is. I stop looking once I find something.Much more happens in workplaces than what gets formally measured—or, at least, more than what gets measured in the traditional sense.
Measurement matters. Because it matters, we need to understand it with more nuance than a five-word catchphrase allows.
What is measurement?
In the maturity model, we speak about measurement becoming part of systems at Level 4. This alone has caused many conversations, which is a sure sign that the idea needs a little more unpacking.
We could say that identifying a system’s maturity level is a form of measurement. We are looking at the current state of the system and making an assessment.
But that can create confusion because “measurement” now has two different meanings:
Assessing the current state and maturity of a system.
Gathering performance data to manage a system at Level 4.
These are related, but they are not the same thing.
Recognising that a system is ad hoc or reactive because success depends on who is in the seat when the call comes in is different from producing monthly financial profit-and-loss reports for an executive team. Both help us understand the current state. But the second is more formal, recurring, and designed to support active management.
I have found it helpful to borrow some thinking from developmental evaluation.
Measurement is one form of evaluation. From a maturity perspective, it becomes useful where there is enough rigour in the system to sustain the conditions for good performance measurement: a reasonably stable baseline, a clear reporting cycle, agreed measures, and governance that can act on the results. (Level 4 Maturity)
All Measurement is evaluation but not all evaluation is measurement.
Why Measure, or Evaluate? Outcomes
To understand why we measure—or why we might instead choose a different form of evaluation—we need to add more nuance to “What gets measured gets done.”
This time, we need to think more deeply about the “gets done” part.
Measurement for measurement’s sake is not the point. All forms of evaluation require time, attention, skill, and effort. That effort needs to contribute to something useful.
Evaluation is purposeful inquiry into the value, quality, effects, or performance of something. It should create learning that informs decisions, and decisions should inform action. Without that connection, reports can multiply until the decisions that need support are buried under so much information that the signal is lost in the noise.
A more useful formulation is:
What we actively evaluate creates learning. Learning informs decisions. Decisions lead to action. Action generates outcomes.
The evaluation is the start of the process not the outcome
So what?
Measures, evaluation, decisions, actions, outcomes—does any of this stuff matter?
“All I want to know is whether what I am doing is working. Can’t we just measure it?”
Well, I am sorry, but no. You cannot just measure it.
Measurement and evaluation are interventions. When we choose to evaluate something, we impact the system we are evaluating. That means how we evaluate matters.
If a contact centre measures people only on how quickly they end calls, staff may shorten conversations that need more care. If a team measures only the number of cases closed, it may favour the easy cases over the difficult cases that matter most.
The measure does not simply describe the work. It changes what people pay attention to.
This is why “just coming up with a measure” is a very good way to fall into a trap.
Measurement traps and snares
And “just measuring” is a fine way to get data, but data alone does not guarantee useful learning.
When I was working as a national lead for conservation volunteering, a study was commissioned to gather data on the types of people who volunteer for nature. The data showed a large number of older retired people. They were the majority of volunteers.
One team looking at the results concluded that there was not much to be done: people volunteer when they have spare time, and the only way to grow volunteering would be to wait for more people to retire.
But there was another way to read the same data.
We looked at the volunteers who were not retired and noticed a high proportion of PhDs and biologists: people who had spent time exploring wilderness and communities. We then looked for common factors across the different volunteer groups. What we saw was not simply available time. Volunteers often had confidence in themselves and valued community.
Retirees may have developed confidence through a lifetime of experience. PhDs and biologists had confidence in their knowledge and practice. Explorers knew their wild places. Community volunteers had confidence because they belonged to something bigger than themselves.
Looking at the data in this different way allowed us to design new pathways into volunteering—pathways that did not exist when we focused only on the largest group and its most obvious trait. We could support volunteers to build confidence and grow our volunteering base that way.
How we make our evaluations matter and how we translate the data from the evaluation matters too.
Now what?
We know that evaluation is powerful, and important, and we know that the power of evaluation can create unintended consequences, send us in a wide range of directions so how do we decide what enough evaluation looks like?
Evaluations of systems evolve as the systems mature, giving us some guidance.
What we look to evaluate
The aim is not to measure everything. It is to evaluate enough, in the right way, to help people make better decisions about what to do next.
