Maturity Matrices: A tool for DevX KPIs and OKRs (Part 2)

A DevX feature list will comprise of, at least, the following items:

• The list of customer-developer visible features of your product
• The list of non-feature support functionality exposed to customer-developers (e.g. observability tools, or logging)
• The list of processes, automated or manual, that a customer-developer may engage in (e.g. onboarding, support requests, feature enablement)

Broadly speaking a customer type identifies any developer with a distinct use case. For me this maps, or should map, 1:1 to a user profile. If you have user profiles you can probably just refer to each case by name - if your team are used to talking about "Twanna", "Antje" and "Rakesh", then it makes sense to use them here. As you'll see we won't surface this level of detail to anyone who wouldn't get those references.

If you don't have user profiles, then you might be able to use job titles (for internal customers) or some kind of "stub profile", for example "Python developer working on ML powered backends".

Remember, a maturity level is a quantative statement about the current level and desired quality of developer experience.

Unlike with the Capability Maturity Model I don't just do this as a simple, linear progression, because the cases differ too much. Instead, what I do is this (noting that these are examples, you need to think about your own rules):

1. I break down developer experiences into mulitple types. For me these are:

   • interface maturity
   • supporting services
   • documentation and supporting materials
2. I define a maturity level for each of these types as the sum of scores from a series of questions.

Each question I ask has an assigned value which represents its relative importance to our perception of a good DevX.

Defining all these metric values is a significant amount of work. If you have the time to do a "big bang" first round assesment it's a great way to start, but if you don't my suggestion would be to pretend you're starting from scrathc and iteratively build up the list as your work on specific areas. That's how ongoing maintentance of this data works too. Essentially before you're done with an item of work you visit the "maturity sticker" for each DevX feature.

I call it a "maturity sticker" because I think of it like the labels they print on products in shops, breaking down the nutrients, or grading the sustainability of the product. The "maturity sticker" is the most detailed breakdown of maturity, it represents the calculation of maturity for a unique combination of product, feature, customer/developer type, and maturity type.

Let's start looking at some realistic examples to try and pull this idea together. What follows might give you an idea of the volume of data required and how we aggregate it. In reality I drop all of the "N/A" questions to keep things terse, but I'll show them here so you see how that's handled.

The product in this example is an imaginary cloud based customer service application for companies offering building services.

I'll create sticker for just 1 process: the onboarding process for engineers wishing to develop client applications based on this product. What we'll see in the data is that this process is currently only documented for Python developers, and that it isn't something that's available programmatically. I think that's a useful case in that it indicates that there's quite a lot of data we could choose to leave out, but in this case the user profiles are important.

For the sake of simplicitly we'll use just two "stub" user profiles. Both of these engineers work on software that automates aspects of the business. Rakesh uses machine learning to help automatically generate quotes based on information provided by customers. Antje builds frontends for displaying invoices to the customers.

Now we'll produce the individual maturity stickers. As described above this will a series of questions with numeric values as their answers. In this case (and actually in reality) my questions are mostly about coverage rather than qualitative assesments. There's no reason why quality measures can't be included, but personally I've found it hard to find reliable qualitative assesments.

So, what happens next should be pretty obvious. If you look at all the "total" line in bold above it should be clear that we can just extract those ones and ignore the individual questions. Note: You still need to store the individual questions and answers, if you don't you'll have trouble knowing how to update them when you complete some task, and you'll end up having to go through all the questions again. I learned that one the hard way!

Our totals only table looks like this:

Now there are two obvious ways to reduce this data to a more concise summary. We can either sum across all user profiles or sum across all maturity types: