This was a question posed by my esteemed colleague Bernd Schiffer (@berndschiffer) on Twitter. This is a complicated thing to measure. We all know what effects unsustainable pace has, AND often when they show up, it’s too late. I’m going to divide this post into two segments; first will be early warning signs that a pace may be becoming unsustainable, so you can try and nip it in the bud early. Second will be how to determine that your pace is likely sustainable. (I say likely as it’s always possible these possible metrics may not catch everything.)
Early Warning Signs
So let’s start with signs that one (most likely a coach or Scrum Master or other suitable servant leader/team facilitator) may use to find out if a team may start to hit an unsustainable pace. None of these on their own may be enough to indicate encroachment of an unsustainable pace, so ensure you get a balanced view.
If you have access to time card data, and if people are honestly reporting their hours, you can look at how the hours they record are trending. This can help you spot both individuals that are undertaking an unsustainable pace as well as the team. This really needs to be complemented with a gemba walk so you know what are normal hours for people and when people may be burning the midnight oil. The occasional late night may be ok; perhaps the individual or team has an interesting problem they want to solve before going home. If this is a regular occurrence, then you most likely have an indicator that an unsustainable pace is starting to happen.
Look at check-in trends. Do they become more rapid in pace at the end of an iteration or before some significant milestone? Do these correlate to an increase in broken builds or failed tests? Does test coverage slip around these? These may be indicators that people are getting rushed in their work. Along with these, trends in escaped defects may also indicate whether a team is receiving pressure to deliver.
If you are running a Niko-Niko or otherwise taking a pulse of the team, then downward dips or loss of stability in happiness may likewise be an indicator that the pace may be becoming unsustainable.
Lastly, consistent carry over of uncompleted work between iterations, measured in story points if you are using them or just in the number of stories, tasks, et cetera, can be an indicator of possibly approaching an unsustainable pace.
None of these on their own though make for a good set of measures, only by cross-correlating them can you possible get a sense of a team’s pace getting to where it may begin to collapse under pressure.
So what should you do? Well if you are seeing these metrics, you will need to 1) confirm that the pace is truly becoming unsustainable and 2) remove the pressure in a manner suitable to getting the desired results. For both of these, sitting down one-on-one and having some conversations as well as conducting some retrospection as a team on what is happening may give more insight than any metrics could. This will also uncover the pressure that is causing it in the first place. The pressure may be appropriate; what may not be appropriate is the reaction. For example, suppose the product owner is not accepting many stories because of a team not meeting all their acceptance criteria. The team decides to begin working late rather than reduce the number of stories they take in as a means for ensuring they get all the acceptance criteria completed. The response is the issue, not the pressure.
Measuring Sustainable Pace
With the above, you’d think it perhaps good enough to just look for the absence of those negative signs. Presuming people are truly being open and honest AND also aware of the pace becoming unsustainable then yes, the absence may work. But what if workdays have been regular, but then every so often a few extra hours go in, then suddenly there is a sharp uptick in the number of hours? That may take what was tolerable to intolerable, particularly if it causes stresses between workers where some have family commitments and others don’t and thus get ‘stuck’.
A better measure for maintaining sustainability on a team may be noting how often the team can push back on requests that regularly cause these extra hours. This means the team can choose when they want to do them or not. This is more of an observation. Combined with the Niko-Niko and absence of some of the metrics above, now we may be getting an additional dimension that says the team is working sustainably.
Another dimension can be how well the team honors the Retrospective Prime Directive, not only in retrospectives, but also during standard work. If they aren’t casting work in us vs them or treating other external stakeholders with contempt then the team is probably working sustainably.
And lastly, how often does the team celebrate? Do they take time to appreciate each other? These all are indicators of sustainable pace as well.
As you can tell, it’s probably difficult without an excellent Scrum Master/team facilitator, manager, or Coach paying attention to this.