Using Economics to Encourage Testing Incrementally (or As You Go Along)

At TriAgile, I had an interesting conversation with a Product Owner. She described to me a problem where the testers could not keep up and their behaviors were actually holding them back. Let me describe her situation…

In her content development team, they had a couple of testers. They manually tested hyperlinks and other HTML/JavaScript/CSS elements towards the end of the iteration. While she would love to move to automated testing, there were some hurdles to get software approved for use, plus she had this whole behavioral mindset she needed to overcome. The testers on her team felt building and running tests incrementally as a developer completed work on acceptance criteria was wasteful. They preferred to do it after a story was completed by teh content developer; this then always put them being crunched. No matter how hard this Product Owner tried to convince them to do their testing as they go, they resisted. Her Scrum Master was also not providing any influence one way or the other.

As we discussed this at TriAgile, I finally settled on economics to help her understand the situation.

Suppose a content developer produced a defect that prevented a CSS library from working by using a faulty assumption (let’s say it was as simple as a misspelled directory in the URL). And let’s suppose this faulty assumption caused the error to be reproduced 10 times. And further, let’s say each time the person did this, it took them 10 minutes to implement each instance. Lastly, it took the tester 5 minutes to test EACH instance.  So let’s do some math. (All of the times are hypothetical; they could be longer or shorter.)

So first up, testing at the end: 10 x 10 minutes implementation + 10 x 5 minutes testing = 150 minutes. But wait, we now have to fix those errors. So presuming that great information got passed back to the developer and it only takes them 5 min to correct each instance, we need to add: 10 x 5 minutes fix time + 10 x 5 minutes retesting = 100 minutes.  So our total time to get to done is 150 + 100 = 250 minutes to implement, test, correct, and retest the work. Our Product Owner had actually said that this kind of error replication had happened multiple times.

OK, what would have happened if it happened incrementally? Well our implementation time is the same, but after the first implementation occurs it would go to get tested. If an error is found, it goes back to the content developer and having seen the error she or he was making, they can now avoid reproducing it. So the time would be something like this: 1 x 10 minutes implementation + 1 x 5 minutes testing = 15 min, then 1 reworked item x 5 min + 1 item retested x 5 min = 10 min, and finally 9 remaining items x 10 minutes implementation + 9 x 5 minutes testing = 130. Total time now is 160 min.

If the cost was $2/minute (assuming a $120/hour rate employee), you easily wasted

$250 – $160 or $90.

Now multiply this by however many teams are not testing as they go and how many times they have this happen.

Of course there could be items caught that are not recurring, but the fact of the matter is, every recurrence of an error that has to be backed out introduces a lot of waste into the system (defect waste for you Lean/Six Sigma types). Testing as you go and stopping ‘the line’ to prevent future defects from occurring saves money in the long run since labor time is what we are talking about.

In addition to this direct savings as calculated above, one ALSO has the queue time for each test that is awaiting to being tested before it can be OK-ed to produce value. In the first instance, this may be building up considerably, delaying production readiness. And suppose out of the 10 occurrences above, only 8 could be completed because we’re near the end of the iteration? Then we’re probably not going to get all of them tested and any fixes done in time. If we had been testing along the way, then if something didn’t get tested, we could talk with the Product Owner about releasing what was completed and successfully tested. Something of value is completed as opposed to deploying nothing. There is a real opportunity cost for this delay.

So there is something to be learned by each area with this. For the tester, testing completed work, even manually, incrementally keeps you from becoming a bottleneck to producing value. For the developer, giving developed items to the tester incrementally and getting feedback after each item allows you to correct along the way, possibly avoiding future errors. And for the business, having this occur incrementally actually reduces both the real and potential opportunity costs of the work.

A Principle for When Not to Estimate

Based on a prior post, I was ‘asked’ to extract this into a generalized principle. (I invite you to read that in full, so you understand the context.)

Of course, one has to open your mind that the value of estimating has to be questioned in this circumstance and what you are trying to accomplish with estimates and how they may or may not contribute to your work. And I would be remiss not to mention 3 things:

  1. My push on the #noestimates tag is not one of #neverestimate; on the contrary, I am interested in circumstances where estimation may need to improve OR it may need to be ditched to increase value production. I would never advocate to never estimate under all circumstances, yet at times estimation may be wasteful and should be eliminated. Depending on risk understanding, one may absolutely need to estimate. The more potential (in terms of probability and impact) for loss of life or loss of money, the more need to estimate. I do have a keen interest in when estimation may not be needed though beyond just dropping story points.
  2. If you decide to apply this principle, also determine the circumstances and how you would detect that perhaps this principle doesn’t apply in your context. Ask yourself, how does this apply? What will tell me whether I applied it incorrectly, and lastly what possible paths should I take to rectify the situation?
  3. Lastly, this whole set of arguments or debate is very binary. (I’ve only ever seen actual dialog on the topic at Agile Dialogs.) #nobinary is something I advocate on a variety of subjects and you can learn more about how to drop binary thinking and contribute to interventions at Agile Brambles (an evolving site). If your mindset is one where you only see your point of view, then you may find thinking through the paths on the site fruitful.

OK, onto the principle….

When the estimate is not going to be used, regardless of how quickly you can produce it, don’t create one.

Looking back at my context briefly, the old way was to estimate all work that came in and then regardless of the answer, we did the work.  This delayed the start of the work and actually digging into really understanding it by doing it. It doesn’t matter whether you are using a super sophisticated model or a simple guess. In my case it at a minimum delayed work by a week and in some cases more. From a any form of value production standpoint (like weighted shorted job first), the estimation part of the process was hurting the organization. I’m certain many of us can think of times when an estimate wasn’t used and really had no intention of being used.

Be sure though that it truly will not be used. This is important. If this is the case, then it has some degree of importance and then the level of accuracy and precision should be quickly assessed and the estimate done at that level. This brings us to a corollary:

Only estimate at the level of accuracy and precision required (or that has no incremental cost above what is required).

Often estimation takes time and effort. When estimation is needed to understand risk, then only do it at the level required OR if you can do it more accurately/precisely with no additional effort, then go ahead and do it the higher level. This also goes into thinking about creating sophisticated models or simulations that may be used. Bring these in when the situation warrants it, a good prioritized backlog of work, projects, etc. will reveal whether there may be a future for using one.

You may notice that both of these boil down to –

Simplicity — the art of maximizing the amount of work not done — is essential.

Hmmmm… I’ve seen that before.

One last principle:

Question whether this principle (or its corollary) continues to apply.

You may have determined you needed an estimate or perhaps you determined that it wasn’t needed; be sure this still remains true on a periodic basis.

I’ll close, ‘anecdotal’ circumstances are evidence. If you find that you don’t need to estimate under some circumstance, then that is a set of evidence showing a different reality. It’s very similar to observations made by Galileo and Copernicus that indicated that a geocentric view was incorrect and a heliocentric view was more accurate. Or perhaps an even better metaphor would be when Einstein was able to explain observations being made through relativity theory. It didn’t negate how classical mechanics worked under most circumstances, but there were circumstances (context) when mechanics needed to be replaced. If we never examined observations (anectdotes), then everything would remain status quo.

Leadership in Agile Transformations: A Haiku

In keeping with my thoughts on transformation; I wrote a haiku on good leadership that is needed in Agile transformations.

Farmers cultivate

Burros make furrows in minds

More emerge to join

Can you see what leadership is happening in the above? How has leadership been happening in your organization?

 

A bit of Agile Transformation Haiku

One of the Agile Retroflections of the Day I submitted is this one, “What haiku can you write that reflects on your transformation efforts thus far?”

I’m a huge fan of haiku; it uses powerful imagery to convey messages. This was the one I came up with for my customer’s transformation thus far:

Strong burro works hard

Bureaucratic harnesses

But it’s pull, not push

I was inspired to use a burro as they are real workhorses ummm… really burros and because I know a fellow Agilista, Lisa Crispin, raises them.

But upon additional thought, I’d rewrite the haiku slightly to become:

Burro of strong heart

Bureaucratic harnesses

But it’s pull, not push

What would your haiku be?

 

 

Team Launch Misfires…

I’ve been conducting Team Launch workshops for awhile now; some for teams doing software development work and some for other types of supporting products.

I_love_my_awesome_teamI’ve noticed teams that go through a significant amount of effort on these types of chartering exercises seem to always be “awesome” compared to most of their peer teams. They get more done; what they get done is more on target with what the end user needs; and they have less post-deployment problems. I really don’t think this correlation is a coincidence.

My views on launching teams are 99% +/- 2% congruent* with Ainsley Nies’ and Diana Larsen’s views as expressed in their book Lift-off. (If you don’t have this book, go get it!) I had about 5 categories of agreement I used, but they easily collapsed into the 3 from the book: Purpose, Alignment, and Context. Now I use those.

Out of these I view Alignment as the most important one. If I can get to a team that is aligned in HOW they will work together, they can overcome the shortcoming in understanding of the purpose (or on a project where the purpose is emergent) and certainly they can handle an evolving context as they probably thought about those ever shifting sands.

The teams I see that are challenged almost always short change this area as it feels soft and squishy. Why should I understand what my co-workers value? Who cares how decisions are made – let the boss do it as he always has… Why do we need to think about what and how we celebrate things? Ignoring this ignores the first Agile Value: Individuals & Interactions over processes and tools. Launching teams correctly celebrates the former and ensures the latter is subservient to it. Every team I see struggling as of late has short-changed this Team Chartering, particularly around Alignment. A guarantee the missile will miss the mark.

Want more info on how to be successful? See my post on Excella’s blog. And/or get the Lift-Off book and think about what Purpose, Alignment, and Context mean in your environment.

 

*You’ll notice that the upper end of this is 101% which is where I think my congruent thoughts are most of the time…

Symptom of Anti-Agility: CRM Groups

Collapsed_Bridge_Narrower

I haven’t written in a bit (well I have actually, but on Excella’s Insight blog), but I wanted to write about an anti-pattern I am seeing in one of our clients. Many have written that management seems to think of “Agile” as only within development teams, and forget that this Lean and Agile thinking needs to permeate everywhere.  It’s a dysfunction to continue working on…

So let me give you the example I am seeing so we can talk in more concrete terms.

The organization has had a rocky past between central IT and the business. Software development is distributed in various business units as well as a central IT shop (I’m staying away from actual terms the customer uses BTW). Central IT also does considerably more though, such as manage the network, run a server farm or three, manage cloud providers, email and content management services, etc.

The complaints from the business have been articulated as (in no particular order):

  • I have to know who to call in order to get good service; any official channel for requests is difficult to know.
  • Service is often slow and paper/form intensive.
  • When I use any official channel, it doesn’t necessarily get routed to the right location.
  • I’m not told everything I need to do to get my request fulfilled, resulting in delays.
  • I don’t know the status on requests that have a longer time to fulfill, nor have I been told or know where to go to find out my status.
  • I’m uncertain if any feedback I provide is acted upon.

I’m certain everyone has seen these before in some mix.  IT also has a set of complaints:

  • People go to whoever they happen to know that is related to the request they want to make to get service. This eats into people’s time without management having much knowledge of it (perhaps the immediate supervisor knows, but no one else).
  • IT needs to begin a charge-back model due to budget restructuring that is removing a great deal of central ITs funding, thus more knowledge of requests and services is needed to be known. The manner in which requests are coming does not provide a means to track it.
  • The business sides doesn’t see the great work we are doing.
  • The right people in IT don’t receive the needed feedback.
  • IT has a blemished image/brand and are thus not viewed as a valuable partner.
  • The CIO can’t keep up with the requests given to him by his peers; too many are bubbling up and being made directly to him.

If you work in an IT organization, particularly as a senior manager type, how many of these have you seen? Before I go into how to handle these with some Agility, let’s look at what this organization is doing that I consider an anti-pattern; literally anti-Agility thinking.

The senior manager tasked with solving this sees this primarily as a communications problem in 3 areas:

  • Routing to the right people to do the work
  • Taking in feedback on ITs performance
  • Improving the communications on good performance (improving the “brand”)

To solve this, they are standing up a Customer Relations Group that will take in initial requests/requirements, receive feedback about performance during and at the end of the work and provide it to the group(s) that performed it, and get the word out on the good stuff IT does.

They are starting by creating a master catalog of services IT provides (which on the back-end will have costs associated with them). It’s unclear whether these costs and/or the parts of the IT organization that will perform them will be a part of this catalog.  So far not bad… They also will stand up a single intake service (along with an online form) to process these needs, then ‘interview’ the customer to get any additional information, and route this request to the proper parts of IT.  It seems at this point there will be a hand-off; what is unclear is how this hand-off will work if multiple parts of the organization are involved simultaneously. If software development is the primary concern, it will be handed off to a development team (or a new team will be stood up).

This new Customer Relations Group will also query the customers periodically to find out how well the various parts are doing and give this to the right part of the organization. It will also give any responses back to the customer. And finally it will create some marketing material and positive ‘press’ to help sell IT services.

To keep this from being a burden on the current IT organization, the bulk of this work will be contracted out to a big name firm that everyone respects. Sounds great doesn’t it!? Every organization needs this, right?

As well as intentioned at this group (and its contractors) may be, this creates several dysfunctions.

  1. This is actually adding an additional step that will actually slow requests getting to the right people. This will increase the time to service, probably decreasing happiness, perhaps not initially, but certainly in the long run.
  2. It adds a hand-off of information so that the people needing it are getting it second-hand. This will lead to greater misunderstandings between the groups that need to work together.
  3. Feedback and working communications are at least partly being removed from the group that needs them directly.
  4. There is an assumption that a one-size-fits-all intake process can accommodate getting all of the unique needs a customer needs to portray to a unique supplier. Requests for an email distribution list would be vastly different than one for a software development project, or just even business analysis services for a development project.
  5. Messaging (branding) will be coming from a third party not responsible for any of the results.

Even worse though, there are fundamental root-cause problems being masked over.  For example, why does the business feel feedback they give is not listened to and acted upon? This solution isn’t going to address this root-cause concern. What is causing the long lead-time for IT to respond to business needs? Why are we trying to create a standard process for intake and routing as opposed to simply better connecting people to those that would supply the services and give proper visibility into the incoming work, but allow a self-routing approach? Why do we need to do a charge-back for the services internally as opposed to getting the budget reallocated properly to pay for them?

So how would I approach these items? I’d start with challenging the fundamental way things are done. I would learn where are the bottlenecks causing the unacceptable lead-time. I’d investigate the root-causes to the image/branding and see how to solve those. I’d see how I could make a catalog of services attached to the people that provide the services and work them to create mechanisms that give me an understanding of the allocation of requests. And I would at least talk with the budget personnel to find out how I could simply get the budget allocated properly; if that couldn’t be done, I’d make my service costs transparent to those when they look up my services. If I wanted something Customer Relationship-like, I’d perhaps think about deploying customer relationship software to all the groups directly, if evidence showed it would help.

Bottomline: I’d reduce hand-offs and keep with the spirit of individuals and interactions over processes and tools. I’d do the simplest thing I think is in the right direction and then retrospect on how that is working for me.

 

TWEET Your Feedback

pixar-birds-wallpapers-movies-widescreen-images-157470

This post was prompted by Jurgen Appelo’s excellent post on giving a Feedback Wrap on Forbes.

I really like this post as it gives you an extremely useful way to express feedback in a manner that will help the receiver actually take action.  It also helps them become aware of future behaviors they may need to change.

Giving good feedback is important, whether it is to superiors, subordinates, or peers. Like Jurgen, I won’t claim to be perfect at doing this and more than once, I know I have provided useless and sometimes hurtful feedback. So to provide a bit of additonal advice as a wrapper around what Jurgen is recommending, I wanted to share an acronym I learned at Culture Camp DC 2012 from Chad Wolfsheimer of the Motley Fool.  The acronym is TWEET; here’s how it breaks down:

Take note of impact

This part is recognizing a meaning to the behavior you want to give feedback on; what is this behavior doing to you, others, a team, and/or the organization. If there isn’t any impact (or perhaps if it truly is trivial), then ask yourself is this feedback going to be useful?

Write down (organize thoughts)

Chad recommends writing down your thoughts, but he did offer up that due to the necessary timing you may not have this opportunity. None-the-less, take a mental step back and organize how you plan to present it; non-organized feedback will come across as a rambling complaint and not achieve what you want.  Using Jurgen’s Feedback Wrap technique is an excellent way to do this.

Empathize

Before jumping and giving feedback, try to understand the context the other person may have. Empathy in this case is not only what they may be feeling emotionally, but also what their mental model may be on why they are exhibiting the behaviors they are. Trying to understand this may help give you insight into how to deliver it so that it is received well.

design it be Effective

Using the Feedback wrap as guidance to organizing it and any insights you may have gained through empathizing, think through how the delivery can be made effective.  After all, if the feedback is ignored or it spawns a defensive mechanism, it probably won’t likely alter the behavior you want changed.

Time it appropriately

We’ve heard how feedback should be timely; Chad recommends and I agree to think about timing. Often just after the behavior is exhibited is the right time, but at other times it may be worth making a determination as to the most appropriate time to deliver the feedback to maximize its reception.

On top of the two great ways to look at feedback that Jurgen and Chad have presented, I also recommend including inquiry.  Asking a few key questions can help you both empathize AND open up the recipient’s mind about the behavior you want to change. Be careful what questions you ask though; for example ‘Why’ questions may put the recipient on the defensive. Try and use open-ended questions as well as this prompts some thinking.  Here’s an example of a question that may work –

After you presented your critique of John’s database design, what did you notice about people’s reactions, and in particular John’s, to your statements?

If the person you are providing feedback hadn’t noticed anything, this question may prompt them to think through what may have been happening and help the recipient self-realize the impact. This makes your job much easier.

I’d avoid the following –

Did you notice how people felt dejected, and in particular John, after you critiqued his database design?

While that may be indeed what you noticed, it is your own mental model that produced that. Even if your recipient might come to the same conclusion, this closed question places her or him on the defensive and not in a place for self-reflection of their behaviors.

I hope you found this post useful, if you have any tips or tricks you use in giving feedback, feel free to leave a comment or tweet at me!