Using Economics to Encourage Testing Incrementally (or As You Go Along)

At TriAgile, I had an interesting conversation with a Product Owner. She described to me a problem where the testers could not keep up and their behaviors were actually holding them back. Let me describe her situation…

In her content development team, they had a couple of testers. They manually tested hyperlinks and other HTML/JavaScript/CSS elements towards the end of the iteration. While she would love to move to automated testing, there were some hurdles to get software approved for use, plus she had this whole behavioral mindset she needed to overcome. The testers on her team felt building and running tests incrementally as a developer completed work on acceptance criteria was wasteful. They preferred to do it after a story was completed by teh content developer; this then always put them being crunched. No matter how hard this Product Owner tried to convince them to do their testing as they go, they resisted. Her Scrum Master was also not providing any influence one way or the other.

As we discussed this at TriAgile, I finally settled on economics to help her understand the situation.

Suppose a content developer produced a defect that prevented a CSS library from working by using a faulty assumption (let’s say it was as simple as a misspelled directory in the URL). And let’s suppose this faulty assumption caused the error to be reproduced 10 times. And further, let’s say each time the person did this, it took them 10 minutes to implement each instance. Lastly, it took the tester 5 minutes to test EACH instance.  So let’s do some math. (All of the times are hypothetical; they could be longer or shorter.)

So first up, testing at the end: 10 x 10 minutes implementation + 10 x 5 minutes testing = 150 minutes. But wait, we now have to fix those errors. So presuming that great information got passed back to the developer and it only takes them 5 min to correct each instance, we need to add: 10 x 5 minutes fix time + 10 x 5 minutes retesting = 100 minutes.  So our total time to get to done is 150 + 100 = 250 minutes to implement, test, correct, and retest the work. Our Product Owner had actually said that this kind of error replication had happened multiple times.

OK, what would have happened if it happened incrementally? Well our implementation time is the same, but after the first implementation occurs it would go to get tested. If an error is found, it goes back to the content developer and having seen the error she or he was making, they can now avoid reproducing it. So the time would be something like this: 1 x 10 minutes implementation + 1 x 5 minutes testing = 15 min, then 1 reworked item x 5 min + 1 item retested x 5 min = 10 min, and finally 9 remaining items x 10 minutes implementation + 9 x 5 minutes testing = 130. Total time now is 160 min.

If the cost was $2/minute (assuming a $120/hour rate employee), you easily wasted

$250 – $160 or $90.

Now multiply this by however many teams are not testing as they go and how many times they have this happen.

Of course there could be items caught that are not recurring, but the fact of the matter is, every recurrence of an error that has to be backed out introduces a lot of waste into the system (defect waste for you Lean/Six Sigma types). Testing as you go and stopping ‘the line’ to prevent future defects from occurring saves money in the long run since labor time is what we are talking about.

In addition to this direct savings as calculated above, one ALSO has the queue time for each test that is awaiting to being tested before it can be OK-ed to produce value. In the first instance, this may be building up considerably, delaying production readiness. And suppose out of the 10 occurrences above, only 8 could be completed because we’re near the end of the iteration? Then we’re probably not going to get all of them tested and any fixes done in time. If we had been testing along the way, then if something didn’t get tested, we could talk with the Product Owner about releasing what was completed and successfully tested. Something of value is completed as opposed to deploying nothing. There is a real opportunity cost for this delay.

So there is something to be learned by each area with this. For the tester, testing completed work, even manually, incrementally keeps you from becoming a bottleneck to producing value. For the developer, giving developed items to the tester incrementally and getting feedback after each item allows you to correct along the way, possibly avoiding future errors. And for the business, having this occur incrementally actually reduces both the real and potential opportunity costs of the work.

TWEET Your Feedback

pixar-birds-wallpapers-movies-widescreen-images-157470

This post was prompted by Jurgen Appelo’s excellent post on giving a Feedback Wrap on Forbes.

I really like this post as it gives you an extremely useful way to express feedback in a manner that will help the receiver actually take action.  It also helps them become aware of future behaviors they may need to change.

Giving good feedback is important, whether it is to superiors, subordinates, or peers. Like Jurgen, I won’t claim to be perfect at doing this and more than once, I know I have provided useless and sometimes hurtful feedback. So to provide a bit of additonal advice as a wrapper around what Jurgen is recommending, I wanted to share an acronym I learned at Culture Camp DC 2012 from Chad Wolfsheimer of the Motley Fool.  The acronym is TWEET; here’s how it breaks down:

Take note of impact

This part is recognizing a meaning to the behavior you want to give feedback on; what is this behavior doing to you, others, a team, and/or the organization. If there isn’t any impact (or perhaps if it truly is trivial), then ask yourself is this feedback going to be useful?

Write down (organize thoughts)

Chad recommends writing down your thoughts, but he did offer up that due to the necessary timing you may not have this opportunity. None-the-less, take a mental step back and organize how you plan to present it; non-organized feedback will come across as a rambling complaint and not achieve what you want.  Using Jurgen’s Feedback Wrap technique is an excellent way to do this.

Empathize

Before jumping and giving feedback, try to understand the context the other person may have. Empathy in this case is not only what they may be feeling emotionally, but also what their mental model may be on why they are exhibiting the behaviors they are. Trying to understand this may help give you insight into how to deliver it so that it is received well.

design it be Effective

Using the Feedback wrap as guidance to organizing it and any insights you may have gained through empathizing, think through how the delivery can be made effective.  After all, if the feedback is ignored or it spawns a defensive mechanism, it probably won’t likely alter the behavior you want changed.

Time it appropriately

We’ve heard how feedback should be timely; Chad recommends and I agree to think about timing. Often just after the behavior is exhibited is the right time, but at other times it may be worth making a determination as to the most appropriate time to deliver the feedback to maximize its reception.

On top of the two great ways to look at feedback that Jurgen and Chad have presented, I also recommend including inquiry.  Asking a few key questions can help you both empathize AND open up the recipient’s mind about the behavior you want to change. Be careful what questions you ask though; for example ‘Why’ questions may put the recipient on the defensive. Try and use open-ended questions as well as this prompts some thinking.  Here’s an example of a question that may work –

After you presented your critique of John’s database design, what did you notice about people’s reactions, and in particular John’s, to your statements?

If the person you are providing feedback hadn’t noticed anything, this question may prompt them to think through what may have been happening and help the recipient self-realize the impact. This makes your job much easier.

I’d avoid the following –

Did you notice how people felt dejected, and in particular John, after you critiqued his database design?

While that may be indeed what you noticed, it is your own mental model that produced that. Even if your recipient might come to the same conclusion, this closed question places her or him on the defensive and not in a place for self-reflection of their behaviors.

I hope you found this post useful, if you have any tips or tricks you use in giving feedback, feel free to leave a comment or tweet at me!