Tag Archives: software development

Using Economics to Encourage Testing Incrementally (or As You Go Along)

At TriAgile, I had an interesting conversation with a Product Owner. She described to me a problem where the testers could not keep up and their behaviors were actually holding them back. Let me describe her situation…

In her content development team, they had a couple of testers. They manually tested hyperlinks and other HTML/JavaScript/CSS elements towards the end of the iteration. While she would love to move to automated testing, there were some hurdles to get software approved for use, plus she had this whole behavioral mindset she needed to overcome. The testers on her team felt building and running tests incrementally as a developer completed work on acceptance criteria was wasteful. They preferred to do it after a story was completed by teh content developer; this then always put them being crunched. No matter how hard this Product Owner tried to convince them to do their testing as they go, they resisted. Her Scrum Master was also not providing any influence one way or the other.

As we discussed this at TriAgile, I finally settled on economics to help her understand the situation.

Suppose a content developer produced a defect that prevented a CSS library from working by using a faulty assumption (let’s say it was as simple as a misspelled directory in the URL). And let’s suppose this faulty assumption caused the error to be reproduced 10 times. And further, let’s say each time the person did this, it took them 10 minutes to implement each instance. Lastly, it took the tester 5 minutes to test EACH instance.  So let’s do some math. (All of the times are hypothetical; they could be longer or shorter.)

So first up, testing at the end: 10 x 10 minutes implementation + 10 x 5 minutes testing = 150 minutes. But wait, we now have to fix those errors. So presuming that great information got passed back to the developer and it only takes them 5 min to correct each instance, we need to add: 10 x 5 minutes fix time + 10 x 5 minutes retesting = 100 minutes.  So our total time to get to done is 150 + 100 = 250 minutes to implement, test, correct, and retest the work. Our Product Owner had actually said that this kind of error replication had happened multiple times.

OK, what would have happened if it happened incrementally? Well our implementation time is the same, but after the first implementation occurs it would go to get tested. If an error is found, it goes back to the content developer and having seen the error she or he was making, they can now avoid reproducing it. So the time would be something like this: 1 x 10 minutes implementation + 1 x 5 minutes testing = 15 min, then 1 reworked item x 5 min + 1 item retested x 5 min = 10 min, and finally 9 remaining items x 10 minutes implementation + 9 x 5 minutes testing = 130. Total time now is 160 min.

If the cost was $2/minute (assuming a $120/hour rate employee), you easily wasted

$250 – $160 or $90.

Now multiply this by however many teams are not testing as they go and how many times they have this happen.

Of course there could be items caught that are not recurring, but the fact of the matter is, every recurrence of an error that has to be backed out introduces a lot of waste into the system (defect waste for you Lean/Six Sigma types). Testing as you go and stopping ‘the line’ to prevent future defects from occurring saves money in the long run since labor time is what we are talking about.

In addition to this direct savings as calculated above, one ALSO has the queue time for each test that is awaiting to being tested before it can be OK-ed to produce value. In the first instance, this may be building up considerably, delaying production readiness. And suppose out of the 10 occurrences above, only 8 could be completed because we’re near the end of the iteration? Then we’re probably not going to get all of them tested and any fixes done in time. If we had been testing along the way, then if something didn’t get tested, we could talk with the Product Owner about releasing what was completed and successfully tested. Something of value is completed as opposed to deploying nothing. There is a real opportunity cost for this delay.

So there is something to be learned by each area with this. For the tester, testing completed work, even manually, incrementally keeps you from becoming a bottleneck to producing value. For the developer, giving developed items to the tester incrementally and getting feedback after each item allows you to correct along the way, possibly avoiding future errors. And for the business, having this occur incrementally actually reduces both the real and potential opportunity costs of the work.

The UX-CX Dance

How seriously do you consider user experience for your internal applications? There  seems to be much discussion around creating good user experiences for outwardly facing applications; however, equally important is those that are internal, particularly if they support people that directly interact with your organization’s customers.

Here’s an example of what I mean; let’s start with some context.

My family and I were flying back from Australia earlier this week. In order to make our flight, we got up very early (4am) to do final prep before our taxi picks us up.We made plans to arrive a little over 2 hours20140523_VH-XFB_Keith_Anderson prior to our flight knowing that would be sufficient to check-in, make it through security, and have a coffee. We’re going to be flying domestic to Sydney before boarding our international flight to LAX and then transfer to their affiliate airline domestically back to DC. We have a premium economy seating internationally and domestic internally; that isn’t offered domestic in Australia (only economy and business).

We arrive at the airport without a hitch. Because we are starting off domestically and not going straight international from Melbourne, we go up to the domestic agent to do our check in. She is very pleasant and nice! (Particularly for this early in the morning, which is about a 5:40ish and a bit earlier than even we anticipated arriving.)

First up, we provide our information and US passports and at her request place our first bag up for weighing. She states that we’re overweight on our bag. The policy of the airline as we had checked it, was 32kg international premium economy (it was 23kg for just economy). I personally checked each bag with a sale we bought as we packed it and ensured we were well under for each bag (greater than 4kg). My wife pleasantly points this out; our agent wasn’t argumentative, but stated she would have to check since we started off only on economy. About 5-6 minutes later when she returned, she had her answer that yes it was allowed. By the time she had gotten back, my wife already had the policy pulled up on their own website. UX point #1: This should have been available on the screen to her without her having to go check with someone (presumably a manager).  Perhaps an agent at a check-in for international would have had this available, but I doubt it; most likely they would just know the policy due to necessity. Given the airlines current route structure, MOST international flights to places in other parts of the world would fly from airports other than Melbourne, thus this ‘help’ feature would have made sense to be made available to every agent.

So she returns to entering our passport information. She apologizes that she has to key in the address we are going to (in this case our home address) for each person separately and thus consumes more of our (and her) time. We casually discuss that this multiple entry seems inconvenient. UX point #2: There should have been a way for identifying people flying together as being members of the same household so that the address field would only be need to be entered once. There is good reason for having this as an option; I could tell she was a bit frustrated about it and it was preventing her from helping others in the line as the airport got busier. She remained very pleasant to us and it never impacted us being able to depart on time.

My wife is both a US and Australian citizen, where as I and my son are just a US citizens. The next issue came up when she entered my wife’s passport information; it didn’t want to let her complete the transaction since she hadn’t entered in on a visa.  She had flown in on her Australian passport since she is an Aussie citizen and didn’t need a visa, where as my son and I entered on visas. So she swapped to using my wife’s Aussie passport; now it wanted a visa for entering the US. After a bit of hassle and finally asking someone, she found out that it wasn’t possible to enter two passport numbers on the screen without having someone link the records on the back-end (presumably some configuration/database entry) to enable that feature. UX point #3: the developers had not considered the persona of a dual citizen and now it had become a clunky customer support  feature. There are lots of dual citizens in Australia, particularly with Britain.

So at this point, let’s stop for a moment and consider perhaps a deeper cause to these three UX points. (BTW, I never saw her screens, but the last two had her frustrated enough that she was pleasantly talking through what she had to do.) I would venture to guess that the development team, and particularly the product owner/business representative, of this application never fleshed out many personas of either the agents nor the customers they would be helping. They probably ONLY considered ‘agent’ role as the one possibility and never the people they help.

Want to improve the product owner’s ability to support her or his user base? Help them understand their customer and that customer’s customer using customer journey maps. (I particularly like using the Lego Customer Experience Wheel or the Innovation game Start Your Day.) Flesh out the personas with Empathy maps and further refine your backlog based on these.  If you want to understand better how backlogs change based on personas (whether it be customer persona or role), check out the game “Backlog is in the eye of the beholder” game.

Organizational or business agility means attending to customer needs; gaining the right UX/CX experience in your product, release, and iteration planning is key to doing that right.

(Incidentally, we had overall CX impacts with how the airline had negotiated arrangements in how people are physically moved by a bus between terminals in Sydney as well, so using customer journey maps can really help give you a holistic view in how to improve your relationships with them, something that is all important these days.)

Another Move Back to Central IT for the Feds?

Federal Computer Week recently ran an opinion article by Michael Garland, a former BearingPoint procurement veep and currently under contract to help the government with Acquisition Reform; you can read this article here: “Why acquisition reforms fall short“.

In this article, Mr Garland states the following, “…there is no centralized, integrated strategy supported by a corresponding organizational structure. The government has been intensely fragmented when it comes to IT acquisition.”  He then purports how the Government should act as a centralized enterprise when it comes to acquisition and management of IT. He begins with a statement that the Government acts as a “holder” of a portfolio (much like a financial portfolio) and then later states the following:

“That approach lacks cohesion and inhibits the ability to develop and exploit best practices. It has been an ad hoc structure devoid of an enterprise strategy. The fragmentation also hinders the ability to develop valued expertise or deploy any of the various continuous improvement methodologies that have been so useful for the private sector.”

Having worked for many an agency as a Fed, Navy officer, or contractor, I can say that centralization causes many woes, the largest one being that no central IT strategy (acquisition or otherwise) can be attuned to an Agency’s unique needs.  Integrating custom systems of systems command and control software for a Naval ship is entirely different than acquiring a human resources application, to deploying a mortgage loan lending package, to building a custom-built analytical program for determining pesticide safety.  It appears his mental model is all about cost efficiencies and not value production.

What can be promulgated to help each agency is guidance (not rules) for helping procurement officials make the appropriate decisions on contract types so that risk can be minimized and managed effectively locally, while also producing maximum benefit/value for the Agency’s mission for supporting the American people. Central rules destroy customer experiences in almost every industry. (Actually only the most risk averse/safety-centric industries gain from central rules.)  Central rules sets are designed for cost efficiency at the expense of effectiveness in delivery; I contend this is not a strategy to pursue.

I also contend that a portfolio of holdings is EXACTLY what we want; and what we want to extend is thinking that helps our brokers (the Agencies) invest better on our behalf in ways that respect their missions.

Think alignment over control…

—–

Follow-up: a colleague passed me a pointer to an 18F article on Ghost Writing RFPs. This is a step in the right direction if this is used to enable improvement throughout agencies in how they write RFPs.  Unfortunately, at the point of writing an RFP, an acquisition strategy may already have been made and depending on the flexibility and remaining options (in terms of time) to modify these later, the various Departments and Agencies may have a tough time taking advantage of this service.