Views of Estimating and Not Estimating for an Executive

This post was developed in order to give a longer response to Henrik Ebbeskog’s.

My personal response to this tweet is that it represents a one-sided static and unsophisticated view of what a CEO may want. I am going to use this question as launch point to show that a range is possible. Which one may be ‘correct’ is highly dependent on the CEO’s mental models (and motivations), the organization, and the environment in which the organization finds itself. In this post, I am only trying to disprove the hypothesis that the CEO must understand some estimate from a traditional point of view at the beginning of an initiative. I’m also going to explore this from a financial sense, not so much from what a team may do around story points, et cetera, though I will make a short mention of it at the end.

For context that we can work within, the scenario is a SaaS company that provides financial compliance services. The company already has revenues in the high tens of millions of dollars and thus is not a start-up. The CEO is interested potentially expanding into a new market by launching new product services in helping clients monitor environmental compliance.

A Traditional Estimation Mindset

If the CEO is in a traditional estimation mindset, she or he will be interested in knowing as much about the iron triangle’s values of cost, time, and scope as possible. The CEO will turn to marketing (the Chief Marketing Officer if they have one) and ask of them “what are all the environmental compliance monitoring needs, who are our potential customers, and what is the potential revenue for these services?” Before marketing runs off and does this research, the CEO also asks, “how long will it take you to research these, and how much will this research will cost?” These are of course fair enough questions; the CEO wants to know the potential cost of the information before giving a go-ahead and whether it can be done in a reasonable timeframe.

OK, an estimate is made on cost and time (hopefully using historic data if they have it) the answer sounds reasonable to the CEO, so the green light to proceed is given. So marketing proceeds with the work they do in order to understand the market space taking about one quarter to do so at roughly $150K; this is on schedule and on budget from the estimate they gave the CEO (1 quarter and $150K+/-$10K; woot! win!). They may research the web on compliance needs, survey companies, see if competitors exist, et cetera. It looks promising; the revenue looks like it will be $10M for the first year, $20M the second year, and an estimated $30M the third year.

Now the CEO turns to the Chief Technical Officer asking, “how long will it take you to build this and when will it be done?” as he hands marketing’s finding on scope to him. Of course the CTO doesn’t give her or him a flippant answer, so the CTO goes back and pulls together a cross-functional team, including an experienced product manager, (let’s assume they have been using Scrum/XP practices for years) and this team defines an MVP with a rough price tag of $225K+/-$50K to get there. They also come up with an estimate of a first marketable release a quarter after that, and (in talking with marketing) another 2 subsequent improvement releases based on prioritized environmental monitoring needs the next two quarters after that for a total cost of $900K+/-$200K. Cool beans! Let’s go!

They execute and for simplicities sake they stay true to their estimate of $900 and $225K per quarter. I want to state that once the team was pulled together, the cost over a time interval is known if it is a stable cross-functional team.

The mindset here is understanding risk before executing (and of course managing it during execution).

A Lean Start-up Mindset

The CEO is interested in exploring the same environmental compliance space. He talks with his other executives and they decide to form a cross-functional team of marketing, which includes an experienced product manager, and IT personnel. They pull together a hypothesis of customer, problem, and solution and identify a set of assumptions about this. The team and executives set a vision for the need and boundary constraints that ensure it stays aligned with the company’s core vision and mission. Within these constraints is a set of questions that if understood state criteria for a transition state to a development effort as it provides enough detail to define an MVP and MMP as well as what the revenue stream for the MMP and other potential known releases of the SaaS product will be. The CMO is appointed as the oversight on this effort and agree that after each assumption is tested he will review whether to proceed, pivot, or kill. It’s worth noting at this point, none of the iron triangle is known. Costs per week of this assigned team are known (just like AFTER marketing provided the estimate above and were told to execute their research).

The first assumption is that monitoring a particular environmental compliance aspect is unserved in the marketplace. The team tries to find evidence of this through market research and a survey to their financial service customers in the same space. It is not disproved, so the CMO gives a proceed (or in start-up terms persevere) signal. They test the next assumption, and then the next. Sometimes, they build a quick prototype to see if a particular compliance rule can be enabled (it was the riskiest assumption). At the end of a quarter and $150K they know what the MVP and MMP look like and what the next 2 releases look like. They have the same revenue stream predicted as the traditional team.

At this point the team is reconfigured to look like the Scrum/XP team above and proceed (however, no estimate is asked for) and we’ll say that they deliver just as the team above costing $900K. The important point is that the same stable cost over the time intervals operates as above. At each quarter a proceed or kill decision is made based on the throughput of work was completed to what remains. This is a form of estimation – yes I realize it. What is different in this case is when the estimate is made; I didn’t start with an estimate. A rougher approach is simply looking at the remaining backlog compared to what’s left. I could also evaluate the marketplace at this point using the competitive analysis approach I had done and see if I want to continue (basing a decision on expected value in terms of a change in potential revenue – another form of estimate).

Other Alternatives

I could choose to do a traditional approach to marketing and then build without the estimate on how much it will cost. I could reverse this and do a lean start-up approach to understanding the market demand and transition to an estimated approach as well.

One thing to note: the lean start-up approach could be modified so that once the MVP (or MMP) was defined, the company could actually start building knowing they have some revenue stream that will come in; they may not know whether the company will get to the revenue stream as predicted though. This would decrease time to market for the MMP and may allow it to even expand into the other markets or it could decide to not and become a complementary service companies purchase.

Take-Aways for the Reader

First, I painted a rosy picture of delivery. It is very likely delivery will not go as smoothly as this. Thus when I reach the end of a quarter where a release is defined, I need to decide whether to continue or not or release with what I have. In a traditional, estimated viewpoint, I am deciding whether to add more time to the schedule (and potentially run over budget as a result) to release with the fully expected scope or release with less scope on time and at budget. Regardless of whether I estimated the length of time it took me or not, I can use my actual throughput of work as the predictor or not on whether to continue. Again, as I mentioned before this is a form of estimation; I am just choosing to do it later.

Second, I didn’t get into estimation that may occur (or not) by the team.

The primary use of story points (or another team estimation method on stories) is to know whether a story is small enough to be completed easily within an iteration. Some teams get really good at understanding their sizing and can stop using story points. (Lunar Logic’s estimation cards are a good insight into this, every story is either a 1 – we can take it on, TFB – too f-ing big, or NFC – no f-ing clue.) I encourage teams to examine the story’s independence and testability to gain this understanding as these two parts of the INVEST criteria are what feed the complexity thinking one needs to understand in applying story points. Teams can still measure throughput and lead time as these can be useful for later questions when an estimate may be needed about ‘how much’ or ‘how long’.

Third, I’d like to change the convo a little a bit. I personally think value and cost are decoupled. Net value (value minus cost aka ROI) are coupled. A great place to get a sense of this is Reinventing Project Management by Shenhar and Dvir. In this book, the authors describe two scenarios where the cost of the project had nothing to do with the end value of what was produced. Another change is ridding ourselves of the use of project thinking when we are doing product efforts. Product life-cycles extend beyond initial delivery and when use project thinking we often short change understanding of both costs and value in the long-term, whether we estimate or not.

Fourth, I hope this gives some insight into when choices can be made about estimation; it is not a simple binary answer, but one of fidelity. In some cases, one will want to run several detailed simulations in order to understand whether an undertaking should be done. In other cases, maybe we can just get started with none what-so-ever. Humans actually never escape mental models of estimation however, even a zero on this range assumes we will get some learning insight that has value and that in itself is an intuitive estimate. We certainly discovered this at the first Agile Dialogues unconference. (Biggest personal disappointment at this unconference is that the person that helped shape the theme then chose not to come after indicating they would.) What the thinking in the #noestimates ‘movement’ is trying to do is change the nature of this and question what our assumptions and beliefs are about what and when to estimate.

I’ll close with saying that there are people that add well to this conversation – they bring in well-formulated opinions. There are others that prefer to provoke – this occurs on both sides unfortunately. I personally seek actual dialogue so we can get out of binary thinking on this (see the Agile Bramble). I point out circumstances where not estimating work not to debate that not estimating is the path to follow, I’ve never said ‘never estimate’, but to have more dialogue of when we should undertake it or not and what we should estimate. Notice I didn’t say ‘if’. I’ve had someone state I evidently had no evidence when I have given some. I’m also not interested in endless debate – ask yourself do you feel you need to ‘win’ an argument. If so, you are not in a mindset for dialogue or learning, but to prove a point.

With this, I hope I have shown that the Rule of 3 applies 🙂

 

Leadership in Agile Transformations: A Haiku

In keeping with my thoughts on transformation; I wrote a haiku on good leadership that is needed in Agile transformations.

Farmers cultivate

Burros make furrows in minds

More emerge to join

Can you see what leadership is happening in the above? How has leadership been happening in your organization?

 

A bit of Agile Transformation Haiku

One of the Agile Retroflections of the Day I submitted is this one, “What haiku can you write that reflects on your transformation efforts thus far?”

I’m a huge fan of haiku; it uses powerful imagery to convey messages. This was the one I came up with for my customer’s transformation thus far:

Strong burro works hard

Bureaucratic harnesses

But it’s pull, not push

I was inspired to use a burro as they are real workhorses ummm… really burros and because I know a fellow Agilista, Lisa Crispin, raises them.

But upon additional thought, I’d rewrite the haiku slightly to become:

Burro of strong heart

Bureaucratic harnesses

But it’s pull, not push

What would your haiku be?

 

 

Agile Dialogs Recap

This will be a short recap of the Agile Dialogs unconference held yesterday.  We discussed ways of predicting value production with and without estimates.  Over the next few days I’ll blog more of what we uncovered, but this will be a simple post on how the unconference was approached.

We had a good mix of people that were passionate, though no one was at I’d say fully at each end of the spectrum. The big takeaway was that both sides are right in many ways and wrong in many ways.  The idea of not using estimates of time, money, and/or story points can be done and is highly context dependent. As with any approach it may nor may not work in your context; it depends, or YMMV.  The best you can do is try it as an experiment and see whether it works for you.

What we did at Agile Dialogs was –

  • register with one side or another along a continuum (how strong we felt on the issue),
  • post the types of things we estimate,
  • tell our stories of both our successes and failures on both sides – with and without estimates
  • explore our objectives for either using or not using objectives and the techniques we use for each side
  • Explore the assumptions used when using estimates
  • Explore the assumptions used when not using estimates
  • Explore what each side could learn from the other
  • Posted and voted on what could possibly be the next thorny topic we tackle
  • and retrospect on how the Agile Dialogs unconference could be better

Here’s a few teasers of some of the discoveries… I’ll go more in depth on what was discussed in future posts as well as post some proceedings on the Agile Dialogs site.

  • When management or customers are asking for estimates, it is more important to understand their need for it; then more valuable alternatives to fulfill that need may be explored. Estimates may prove best for fulfilling that need though, so don’t force fit an alternative technique.
  • Estimation has become a scapegoat for other dysfunctions within the works system. Removing estimation won’t fix these dysfunctions, but it may help uncover them.  Whether at the end of the day, you remain with or without estimation, if these more fundamental dysfunctions can be fixed, then the work climate will improve.
  • Estimation always exists, but when pursuing a noestimates approach, the nature of the estimation actually changes from cost, time, and/or complexity to value (which is not based on those in most environments).
  • Focusing on understanding time and money estimates tends to introduce longer feedback loops for actual learning. If it is possible (and that is an IF), then removing them can eliminate waste in the work system to that learning.
  • Measurement is important in both approaches; when doing estimates we sometimes get lulled into a false sense of security that good measurement exists, when often it doesn’t.
  • Humans suck at estimation except on conceptually obvious items (obvious equating to the obvious domain in the Cynefin framework); mathematical models (particularly when the underlying assumptions on those models are validated by the team doing the work) can really help produce accurate results in the complicated domain.  The complex domain can be assisted greatly by these mathematical models, but the loop through is validating a hypothesis.
  • Another way to test a hypothesis is to set time or cost box and see if the solution at the end of the box is on track decide whether to spend more, accept as-is, or abandon; think Lean Start-up approach.

I have set-up The #AgileDialogs Daily that curates information from both sides of this thorny topic; other thorny topics will get added as a discussion on them emerges.

What’s This Agile Dialogs Thing Anyway?

If you haven’t caught it, I’m running an unconference called Agile Dialogs; you can find out more about it at http://agiledialogs.org.

So why would I want to take on thorny topics, ones that seem to bring out flamewars? Because the lack of listening to each side as we argue from each other’s sidelines seems an inane way of advancing our craft.  If we want organizations to advance their thinking, we in the community need to advance ours and listen to those with differing opinions. It doesn’t mean we need to agree, but we do need to listen, truly listen to what the other side is saying.  When we decide to challenge the other side, we need to do it in a manner that isn’t trying to cole them into accepting we are right, but to have them think through why they are taking the position they have chosen. We may reaffirm it, but in the process, we will have had them rethink underlying assumptions.

Dialog is about understanding and elevating assumptions so we can find answers to our questions and perhaps a new better way forward.  I know I am a believer in good estimates when they make sense and when they don’t not even bothering with them. But perhaps when I thought they weren’t useful, there was a better way to have made them useful.  I certainly welcome learning that in a manner that doesn’t start out with – hey bud you are wrong. That closes down dialog as that is about winning an argument. Save the arguments for a debate, let’s find out what makes each side tick and see what we can learn.

I hope you will join me!

Using Dollars as a Constraint on a Project

I’ve been planning to write this for awhile, and this seemed to me more important to post after seeing an update from a Kickstarter campaign I am backing.

So I backed a board game, I was particularly interested in that it i intended to be small so I can take it with me almost anywhere I go.  What was amazing to me was how they calculated what they needed for funding.

Before I dive into that, I’ve backed quite a few boardgames on Kickstarter (along with music albums, music gear, and camping gear…) Most Kickstarter projects go in with varying degrees of estimations; one nice thing Kickstarter does is if you don’t reach your funding goal you don’t owe to make anything and the backers keep their money.  If you get funded, your estimates hopefully allow you to produce the game and have at least a small measure of profit. Most projects offer stretch goals that when they are reached, component upgrades and such kick in – these usually have a change in your estimate.

The gentleman that developed Carrier Commander, decided on a price point he wanted to be able to sell the game ($3 as it is a nanogame; I love small games to take with me when I travel). From there he reversed everything into size and weight by calculation based on what would be possible should he hit his stretch goals.

On the campaign page, he reveals the cost breakdown including the “Uh-Oh” zone which is the profit area…

To read up on how he calculated his way into the $3 price point without estimating, see this update:

https://www.kickstarter.com/projects/1078944858/star-patrol-carrier-commander-3-sci-fi-strategy-na/posts/1348731?ref=dash

Should all Kickstarters work this way?  Probably not…  The larger the game, the more the calculations would become overly cumbersome, particular as stretch goals needed to be calculated, so using estimation and factoring in reserve to cover the uncertainty would probably suffice.  In his instance, his upgrades were in cardboard only, so this made it much easier.

So how would this relate to software development? As I wrote in my post “When I Have Skipped Estimates”, one could use a team size as a constraint and then measure throughout.  Once the constraining bottleneck is understood and all worthwhile options for increasing throughput there have been exhausted, you could increase capacity.  This really works well for software maintenance.

One could also use something akin to what this gentleman did for his Kickstarter game; establish a fair market value for the cost of what you are building (i.e. how much is someone willing to pay to have something by a particular point in time).  Once you have this you have both time and budget constraint and now you can see how much that would pay for in terms of people and other infrastructural resource one may need; i.e. what is the capacity it can purchase?  Let’s say we got enough money that it would pay for 7 people for 6 months (+ servers, desktops, software licenses, etc). We can then execute and develop based on that.

One may ask at this point, how do you know if you will make what is needed? You actually don’t. What you do know is that this is what the person or people that set the constraint said would be what they are willing to pay. Like a venture capitalist, they have in their mind, I am willing to risk this amount of money to see if I can get what I want.  Yep, no guarantee. But then, an estimate doesn’t produce one either.

Should you do this under all cases? Absolutely not. In fact, I would say estimation is needed more often than not when deciding to fund a project (or program). And for those cases, we as an industry need to improve in estimation. However, there are cases,where estimation doesn’t necessarily help us. The more novel the project (and thus its approach), the greater the uncertainty and at some point it may be best to establish a cost (and perhaps schedule) constraint and see what you get at the end of that.  Got something valuable? Perhaps continue forward (and perhaps now introduce estimation); what you got isn’t valuable? Then you can use the knowledge you gained to decide to continue or not (and perhaps add in estimation or not).  You can use the knowledge you have to make a decision.

At least those are the cases I have for when I would go a #noestimates route… What are yours?

I’m interested in exploring each side; if this interests you, I hope you will consider joining me at the first Agile Dialogs unconference I am putting together.

Agile Coach Camp US – Neat Learnings

I attended several sessions at Agile Coach Camp; I was really impressed by the topics proposed this year. I went to some on Business/Organizational Agility, improving feedback/listening skills, one on creating Joy at work, and several related to using games to teach various Agile concepts. I’ll have to admit, I got lighter on the subject matter as the Camp wore on… Anyone that knows me usually knows I have no fear in proposing 2-3 topics.  This year I proposed none.  I was a bit too dain bread to host one given all the distractions and effort that went into running the Camp itself.

Before I jump into my key learnings/highlights, I was very glad to see one of the emerging themes be one of invitation over imposition. So many organizations are now jumping onto the Agile bandwagon and imposing Agile from above as opposed to helping it emerge; and then we wonder why there is resistance! I also really liked that there was good discussion on various technical topics as well; I often feel these get forgotten.  It’s important for us as a coaching community to understood how we can help organizations adopt things that matter and for software development they ummm… seem… to be technical in nature.

So my highlights; I would be remiss if I did not say one highlight was our extremely energetic facilitator Trica Chirumbole.  I think she brought a great energy to the Camp form opening to closing circles.

I was glad that my first session was one that Ryan Ripley ran to clear up some of the misperceptions people have about why an organization should adopt Agile. We seemed to come up with some great clarifying points to help our organizations or clients understand what to expect as an end result as well as various interim improvements to expect along their journey. Here were some of the key take aways:

  • a focus on improving organizational adaptability/responsiveness
  • use of data to make decisions, but not without regard of what the organization’s people will be undertaking
  • more transparency into organizational performance; risks more visible so better decisions can be made
  • better trust within the organization
  • containing failure and learning from it
  • improved employee engagement and retention

The title of the session was it’s NOT about being Better, Faster, Cheaper; though we rearranged it to mean this by stating: Better = more predictability and customer-focus, Faster = is time to market, not just meeting a schedule, and Cheaper = a focus on producing more value, but not reducing costs.  The hard part we found for measuring organizational performance on these is few organizations have a baseline measurement for any of them; in fact we came up with the hashtag #nobaseline to tweet about these instances. Reminding me I could use that with my current client 🙂

Ryan later ran a follow-in discussion from a session we had in the Open Jam session of Path to Agility in Columbus on creating Joy at work.  It was a complementary session to the earlier session as it focused on the human aspects of making those aspects happen. Since we had a new crowd, we really spent a third of the session kind of bringing them up to speed on our thoughts (at least it felt that way). I have an earlier post to help you. Once there though, we explored why Joy was more important than happiness though several people still thought they were synynomous.  Quite a bit of the conversation focused on how NOT imposing choices on people (what Daniel Pink would refer to as Autonomy) is key to this.  Some other also had it relating towards accomplishment (there’s Mastery) towards a purpose. I mentioned that I like Jurgen Appelo’s CHAMPFROGS; it feels more complete.  Since then, after reading Frédéric Laloux’s book, Reinventing Organizations, I might also say Joy is the integral of Wholeness from time = 0 to the present.  I still also stand by our earlier equation as well from Path to Agility.

I’m going to go quick over some of the rest as I feel I have been rambling a bit; I went to a games session hosted by Declan Whelan and George Dinwiddie on games they had come across or developed.  Declan presented Tom Grant’s tech debt game; everyone played it different and got results that demonstrated WHY we should make investments into things like automated testing and continuous integration. George showcased a game that he has been slowly evolving to show how refactoring works – it more demonstrated how software is malleable and we should treat it as such.  This is of course on its own very valuable.

I attended two other sessions I want to highlight, also both ‘games’-oriented: Mark Sheffield held sort of a games round-up.  I learned several new games to research and variants of games that would prove useful for helping teams and managers understand things better.  Andrew Annett ran a session on the Empathy Toy, which is all about common cognitive empathy (aka developing shared mental models).  This toy is fantastic, every coach should have to play this – you are always trying to find ways to bridge the gap in understanding.  My cohort Ken Furlong and i are already developing new ways to use it.

We had 2 happy hours before and during Camp as well as some food shared in various locations – it was awesome catching up with Diana Larsen, Daniel Mezick, Aaron and Brian Kopel, Jeremy Willets, Kevin Goff, faye Thompson, Declan Whelan, Tim Ottinger, and Ellen Grove at length (during Agile2015, I also had the chance to spend some time with my friends Woody Zuill, Pawel Brodzinski, and Chuck Suscheck at length too).