Sunday, March 30, 2008

March 31, 2008 – Estimating Lessons Learned – Communicating the Results

Back in December my 1997 Plymouth Neon was totaled when someone rear-ended me, forcing my car into the next one. It is an odd feeling to look into your rearview mirror and know that the car behind you isn’t stopping.

The insurance company sent me a check for a whopping $2,545. To quantify that massive amount, they compiled an 18 page document comparing my car to similar vehicles using mileage adjustment, feature assessments and other data. All that information was their justification of my car’s estimated value.

Lesson 13 – Tell them where you got the numbers.

An estimate is a guess based on the approach, assumptions and constraints that make up your understanding of the project. Theoretically your team’s estimate has some basis in reality. The key is to communicate that reality to your key stakeholders without them asking what color the sky is on your planet. In Lesson 10 we said to “Document your reasoning.” Let’s look closer at what that means.

Begin with the planned approach for the project. Are you cloning an existing application or starting on a new platform? Will there be a proof of concept performed? Is a pilot phase part of the process? Will it be a “big bang” conversion or phased in over time? Laying out the approach sets the foundation for the numbers. Keep track of any concepts that don’t make it and why they were dropped for future reference.

For constraints, any discussion or thought process that includes can’t, shouldn’t, must, or similar words needs to be recorded and confirmed. If you are saying “we can’t X if they don’t Y” or “we must have Z” then put it down on paper and bring it up.

Assumptions are more difficult to call out. In our minds we see them as obviously true. The hard part is pulling them out for examination and making sure everyone agrees. These can be negative assumptions (any discussion or thought process that ends “we’re not doing that”) or positive (“Marketing will surely have the brochures ready by then”). You may assume they are out of scope but unless it is stated there’s the chance others may think it is in.

Document these and you won’t end up looking at your estimate and saying “what were we thinking?”

Lesson 14 – Email doesn’t cut it.

Schedule a meeting and present your estimate. Sending out an email and expecting a meaningful response isn’t going to cut it. Either face to face or via an on-line conference, you need to walk it through. Explain the Type of Estimate (link) and your confidence level. Recap the estimating process used.

This is your chance to review and verify the approach, assumptions and constraints used to generate the estimate. Make sure everyone is heading in the same direction. Save time and energy by identifying the differences and adjust course earlier in the process.

Had my insurance company simply sent me a check for my Neon, I would have been disappointed and resentful, causing me to switch companies. Because the agent walked me through the numbers I understood that they probably gave me more than it was worth.

Lesson 15 – It is still just an estimate.

In the end, however precisely thought out, meticulously defined and well communicated, it is still an estimate. Things change and assumptions are proven untrue, but with a solid estimate as at its base, your project will be off to a good start.

Sunday, March 23, 2008

March 24, 2008 – Estimating Lessons Learned – Estimate Creation

No matter which estimating type you use, there are common steps to creating one.

Lesson 6 - Include your Team. Don’t estimate in a vacuum. Using the team sets you up for success on two fronts. First, the estimate will be more accurate. They know the details of what needs to be done, without which you are just pulling a number out of your hat. Second, it builds buy in and commitment. If they are involved in the process they have ownership of the final result.

Lesson 7 - Break it down. The question always asked is “How do you eat an elephant?” Frankly, I don’t believe anyone eats elephants any more. It is probably a cholesterol thing. How do you get in shape is a different animal. Reduce it to chunks: Diet, Exercise, and Support. Then break those into pieces. Diet - What to eat and what to avoid. Exercise – Classes, weights, aerobics, swimming or walking. Support – find a group and meet regularly. Broken down you can wrap your mind around it.

Traditionally projects create a Work Breakdown Structure (WBS) to identify the work products. Another approach is using Use Cases to drill to specific stories. However you do it, chop the big chunks into smaller units that can be more accurately estimated.

Lesson 8 – Mix and match estimate types. There is no rule that says you must estimate all pieces the same way. If there is a piece that looks like something you have done before, use the Analogous technique to estimate it. For reports the Parametric type may be appropriate.

Lesson 9 – Ask the right questions. One of my previous employers estimated a project to complete a previously implemented system across an enterprise. The question they failed to ask? “Does it work in the area it was first implemented?” The answer would have been “no” and the estimate would have been considerably higher.

Lesson 10 - Document your reasoning. How you came up with the number is as important as what the number is. We’ll delve into this more next week.

Lesson 11 – Sketch a calendar. Create a sanity check for your estimate. Lay out the high level estimate on a timeline by month. Estimate the number resources you need, when you will need them and for how long. Include the number and type of resources broken down by week. Keep it simple as illustrated in the graph below.

I purposely used Excel and kept the image rough so that it does not convey any semblance to accuracy. Does the resource effort look accurate compared to the numeric estimate? Sometimes this rudimentary method allows you to see it from a different angle and find gaps in your estimation.

Lesson 12 - Add Project Management. Include time for status reporting, timekeeping, status meetings and other items. Be aware of other tasks that Project Managers may be pulled into like steering committees, metrics management and resource management. For larger projects a Project Coordinator or even sub-project leads will be necessary to help manage the effort.

In some respects establishing the actual estimate is learned and perfected as you do more estimates. What are some of the tricks you have picked up over the years?

Tuesday, March 18, 2008

March 18, 2008 - Estimating Lessons Learned - Agile Methods

One of the most pathetic looking sights on the highway is a man walking, head down, with a gas can. He represents the ultimate in poor estimating practices. It is bad enough that he ran out of fuel, but he was probably already late or he would have stopped to get gas. Having the embarrassment of trekking back to the car only adds to his dilemmas.

One of the misperceptions of Agile development is that the team continues to code until they “run out of gas” by hitting the target date.

Lesson 5 – Agile development requires estimation.

Although non-traditional methods differ in their techniques, they still rely on estimates to plan for and meet projected dead lines. We’ll look at three different types: Predictive Modeling, Time Boxing and Planning Poker.

In order to be effective, each of these estimating types requires a prioritized list of functionality and features that define the scope of the project. SCRUM estimates items on a Product Backlog. In the Rational Unified Process (RUP) Use Cases are identified and estimated at a high level (ex. 2 days for A and 5 days for B). XP uses Stories. The difference lies in the way the estimates are determined.

Predictive Modeling involves breaking the product into pieces and assigning relative work units to each one. For example, you may assign feature X 5 units and feature Y 10 units because Y is 2 times bigger or more complex than X. Initially you predict the duration for the first set of features and extrapolate the full project effort based on those numbers. Upon completion of the first set you have a better idea of how long it will take to complete the next, allowing you can refine your prediction. As each piece is completed you can more accurately estimate the remainder of the project.

Time Boxing is based on the concept of set periods of duration, anywhere from 1 to 6 weeks, for each release. Based on their size, complexity and dependencies, Use Cases (or features) are scheduled for specific releases throughout the project. At the end of each release the next set of Use Cases is re-evaluated to verify that they are the right ones and that they can still be completed in the timeframe. If a trend begins to appear showing that the team is consistently falling behind, the overall project schedule will require re-evaluated.

Planning Poker is a variation on the Wideband Delphi technique. In the 1970s Barry Boehm and John A. Farquhar expanded the US Armed Forces’ Delphi method to use multiple experts giving anonymous estimates for pieces of a project. Those estimates would be reviewed and the pieces with big discrepancies would be discussed in meeting and re-estimated.

As with every other aspect of development, Agile decided to speed up the process. In a Planning Poker session, team members are given cards with numbers on them, representing work effort (i.e. 3, 5, 10, 20 hours). Individual stories are reviewed and discussed by the team. Each member then selects a number from their deck that corresponds with their estimate. Only after everyone has picked a number do they show their estimates. If the numbers are close together, the value stands. When there is a large difference, additional discussion is held and another secret estimate is held.

Each of these estimating types has their strengths and weaknesses. Sometimes it is best to use a different technique for different parts of the project. The key will be how you explain and communicate the numbers to the project stakeholders. We’ll cover that next time.

Sunday, March 9, 2008

March 10, 2008 – Estimating Lessons Learned - Traditional

Estimating is probably the hardest part of any project. My wife will attest that my ability to estimate when I will leave work is inversely proportional to the importance of me getting home at a specific time. The plan will be for me to only work until noon the day we leave for vacation. At 2:00 I will still need to finish my status report, change my voice message and set my out-of-office email. Maybe I just need a better estimating technique.

Lesson 4 - There are many different ways to come up with an estimate. This week we will take a look at the more traditional methods: Analogous / Top Down, Parametric, Bottom Up and Work Distribution. Next time we will discuss a some of the newer methods like Time Boxing and Wideband Delphi.

Analogous (Top Down). When a project is first considered, the quickest ways to develop an estimate is to compare it with one from your past. The question then becomes: Is it half as big or 5 times bigger? The focus is on what is different from the previous project. This technique requires the least amount of detail and gives the widest margin of uncertainty. This typically regulates it to Rough Order of Magnitude estimates.

Parametric. Normally associated with construction, Parametric estimates are based on historical data. For building a house a contractor may price it at $103 dollars per square foot. The calculation is built from years of completing similar jobs and calibrated with the current cost of materials. This is similar to the use of Function Point analysis in software development. A Function Point is a single unit of business functionality. The cost (in dollars or hours) of a single unit is calculated from past projects. New projects are defined by the number of function points and the math is done to create an overall estimate. This can be effective for features of similar sizes like reports or system interfaces. If historically it takes 40 hours to design, create, build and implement a report, 10 reports is going to take 400 hours.

Bottom Up. This form of estimate requires a more detailed analysis of the project. The Work Breakdown Structure (WBS) is defined further to identify the tasks associated with the work products. Each task is estimated and the cost for all of the individual pieces is combined to create the overall total. Using scheduling software like Primavera and Microsoft Project helps to define, record and plan the estimate. This is the most accurate type of estimate, but it requires a level of detail usually only known after the requirements are fully defined.

Work Distribution. Somewhere between Bottom Up and Parametric is the Work Distribution method. Usually the development phase of the project is estimated at some level of detail and the time for the other phases is based on a percentage of the project. The percentages are specific to the organization and project team performing the work based on their past performance. Generic numbers could be:
Phase %
Initiation 5
Requirements 15
Design 10
Construction 30
Testing 20
Implementation 5
Project Mgmt 15

The development team estimates the construction effort and the other hours are calculated based on that amount.

Sunday, March 2, 2008

March 3, 2008 – Estimating Lessons Learned – Side Track

After last week’s blog entitled “Estimating Lessons Learned” I received the following comment:

“On the contrary-publishing an estimate of 1,433 hours may be entirely appropriate, unless there's a general disclaimer and/or policy noting that all estimated in a named range are rounded up to the "developer day" (five or six hours) or to a calendar week.

Generally I avoid spontaneous schedule adjustments-over time, they'll distort the schedule and play havoc with historical analysis of estimating accuracy. Any moderately quantitative sponsor or executive stakeholder should question estimates that always end in "0", because it signals subjectivity in the estimating process, and that's something we don't want.”
– posted by "badge number ?"

I don’t have time to write a full blog this week (finalizing a presentation for PSQT’s Las Vegas conference), but I do want to respond to this line of thinking.

Let me begin with the last statement: “...because it signals subjectivity in the estimating process.” If you work in an industry that is able to do quantitative analysis and produce a spot on estimate, go for it. The construction industry has a well established, quantitative estimating practice based on years of historical data. However, by definition (http://www.dictionary.com/) an estimate is “to form an approximate judgment or opinion regarding the worth, amount, size, weight, etc., of; calculate approximately.” The use of the words “approximate” and “opinion” denotes subjectivity.

Having said that let me agree with the reader’s comment on two counts. First, “publishing an estimate of 1,433 may be entirely appropriate.” It certainly is when you are putting together a Definitive Estimate. It is not true for a rough estimate when you lack the necessary information to be that accurate. If you can support the detailed estimate, give it to them. Otherwise, don’t give them a number that they will assume is more accurate than it is.

Second, “avoid spontaneous schedule adjustments.” They are bad. I have never advocated them and didn’t in my entry. There are key points throughout your project that you will have significantly more information. At these junctures it is important to re-evaluate the reality of your estimates and communicate any changes to management.

As we will see over the next couple of weeks, that communication begins by clearly stating the logic behind the original estimate with the estimating assumptions made to get there. When re-working the estimate later in the project, those assumptions are may not be true any longer. If the basis for your estimate changed you have a responsibility to communicate the resulting change in your estimate to management.