March 31, 2008 – Estimating Lessons Learned – Communicating the Results

Back in December my 1997 Plymouth Neon was totaled when someone rear-ended me, forcing my car into the next one. It is an odd feeling to look into your rearview mirror and know that the car behind you isn’t stopping.

The insurance company sent me a check for a whopping $2,545. To quantify that massive amount, they compiled an 18 page document comparing my car to similar vehicles using mileage adjustment, feature assessments and other data. All that information was their justification of my car’s estimated value.

Lesson 13 – Tell them where you got the numbers.

An estimate is a guess based on the approach, assumptions and constraints that make up your understanding of the project. Theoretically your team’s estimate has some basis in reality. The key is to communicate that reality to your key stakeholders without them asking what color the sky is on your planet. In Lesson 10 we said to “Document your reasoning.” Let’s look closer at what that means.

Begin with the planned approach for the project. Are you cloning an existing application or starting on a new platform? Will there be a proof of concept performed? Is a pilot phase part of the process? Will it be a “big bang” conversion or phased in over time? Laying out the approach sets the foundation for the numbers. Keep track of any concepts that don’t make it and why they were dropped for future reference.

For constraints, any discussion or thought process that includes can’t, shouldn’t, must, or similar words needs to be recorded and confirmed. If you are saying “we can’t X if they don’t Y” or “we must have Z” then put it down on paper and bring it up.

Assumptions are more difficult to call out. In our minds we see them as obviously true. The hard part is pulling them out for examination and making sure everyone agrees. These can be negative assumptions (any discussion or thought process that ends “we’re not doing that”) or positive (“Marketing will surely have the brochures ready by then”). You may assume they are out of scope but unless it is stated there’s the chance others may think it is in.

Document these and you won’t end up looking at your estimate and saying “what were we thinking?”

Lesson 14 – Email doesn’t cut it.

Schedule a meeting and present your estimate. Sending out an email and expecting a meaningful response isn’t going to cut it. Either face to face or via an on-line conference, you need to walk it through. Explain the Type of Estimate (link) and your confidence level. Recap the estimating process used.

This is your chance to review and verify the approach, assumptions and constraints used to generate the estimate. Make sure everyone is heading in the same direction. Save time and energy by identifying the differences and adjust course earlier in the process.

Had my insurance company simply sent me a check for my Neon, I would have been disappointed and resentful, causing me to switch companies. Because the agent walked me through the numbers I understood that they probably gave me more than it was worth.

Lesson 15 – It is still just an estimate.

In the end, however precisely thought out, meticulously defined and well communicated, it is still an estimate. Things change and assumptions are proven untrue, but with a solid estimate as at its base, your project will be off to a good start.

March 24, 2008 – Estimating Lessons Learned – Estimate Creation

No matter which estimating type you use, there are common steps to creating one.

Lesson 6 - Include your Team. Don’t estimate in a vacuum. Using the team sets you up for success on two fronts. First, the estimate will be more accurate. They know the details of what needs to be done, without which you are just pulling a number out of your hat. Second, it builds buy in and commitment. If they are involved in the process they have ownership of the final result.

Lesson 7 - Break it down. The question always asked is “How do you eat an elephant?” Frankly, I don’t believe anyone eats elephants any more. It is probably a cholesterol thing. How do you get in shape is a different animal. Reduce it to chunks: Diet, Exercise, and Support. Then break those into pieces. Diet - What to eat and what to avoid. Exercise – Classes, weights, aerobics, swimming or walking. Support – find a group and meet regularly. Broken down you can wrap your mind around it.

Traditionally projects create a Work Breakdown Structure (WBS) to identify the work products. Another approach is using Use Cases to drill to specific stories. However you do it, chop the big chunks into smaller units that can be more accurately estimated.

Lesson 8 – Mix and match estimate types. There is no rule that says you must estimate all pieces the same way. If there is a piece that looks like something you have done before, use the Analogous technique to estimate it. For reports the Parametric type may be appropriate.

Lesson 9 – Ask the right questions. One of my previous employers estimated a project to complete a previously implemented system across an enterprise. The question they failed to ask? “Does it work in the area it was first implemented?” The answer would have been “no” and the estimate would have been considerably higher.

Lesson 10 - Document your reasoning. How you came up with the number is as important as what the number is. We’ll delve into this more next week.

Lesson 11 – Sketch a calendar. Create a sanity check for your estimate. Lay out the high level estimate on a timeline by month. Estimate the number resources you need, when you will need them and for how long. Include the number and type of resources broken down by week. Keep it simple as illustrated in the graph below.

I purposely used Excel and kept the image rough so that it does not convey any semblance to accuracy. Does the resource effort look accurate compared to the numeric estimate? Sometimes this rudimentary method allows you to see it from a different angle and find gaps in your estimation.

Lesson 12 - Add Project Management. Include time for status reporting, timekeeping, status meetings and other items. Be aware of other tasks that Project Managers may be pulled into like steering committees, metrics management and resource management. For larger projects a Project Coordinator or even sub-project leads will be necessary to help manage the effort.

In some respects establishing the actual estimate is learned and perfected as you do more estimates. What are some of the tricks you have picked up over the years?

March 3, 2008 – Estimating Lessons Learned – Side Track

After last week’s blog entitled “Estimating Lessons Learned” I received the following comment:

“On the contrary-publishing an estimate of 1,433 hours may be entirely appropriate, unless there's a general disclaimer and/or policy noting that all estimated in a named range are rounded up to the "developer day" (five or six hours) or to a calendar week.

Generally I avoid spontaneous schedule adjustments-over time, they'll distort the schedule and play havoc with historical analysis of estimating accuracy. Any moderately quantitative sponsor or executive stakeholder should question estimates that always end in "0", because it signals subjectivity in the estimating process, and that's something we don't want.” – posted by "badge number ?"

I don’t have time to write a full blog this week (finalizing a presentation for PSQT’s Las Vegas conference), but I do want to respond to this line of thinking.

Let me begin with the last statement: “...because it signals subjectivity in the estimating process.” If you work in an industry that is able to do quantitative analysis and produce a spot on estimate, go for it. The construction industry has a well established, quantitative estimating practice based on years of historical data. However, by definition (http://www.dictionary.com/) an estimate is “to form an approximate judgment or opinion regarding the worth, amount, size, weight, etc., of; calculate approximately.” The use of the words “approximate” and “opinion” denotes subjectivity.

Having said that let me agree with the reader’s comment on two counts. First, “publishing an estimate of 1,433 may be entirely appropriate.” It certainly is when you are putting together a Definitive Estimate. It is not true for a rough estimate when you lack the necessary information to be that accurate. If you can support the detailed estimate, give it to them. Otherwise, don’t give them a number that they will assume is more accurate than it is.

Second, “avoid spontaneous schedule adjustments.” They are bad. I have never advocated them and didn’t in my entry. There are key points throughout your project that you will have significantly more information. At these junctures it is important to re-evaluate the reality of your estimates and communicate any changes to management.

As we will see over the next couple of weeks, that communication begins by clearly stating the logic behind the original estimate with the estimating assumptions made to get there. When re-working the estimate later in the project, those assumptions are may not be true any longer. If the basis for your estimate changed you have a responsibility to communicate the resulting change in your estimate to management.

February 26, 2008 – Estimating Lessons Learned - Intro

“How long to add text messages to the data transfer communication link?” the director asked.

The project manager was obviously hesitant to give an answer. “We haven’t really considered it yet. Why do you ask?”

“I’m heading to a meeting and the question may come up.”

“I would need more time to think it through completely but just for discussion purpose, I would ball park the figure at roughly $50,000 and a 6 month effort. Don’t hold me to it, though.”

Unfortunately for the PM that was the number that ended up on the IT budget for the next year. It started as little more than a wild guess and suddenly became a commitment. Lesson 1 – never give an estimate over the phone.

Types of Estimates. There are many different names for estimates: Rough, Ball Park, SWAG, Big Picture, Budgetary and Final to name a few. Each name attempts to convey the accuracy of the figure. All of them fall on the continuum between three points: Rough Order of Magnitude, Budgetary and Definitive.

The accuracy of the Rough Order of Magnitude (ROM) Estimate ranges from -25% to +75% off. That means a $100,000 estimate can be as low as $75,000 or as high as $175,000. It is the type of estimate usually given when very little is known about the project but management needs to decide whether to move forward or not.

A Budgetary Estimate is put together at the time the Project Charter or Statement of Work is defined. At this point enough is know about what is in and out of scope that boundaries can be established. The accuracy range is between -10% and +25% ($90K and $125K for our $100K project).

The Definitive Estimate range is between -5% and +10% ($95K and $110). This estimate is done based on detailed project information that, in IT, comes after the Requirements or Design phases.

Lesson 2 – not all estimates are created equal…on purpose. This is a fact that needs to be communicated to management and business a like. Set their expectations up front. When they request an estimate, ask what type they are expecting and how they are going to use it. Explain the range of accuracy and the level of effort as well as information required to make it more accurate.

Lesson 3 – don’t confuse people with artificially accurate estimates. By announcing your “rough estimate is $1,633,284.16” you are implying that every paper clip is already counted. Estimates should end in zeros. Past PMs I have worked with issue initial estimates like 1433 hours. This accuracy is impossible even if your scope is well defined and resources are already assigned in your detailed project schedule at the task level. At least round the number to 1450 if not 1500. Ending in a 3 implies precision that doesn’t exist.

Next week we will look at estimating methods and learn that technique is only half the issue.

March 2, 2007 – Deliverable-based Project Schedules: Part 5

With all the tasks defined and even divided up into deliverable many project managers would be off and running. Truth be told, projects can be successful with the checklist approach. The trouble is a checklist does not allow you to predict when deliverables will be completed, report on the true cost of the project or understand and impact the critical path. I suggest we keep pushing forward, then.

Determine Predecessors. When you are setting up your project schedule, enter the logical predecessors. These are the tasks that legitimately belong linked, like creating the document before you review it and ordering the hardware before you receive and install it. Fake predecessors are based on resource availability or just a random decision because something has to go first. These have their place throughout the project, but not when you are initially setting up the schedule.

The more tasks you can link with predecessors the better your scheduling tool can… well… schedule.

Estimate the Work. You may have noticed we haven’t assigned any resources yet. As you estimate the work for each task, think in terms of 1 person doing the work on that task uninterrupted. So, even if you anticipate it would take 2 people a week to complete a document, set the Work value to 80 hrs. Later we will add the specific resources.

While you are assigning the Work, you are probably thinking of a certain skill level or specific individual. These thoughts influence the estimate. Because of this you should document them as estimating assumptions and include them as part of your proposed schedule. No, this isn’t a fancy word for excuses. If you are expecting an expert Java programmer and end up with a novice, your assumptions were wrong and therefore your estimate will be off. With your estimating assumptions outlined you can recognize the impact and make adjustments accordingly (i.e. more or different resources, additional training, obtain more time, etc.).