Estimating

Index

Sources: Construx (extensively) www.construx.com/ External, Sogeti www.uk.sogeti.com/ External, Saltlane.

Software estimation is hard. Upper management, lower management, customers, and some developers don’t seem to understand why estimation is hard. Why? Because software estimation is a process of gradual refinement. You begin with a fuzzy picture of what you want to build and then spend the rest of the project trying to bring that picture into clearer focus. Because the picture of the software you’re trying to build is fuzzy, the estimate of the time and effort needed to build it is fuzzy too. The estimate can come into focus only along with the software itself, which means that software-project estimation is also a process of gradual refinement.

Table of Contents

Estimation is a Process of Refinement

How much does a new house cost? It depends on the house. How much does a new billing system cost? It depends on the billing system. Some organizations want cost estimates to within plus-or-minus 10 percent before they’ll fund work on requirements definition. Although that degree of precision would be nice to have that early in the project, it isn’t even theoretically possible. That early, you will do well to estimate within a factor of 2. The degree of accuracy is heavily influenced by the project type

Until each feature is understood in detail, you can’t estimate the cost of a program precisely. Software development is a process of making increasingly detailed decisions. You refine the product concept into a statement of requirements, the requirements into a preliminary design, the preliminary design into a detailed design, and the detailed design into working code. At each of these stages, you make decisions that affect the project’s ultimate cost and schedule. Because you can’t know how each of these decisions will be made until you actually make them, uncertainty about the nature of the product contributes to uncertainty in the estimate.

Uncertainty about even a single feature can introduce a lot of uncertainty into an early-in-the-project estimate. Multiplied across an entire project, literally thousands of specification, design, and implementation decisions have to be made before the ultimate cost of the project can be determined. As you do make a greater percentage of those decisions, however, you can narrow the estimation range.

Researchers have found that project estimates fall within predictable precisions at various stages of a project.

image

At the time when developers are typically asked to provide a rough estimate, there can be a factor of 16 difference between high and low effort estimates. Even after requirements have been completed, you can only know the amount of effort needed to within about 50 percent, and by that time most organizations want their estimates to the dollar.

Estimate multipliers by project phase

  Effort and Size Schedule
Phase Optimistic Pessimistic Optimistic Pessimistic
Initial product concept 0.25 4.0 0.60 1.60
Approved product concept 0.50 2.0 0.80 1.25
Requirements specification 0.67 1.5 0.85 1.15
Product design specification 0.80 1.25 0.90 1.10
Detailed design specification 0.90 1.10 0.95 1.05

As the data suggests, your estimates throughout a project should start general and become more refined as the project progresses.

To use the factors in the table, simply multiply your “most likely” single-point estimate by the optimistic factor to get the optimistic estimate and by the pessimistic factor to get the pessimistic estimate. You can then present your estimate as a range rather than as a point estimate. If your “most likely” estimate is 50 man-months and you’ve completed the requirements specification, you would estimate a range of 34 to 75 man-months.

Handling your Stakeholders

You need to explain these four points:

Rules of Estimation (from Book by Paul Combs)

  1. Your estimate will be wrong
  2. You can always give an estimate
  3. Every estimate MUST have a contigency allowance
  4. It is harder to identify the list of items to be estimated than it is to estimate them
  5. The quality of the estimate depends on you familiarity with the project
  6. The more bits its in the longer it gets
  7. Write down your assumptions
  8. Contingency you allow is proportional to the risk
  9. There is no method that works
  10. The project duration in months must be greater than the average nbr on the team
  11. Have someone review your estimates
  12. Hold a project post mortem to learn

Estimating Projects

The process of creating an accurate development schedule consists of three steps:

1. Estimate the size of the product (number of lines of code or function points, number of modules, number of products/artefacts, Product Breakdown Structure,etc.). Some projects jump right into estimating the schedule itself, but effective estimation requires estimating the size of the software to be built first. This step is by far the most difficult intellectually, which might be part of the reason that people so often skip it. Most estimates go wrong,not because you have estimated an item incorrectly, but usually becasue you have missed something. The key is to try as many methods as possible to identify all the items.

2. Estimate the effort (man-months). If you have an accurate size estimate and historical data on your organization’s performance on similar projects, computing the effort estimate is easy. Alternatively ask the experts.

3. Estimate the schedule (calendar months). Once you’ve estimated the size and effort, estimating the schedule turns out to be simple. But getting a realistic schedule estimate accepted can be the most difficult part of the project.

Wrapped around these three steps is a more general step:

Provide estimates in ranges and periodically refine the ranges to provide increasing precision as the project progresses. The need for best and worst estimates applies to both critical path and Critical Chain methods.

4. Improve your estimates as you get more information. Recalibrate and refine.

 1. Size Estimation

You can estimate the size of a project in any of several ways:

Most software programs and some of the algorithmic approaches require that you calibrate the estimation practice to your environment before you use it. Accurate measurement of historical projects is a key to long-term success in using any kind of estimation.

Items to count to build up historical information.

2. Effort Estimate

Once you have identified the size, you can move to the second estimation step, deriving the effort estimate. Although not strictly necessary for estimating a software schedule, you’ll need an effort estimate in order to know how many people to put on your project, and having an effort estimate makes it easy to derive the schedule estimate.

Deriving the effort estimate is a straightforward process. Here are some of the ways you can convert a size estimate into an effort estimate:

3. Schedule Estimation

The third step in estimating a software project is to compute the schedule. A rule of thumb is that you can compute the schedule from the size estimate by:

schedule in months = 3.0 * man-months^1/3

where Man Months is defined as Man Months = (Project Work Effort (Hours)) / (Hours per Month)

If you’ve estimated that it will take 65 man-months to build the project, then the optimal schedule is 12 months (3.0 * 65^1/3). That in turn implies an optimal team size of 65 man-months divided by the 12-month schedules—5 or 6 team members.

This equation is the reason that the estimation ranges  are broader for efforts than they are for schedules. Bigger projects take longer, but they also have bigger teams; and the bigger team size means that effort increases disproportionately faster than schedule. The schedule ranges  assume that you will add or remove team members as you learn more about the scope of the project. If you hold the team size constant, the schedule ranges will be just as large as the effort ranges.

Optimum Staffing Size is given by the following formula. The origin of this formula is currently unknown, but it goes like this:

Optimum Staffing Size = Square Root of Man Months

Jones’s First-Order Estimation Practice

If you have a function-point count, you can compute a rough schedule directly from that using a practice that Capers Jones has described as “first-order estimation.” To use it, you simply take the function-point total and raise it to the appropriate power :

Kind of software Best in class Average Worst in class
Systems 0.43 0.45 0.48
Business 0.41 0.43 0.46
Shrink-wrap 0.39 0.42 0.45

If you estimated your project’s total number of function points to be 350 and you’re working in an average shrink-wrap software organization, you would raise 350 to the 0.42 power (350^0.42), for a rough schedule of 12 calendar months. If you were working in a best-in-class shrink-wrap organization, you would raise 350 to the 0.39 power, for a schedule of 10 months.

This practice isn’t a substitute for more careful schedule estimation, but it does provide a simple means of getting a rough schedule estimate that’s better than guessing. It can also provide a quick reality check. If you want to develop a 350-function-point shrink-wrap product in 8 months, you should think again. The best-in-class schedule would be 10 months, and most organizations aren’t best in class. Jones’s First-Order Estimation practice allows you to know early on if you need to adjust your feature set, schedule expectations, or both.

4. Estimate Refinement

One question that managers and customers ask is, “If I give you another week to work on your estimate, can you refine it so that it contains less uncertainty?” That’s a reasonable request, but unfortunately it’s not possible to act on it the way you’d like to. Research by Luiz Laranjeira suggests that the accuracy of the software estimate depends on the level of refinement of the software’s definition (Laranjeira 1990). The more refined the definition, the more accurate the estimate. This makes sense intuitively because the more the system is nailed down, the less uncertainty feeds into the estimate.

It is simply not possible to know a schedule with ±10-percent precision at the requirements-specification stage. You can control it to come out on the low side, but if you just let the project flow wherever it wants to go, you’ll get no better than +50 percent, -33 percent precision.

This why it is good to provide estimates as ranges:

Point in project Estimate(man-months) 
Initial product concept    25 - 400
Approved product concept   50 - 200
Requirements specification    90 - 200
Product design specification     120 - 180 
Detailed design specification       145 - 180 
Final     170  

 The imprecision isn’t a sign of a bad estimate; it’s part of the nature of software development. Failure to acknowledge imprecision is a sign of a bad estimate.

Rather than losing the customer’s confidence by taking one schedule slip after another, the customer’s expectations are met if you use ranges.

Recalibration

Suppose that you have a six-month schedule. You planned to meet your first milestone in four weeks, but it actually takes you five weeks. When you miss a scheduled date, there is a question about how to recalibrate the schedule. Should you:

1.            Assume you can make up the lost week later in the schedule?

2.            Add the week to the total schedule?

3.            Multiply the whole schedule by the magnitude of the slip, in this case by 25 percent?

The most common approach is #1. The reasoning typically goes like this: “Requirements took a little longer than we expected, but now they’re solid, so we’re bound to save time later. We’ll make up the shortfall during coding and testing.”

A 1991 survey of over 300 projects found that projects hardly ever make up lost time—they tend to get further behind (van Genuchten 1991). That eliminates option #1.

Estimation errors tend to be inaccurate for systemic reasons that pervade the whole schedule, such as pressure from management to use optimistic assumptions. It’s unlikely that the whole schedule is accurate except for the part that you’ve had real experience with. So that eliminates option #2.

With rare exception, the correct response to a missed milestone is option #3. That option makes the most sense analytically. If you aren’t ready to extend the schedule by that amount, and people often aren’t, then you can delay your decision and get more data by monitoring how you do in meeting the second milestone. But if you’re still off by 25 percent in meeting the second milestone, it will be harder for you to take corrective action, and the corrective action you do take will have less time to work than it would if you took corrective action at the first available opportunity.

Changing the estimate after missing or beating a milestone isn’t the only option, of course. You can change the “product” corner of the schedule/product/cost triangle, or you can change the “cost” corner. You can change the spec to save time, or you can spend more money. The only thing not to do is to keep the schedule, the spec, and the cost the same and expect the project to improve.

But if you’ve described ahead of time how your estimate will mature as the project matures and given your estimates as ranges, there will be fewer slips. As long as you’re in the range, you’re doing fine. A “slip” becomes something that’s completely out of the estimation range, and that becomes a rare occurrence.

Estimating Tips

Estimating Tips


Estimating Testing Effort

One method for testing and estimating test effort is TMap. en.wikipedia.org/wiki/test_management_approach External

Test estimation can done at several levels:

Master test plan

Estimate at this level is usually done early in the project. Information on scenarios is often not available. It is generally high level estimate based on number of people for an estimated duration sourced from projects of similar size and complexity (estimate based on size).

Test Phase

By the time the test phases are known, there is usually more information available. An estimate can be made on size and complexity compared with other projects or on ratios.

Ratio rules of thumb

For a given test phase: Preparation (8%), Specification (32%), Execution (40%), Completion (5%), Management overhead (15%)

Design = Analysis and Design Build= Build and Unit test Test = All other testing (System, Integration and UAT etc.)

Design:Build:Test = 2:5:3

Test scenario

Once the test scenarios have been identified, we can use more parametric estimation methods. Test point analysis(TPA) is similar to function point analysis.

Logical Complexity
Tables Simple Medium Complex
<10 25 28 35
10-25 28 35 42
>25 35 42 47
1K lines TPs
C 7
Cobol, Fortran 10
PL1 12
Basic Lisp 16
4GL 25
Smalltalk 49
Query language 60

Like Function points, the test point total is then adjusted as follows:

Productivity factor (P) The productivity factor indicates the number of test hours required per test point. The higher the productivity factor, the greater the number of test hours required. The productivity factor is a measure of the experience, knowledge and skill of the test team. The productivity factor can vary from one organization to the next or from one organizational unit to the next. Productivity factors can be calculated by analysing historical data. In practice the productivity factor has shown to have a value between 0,7 and 2.0. As a rough guide: Very experienced team (0.7), normal team (1.25), inexperienced team (2.0)

Environmental factor (E) The number of test hours required for each test point is influenced not only by the productivity factor, but also by the environmental factor. A number of environmental variables are defined for calculation of the environmental factor. If insufficient information is available to enable rating of a given variable, the nominal rating (printed bold) should be assigned.

Item Rating
Test tools 1 - Testing involves the use of a query language such as SQL; a record and playback tool is also being used. 2 Testing involves the use of a query language such as SQL, but no record and playback tool is being used. 4 No test tools are available.
Development testing 2 A development testing plan is available and the test team is familiar with the actual test cases and test results 4 A development testing plan is available. 8 No development testing plan is available.
Test basis 3 During the system development documentation standards are being used and a template, in addition the inspections are organized 6 During the system development documentation standards are being used and a template. 12 The system documentation was not developed using a specific standards and a template.
Development environment 2 The system was developed using a 4 GL programming language with an integrated DBMS containing numerous constraints. 4 The system was developed using a 4 GL programming language, possibly in combination with a 3 GL programming language. 8 The system was developed using only a 3 GL programming language such as COBOL,PASCAL or RPG.
Test environment 1 The environment has been used for testing several times in the past. 2 The test is to be conducted in a newly equipped environment similar to other well-used environments within the organization. 4 The test is to be conducted in a newly equipped environment which may be considered experimental within the organization.
Testware 1 A usable general initial data set (tables, etc.) and specified test cases are available for the test. 2 A usable general initial data set (tables, etc.) is available for the test. 4 No usable testware is available.

The environmental factor is calculated by adding together the ratings for the environmental variables above, then dividing the sum by 21.

Management overhead (M) Management overhead is typically between 10% and 15%. (factor 1.1 to 1.15) Value depends on:

Total adjusted test points is then:

Adjusted TP = TP * P * E * M

TP = test points from above. P = Productivity factor E = Environmental factor M = Management factor

To convert this into hours, 1 adjusted TP = 2 hours test effort. Adjusted TP * 2 = Total test effort

Test Activity

At this stage most test scripts have been identified. Here a bottom up estimate is done from each individual test script where estimates are supplied by the testers.


25 January 2009 (18)