Read an Excerpt

Monitoring the Progress of Shipbuilding Programmes

Rand Corporation

Chapter One

The Defence Procurement Agency (DPA), part of the UK Ministry of Defence (MOD), provides services and equipment necessary for the security of the United Kingdom. Although the agency does not generate a positive cash flow, as does a private venture, it still must make efficient and effective use of its capital. Defence procurement is expensive, and DPA expenditures, like those for any organisation, are constrained by available funding. Only so many financial resources can be spent on defence, since there are many competing funding issues the government faces. The DPA must not only be selective in the programmes it funds but also procure items efficiently. Efficient capital expenditure requires good programme management.

Accurate estimating and cost and schedule control are central competencies of programme management. Accurate estimating allows an organisation to make the best use of limited funding. Consistently underestimating cost or schedule needs can lead to cash flow problems and possible programme cancellations. Overestimating these needs reduces capital available for additional programmes. The DPA has recognised the importance of estimating and control by focusing two of its five key targets on costand schedule performance. Controlling cost and schedule is the principal focus of this report. Monitoring cost and schedule is important for both operational planning (determining the product delivery date) and financial management (determining the cash flow needed to support the programme). A good control system can also aid programme improvement by helping identify problem areas.

This report focuses on schedule and progress metrics while recognising there is some overlap with cost control metrics. In practice, these issues are intertwined and cannot be easily isolated from one another.

Major UK Defence Acquisitions Are Typically Behind Schedule

The DPA gauges its annual performance against five key targets for its top 20 projects, which are measured by value and documented in the agency's Major Project Reports. Key Target 2 addresses programme slippage, the delay between the promised dates and actual or projected in-service dates. The MOD indicates that average slippage for its top 20 projects is approximately one year (United Kingdom Ministry of Defence, 2002a) beyond the delivery date approved at Main Gate. Because the Major Project Reports cover all areas of DPA programmes, the indicated slippage includes more than just shipbuilding programmes, the focus of this report. Nevertheless, recent delays for programmes such as the Landing Platform Dock, Astute, and Auxiliary Oiler indicate that slippage does occur in shipbuilding. Average in-year slippage for these major projects, i.e., that occurring within the annual reporting period, was 1.1 months, compared with a target of 0.4 months (MOD, 2003).

DPA Often Does Not Realise Projects Are Behind Schedule Until Late in the Production Cycle

Some slippage shows up early, but the bulk of it is often not recognised until the latter stages of the procurement cycle (NAO, 2002). Overall, schedule (or time) variance occurs very early, levels off midway through the procurement cycle, and increases again late in the cycle (see Figure 1.1). Cost variance, conversely, does not occur early, increases at the greatest rate midway through the procurement cycle, and levels off late in the cycle.

There are many possible explanations for these patterns. The early increase in schedule variance could be due to optimism being supplanted by better project definition and more realistic expectations, or after the project is approved and the contractor faces less pressure to 'look good'. The later schedule variance may be a result of technical and integration issues that typically do not surface until the end of the procurement cycle. In addition, changes or growth in cost may be easier to quantify than those for scheduling and thus are recognised earlier.

Commercial Ships Are Typically Produced On Time

Commercial shipbuilding has much better schedule performance (see Figure 1.2). Asked about schedule performance, representatives of commercial firms regularly told us, 'We are never late'. In fact, one of the commercial shipbuilders we surveyed had only once delivered a ship after the contract delivery date since 1985. This less-than-one-month slippage was a result of damage during transport of a long lead item.

To be sure, there are substantial differences between commercial and military shipbuilding. For example, commercial ships are typically built from a well-established design, while military ships are more unique. Nevertheless, the management of change in particular, in both processes and in other industries, may offer some insights for those interested in reducing slippage.

Areas of Inquiry

The issue of programme slippage and the inability to recognise delays early in the programme led the DPA to ask the RAND Corporation to

assess how shipbuilders, and other industries, track programme progress and how they identify a set of metrics that are used to measure progress

consider how the DPA should monitor programmes and recommend the types of information topics that should be gathered from shipbuilders to enable the agency to independently assess shipbuilding progress

identify why ships are delivered late and understand why commercial shipbuilders have much better schedule performance.

Answering these questions requires a broad perspective, much more than a simple focus on metrics. Accordingly, we address other areas of management and production practice, particularly programme management. The ability to track progress ('progress reporting') and estimate schedules is important in identifying and correcting problems or shortfalls as programmes proceed. Being able to manage, limit, and efficiently handle change is important in keeping a programme on time and budget. The ability to forecast progress and schedule needs can be the most difficult aspect of keeping a project on schedule. Finally, incentives and payment strategies can align shipbuilder priorities with those of the DPA.

Realistic baseline expectations can also help avoid later slippage. Unrealistic expectations will cause programme slippage, regardless of the control methods employed, and can create a culture intolerant of critical or honest evaluation of progress. In such circumstances, being the messenger of 'bad news' could be career threatening. Furthermore, control becomes impossible when the baseline expectations are unrealistic.

Knowing where you are with respect to budget and schedule is only part of the problem. More importantly, being able to forecast where you expect to be and how you will get there is another aspect of the often-difficult task of managing progress and schedule.

Methodology

We used a four-part methodology to address our areas of inquiry.

First, we surveyed major shipbuilders in the United Kingdom, United States, and European Union (the survey form is reproduced in Appendix C). The survey included questions about such programme areas as schedule tracking methods, change control, causes of slippage, and forecasting.

Second, after receiving responses to these surveys, we had follow-on conversations with the shipbuilders to better understand the survey information, ensure that the forms had been completed in a consistent manner, and allow the shipbuilders to discuss relevant issues not captured in the survey. Most shipbuilders used this opportunity to show us a sample progress report. From those surveys and discussions, we determined metrics commonly used by shipbuilders. For comparison purposes, we also interviewed representatives from the project management department of a major oil firm in regards to their practices for cost and schedule control.

Third, we reviewed relevant literature to assess the advantages and disadvantages of the metrics we had identified. For example, there are many studies that report on the US Department of Defense's (DoD's) experience with cost and schedule control.

Fourth, we reviewed the primary causes of production delays identified by shipbuilders. Such information may help the DPA not only to monitor progress and anticipate potential delays but to avoid practices that contribute to schedule slippage.

The surveyed firms are a variety of shipbuilders in the United Kingdom, United States, and European Union (see Table 1.1). In the United Kingdom, all the major shipbuilders that produce and repair naval ships and submarines participated in the survey, as did Ferguson Shipbuilders, a producer of commercial and survey vessels. We also discussed schedule control practice with Appledore, a firm that did not formally complete a survey. In the United States, we surveyed most of the 'big six' naval shipbuilders, as well as Kvaerner Philadelphia, a commercial US shipbuilder. EU shipbuilders interviewed comprised cruise-ship builders that provided information on the schedule control methods they employ as well as shipbuilders constructing both commercial and military vessels.

Organisation of This Report

We present our findings in the four subsequent chapters, supplemented by three appendixes. Chapter Two summarises the survey results of how shipbuilders monitor design and production progress. Chapter Three documents information that the DPA should consider tracking to monitor progress. Chapter Four examines the causes of late ship delivery and how they vary for military and commercial shipbuilding. Chapter Five provides overall conclusions and recommendations.

Many of the metrics for monitoring progress are those of earned value management (EVM). Appendix A provides an overview of EVM for those who are unfamiliar with it; Appendix B provides a list of common EVM metrics; and Appendix C reproduces the survey we used to collect data from the shipbuilders.

Chapter Two

The metrics we identified to measure shipbuilding progress can be classified into six categories: earned value related, milestones, task oriented, actual versus planned, area/zone (such as compartment completion), and other (a residual category) (see Table 2.1).

Earned value-related metrics are those associated with EVM (see Appendix A). Earned value metrics compare the budgeted cost of work performed (BCWP) with the budgeted cost of work scheduled (BCWS) at a given point in time. Projects with a BCWP value less than that of the BCWS are considered behind schedule, while those with a BCWP value exceeding the BCWS value are considered ahead of schedule. Essentially, EVM weights cumulative task progress by estimated value.

Milestones are major events in the course of a programme. They range from very high-level events such as first block in dry dock, ship launch or float out, and system light off to those low-level events dealing with specific tasks. One shipbuilder, for example, tracks multiple milestones per structural unit.

Task-oriented metrics are based on the completing or starting of specific tasks or work packages. They are related to EVM methods but don't weight by task value.

Actual-versus-planned metrics are those tracking progress as a ratio of actual results to planned results. They differ from earned value in that, while earned value measures progress relative to that planned, actual-versus-planned metrics focus more generally on actual accomplishments such as cumulative number of hours worked to date compared with estimated total hours needed to complete the job. These types of metrics can be more sensitive than EVM metrics to such issues as performance and estimating errors.

Area/zone, or compartment-completion, metrics focus on areas or zones of the ship. These metrics are generally used towards the end of production.

Other metrics are those we gathered that did not fit into the categories above. Examples of such metrics include available schedule float or contingency and amount of change unadjudicated.

Earned Value Metrics Dominate Primary Progress Measures

We asked the shipbuilders to identify the 'primary' progress metric they used during the various phases of production-that is, their main method of assessing progress. Figure 2.1 shows the amount of shipyards that reported using a metric from one of the six categories. We examined usage during six different phases: design, module block construction, assembly, outfitting, testing/trials, and commissioning. For each phase, we rank primary metrics by their reported prevalence of use and indicate the proportion of shipbuilders that report using them.

The reader should be aware of some qualifications in interpreting reported use of primary metrics. Some shipyards reported using more than one primary metric in a production phase. Not all shipyards reported metrics for all phases; repair shipyards, for example, do not have module block production or assembly activities and therefore did not report metrics for those phases.

Earned value metrics are the most commonly used in all production phases but are less frequently employed in the later phases. Milestone metrics are the second most commonly used.

Earned value metrics are particularly common in US shipyards. In fact, every US shipyard we surveyed uses earned value metrics as its primary metric for evaluating progress in every phase of production (see Figure 2.2). UK and EU shipyards are more likely to use non-EVM metrics such as compartment completion (area/zone) and milestones, particularly towards the end of production. Nevertheless, for every phase of production, UK shipyards are more likely than other European shipyards to use earned value metrics as their primary metric for evaluating progress.

The US Department of Defense Has Embraced Earned Value Management

The reliance of US shipbuilders on earned value metrics stems from the nation's defence policy requiring the use of EVM on most acquisition programmes of more than $315 million. Contractor programme management systems must conform to the American National Standards Institute standards for EVM. The DoD encourages contractors to use the same system and information for EVM that they use for internal programme control. Contractors report their information at least once a month.

EVM data are reported to the DoD in five formats comprising the Cost Performance Report. The first format reports EVM data by the agreed work breakdown structure (WBS). This reporting is typically limited to no more detail than level 3 of the WBS. The second format reports data by the organisation, such as trade areas (e.g., pipe shop), subcontractor, or other organisations. The third format reviews the programme baseline, any contract changes to the baseline during the reporting period, and the value of management reserve. The fourth format outlines changes in the staffing plan, including workforce trends. The fifth lists problem areas (e.g., those with large variances or poor productivity) and management actions being taken to address them.

How Do Shipbuilders Use Information That Metrics Provide?

Internally, shipbuilders typically use EVM information as a basis of control for internal monitoring of progress. Internal reporting frequency ranged from weekly to more common monthly reports. Internal reports usually have a top-level summary (a window pane or dashboard chart) of the aggregate programme data. These summaries include information on budget cost of work scheduled and performed, actual cost of work performed, and cost performance and schedule performance indices. Shipbuilders also report supporting detail by WBS, organisation, and section or zone of ship.

(Continues...)

---

Excerpted from Monitoring the Progress of Shipbuilding Programmes by Mark V. Arena John Birkler John F. Schank Jessie Riposo Clifford A. Grammich Copyright © 2005 by RAND Corporation. Excerpted by permission.
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.

---