Unit 9.0: Post Project Review - National Roads Authority

Search:

You are here: > Home > Publications > Project Appraisal Guidelines > Unit 9.0: Post Project Review

Post Project Review

A pdf copy of this document can be downloaded here.

1. Introduction

1.1. At the completion of any project, it is a requirement in the NRA Project management Guidelines that a post project completion report be prepared. The post project completion report verifies that all aspects of the project have been completed, confirms project budget information and discusses the issues that arose in project development, particularly those that have implications for future road planning and implementation

1.2. The Department of Finance Guidelines¹ indicate that in addition to the post-project completion report, a Post Project Review should be carried out for all projects costing in excess of €30 million, and for a representative sample of 5 per cent of all projects. The aim of the review is to determine whether:

• The basis on which a project was undertaken proved correct;
• The expected benefits and outcomes materialised;
• The planned outcomes were the appropriate responses to actual public needs;
• The appraisal and management procedures adopted were satisfactory, and
• Conclusions can be drawn applicable to other projects, to the ongoing use of the asset, or to associated policies.

1.3. The Post Project Review thus incorporates elements of the post project completion report, but extends beyond that report to consider the full range of issues identified above.

2. Project Management

2.1. The responsibility for the Post Project Review rests with the Sponsoring Agency. The Post Project Review should be carried out by either an employee of the Design Office or a consultant appointed by the Design Team. The person(s) undertaking the Post Project Review should not have been directly involved in the oversight of the project planning or implementation, or in the appraisal of the project.

2.2. The Department of Finance Guidelines indicate that a Post Project Review should be undertaken once sufficient time has elapsed to allow the project to be properly evaluated with sufficient evidence of the flow of benefits and costs from it.

2.3. It is envisaged that, except in special circumstances, the Post Project Review should be commenced one year after project opening. This timing ensures that at least a full year’s experience of operation of the road and of the traffic volumes arising is available to inform the analysis. At the same time, it also enhances the likelihood that personnel engaged in the project planning and implementation are still available to help address data and information gaps.

2.4. The Post Project Review should evaluate four phases of the project vis-à-vis:

• Project conception;
• Project planning;
• Project implementation, and
• Project operational performance.

Project conception

2.5. This part of the review should provide an account of the background to the project, how the project arose and reference relevant road or transport programmes or policies. It should then consider the Project Brief and review its comprehensiveness and relevance. In particular, it should assess whether the need for the project was soundly established within the brief and whether the objectives were clearly specified.

Project planning

2.6. The review should address the following aspects of project planning:

• Traffic analysis and forecasting;
• Route selection process;
• Design processes;
• Project appraisal processes;
• Compliance with procurement, EIS/EAR and other statutory requirements, and
• Consultation processes.

Project implementation

2.7. With regard to project implementation, the Post Project Review should address the following:

• Project management structures;
• Monitoring and evaluation;
• Budget compliance;
• Project schedule compliance, and
• Project scope, value and risk management.

Project operational performance

2.8. This phase of the Post Project Review should consider:

• Achievement of the project objectives;
• Traffic volume outcomes;
• Traffic operation and road safety outcomes, and
• Implications of operational performance for the project’s potential economic return.

3. Focus of the review

3.1. With regard to each of the project phases, the Post Project Review should concentrate on:
• Confirming that all phases of project planning and implementation complied with relevant Department of Finance, Department of Transport, and NRA guidelines²;
• Confirming that necessary approvals were obtained from the NRA at appropriate project planning decision points;
• Assessing whether the project has delivered and is likely to continue to deliver on its objectives;
• Identifying any problems with project planning and implementation and analysing how they arose, and
• Drawing lessons from the project planning and implementation for future management of projects.

3.2. The review should consider the following areas:

Traffic analysis and forecasting

• Traffic modelling approach was appropriate to the nature of the project;
• Projected traffic growth rates were benchmarked against recent trends;
• Project traffic growth rates were compatible with Authority guidelines or other planning documents, and
• The sensitivity of traffic forecasts to relevant factors was considered.

Route selection

• The options considered were sufficiently different to offer real choices;
• The project appraisal processes were fully complied with at route / option selection stage, and
• Appropriate weighting was given to the different impacts of the project in making route choices.

Project appraisal

• Project appraisal was carried at all the relevant project phases;
• Relevant appraisal techniques / software were used (COBA or other NRA accepted techniques);
• Appropriate application rules and parameter values as per the Department of Finance, Department of Transport and NRA guidelines were used, and
• Sensitivity testing was undertaken.

Project scope, value and risk management

• Active management of scope changes was undertaken;
• Risk management was actively pursued throughout the planning and implementation phases, and
• Opportunities for value management were identified and actively pursued.

Design stage

• Road capacity decisions had a sound basis;
• The preliminary design proved to be a sound basis for project implementation, and
• Ad-hoc alterations to the design were necessary during the implementation phase and whether these reflected on the quality of the preliminary design process.

Procurement

• EU and national rules were adhered to, and
• Procurement decision-making processes were sound.

Consultation

• Consultation processes were of good quality.

Project management

• Project management was in line with existing NRA guidance;
• Monitoring reports were timely and of good quality;
• The project remained within budget;
• The project met target costs;
• The project met the target schedule, and
• The project design requirements were fully met.

Project operational performance

• The project objectives were met;
• Post opening traffic volumes are in line with those predicted and used in the project appraisal;
• Any departure of traffic volumes from those predicted has implications for the predicted economic return to the project;
• There are any problems with traffic operation on the road;
• There are any road safety problems emerging, and
• There are any further actions necessary to secure the anticipated benefits of the project.

4. Handover, Review & Closeout CBA

4.1. The undertaking of a handover, review & closeout CBA may be required as part of Post Project Reviews for projects costing in excess of €30m. The final account / closeout CBA should concentrate on establishing whether the bases on which the costs and benefits attributed towards the scheme at planning stage were correct.

4.2. An Appraisal Team within the Design Office or an independent consultant acting on behalf of the sponsoring agent will be responsible for undertaking the handover, review & closeout CBA review.

4.3. Prior to undertaking the review final outturn scheme costs must be established. After 12 months the majority of the scheme costs should be known to allow a CBA to be undertaken. If a substantial proportion of the costs are still outstanding then the undertaking of a CBA can be postponed with the consent of the NRA Strategic Planning Unit. Costs to be used for the appraisal of Public Private Partnership schemes should be agreed with the NRA.

4.4. The handover, review & closeout CBA will require information relating to scheme costs, outturn traffic flows, and journey times. The collection of accident data is unlikely to be required given that it is common practice to use accident records over an extended period (3 to 5 years) to reduce the impact of one off events on overall trends.

Scheme Costs

4.5. The actual costs and when they occurred in delivering the scheme should be entered into the CBA program, which requires all costs to be net of indirect taxation and at the price base year.

Traffic Data

4.6. For a simple network, it would be possible to undertake traffic counts on all links in the network and input these into a new CBA. This would include both counts on the scheme and on links in the rest of the network, particularly where flows were forecast to change significantly.

4.7. For more complicated networks it is too onerous to collect traffic count data for every link in the network. In these instances it may be necessary to revalidate the traffic model to post-opening conditions using a programme of traffic counts. The output from the traffic model could then be input as data for the handover, review & closeout CBA.

Journey Times

4.8. Journey time surveys should be undertaken which will allow measurement of actual journey times on key links in the network – this is required to validate traffic models (in the case of TUBA models, or COBA models where COBA is used for the review.

Comparison with Previous CBA

4.9. A comparison should be undertaken with both the Phase 2 Route Selection and Phase 3 Design CBA outputs. The comparison should look at the overall economic indicators including PVB, PVC, NPV, BCR and IRR. A summary table should be produced showing the values, at constant prices, for these indicators at each phase.

5. Contents of the Post Project Review Report

5.1. The Post Project Reviews should be arranged in the following chapters:

1. Executive summary.
2. Background to the project.
3. Traffic analysis and forecasting.
4. Project appraisal procedures.
5. Route selection.
6. Preliminary design.
7. Procurement and statutory processes.
8. Project implementation.
9. Project operational performance.
10. Overview of issues arising.
11. Conclusions and recommendations.

5.2. The BC will form the basis on which the National Roads Authority makes a decision to progress to the next stage of the project, ultimately, sanction its implementation. In this regard, the Business Case fulfils two functions:

• It enables an understanding of the merit in progressing with the project, and the risks involved with its further development; and
• Through the development of Business Cases using a consistent set of procedures, it enables Business Cases for alternative schemes to be compared, such that available funding can be allocated those the most advantageous projects.

5.3. These guidelines have been developed with reference to the “Guidelines on a Common Appraisal Framework for Transport Projects and Programmes” (Department of Transport, June 2009), which outline the approach to the preparation of Business Cases for transport infrastructure.

6. Project Management

6.1. The Business Case should be prepared at the following project phases:

• Phase 1 Concept and Feasibility;
• Phase 3 Design; and
• Phase 5 Tender

6.2. Note that where Project Appraisal has not formed part of the Phase 1 activities, then a Business Case is required as part of Phase 2 Route Selection.

6.3. The Business Case is prepared by the Appraisal team, with input from the design team as appropriate.

7. Structure and Content shof the Business Case

7.1. The Business Case should be prepared in accordance with the structure outlined in Table 8-1. This structure is consistent with that guidance presented in the DoT Guidance (June 2009).

Table 8.1: Structure of Business Case

Section	Title	Content
1	Introduction	Presentation of the background to the project, including its origins, previous relevent studies and their findings, and the guidance documentation used in the preparation of the Business Case for the project.
2	Project Context	A review of policy at local, regional, national and international level to outline the basis and need for the project. Based on this review, a set of project objectives should be defined and independent of each other, and specifically relevant to the current project. they should be classed under the following headings Economy Enviroment Accessibility and Social Inclusion Integration Safety The project objectives will inform the Multi-Criteria analysis at Phase 2 Selection.
3	Analysis Tools	A description of the anlysis tools used in support of the preperation of the business case. Such tools may include traffic models, economic models, emissions models and accessibility models. A summsry of the development and validaion of the models should be provided, with reference to technical modelling reports as appropriate.
4	Consideration of Options	A description of the process of option development and testing. This section should cover the following: Identification of constraints Defining long-list of options Sketch Appraisal to produce shortlist of options Detailed appraisal of options A Project Appraisal Balance Sheet should be provided for each route option. reference should be made to PAG Unit 4.0: Definition of Alternatives when considering the range of options that should be cosidered.
5	The Preffered Option	Introduction to the preferred scheme option emerging from Phase 2 Route Selection. This section should provide information on the design standards used, and present demand forecasts for the preffered option.
6	Scheme Appraisal	An overview of the detailed appraisal of the preffered scheme, including the following: Cost Benefit Analysis Financial Appraisal (where appropriate) Project Appraisal Balance Sheet
7	Risk Assessment	Presentation of the key risks associated with the project that require consideration in any decision making, and in subsequent stages of the project. Risks should include cost risks, programme risks, funding risks, affordability risks and acceptability risks.
8	Procurement	A discussion on procurement options, or the preffered procurement strategy if such has been decided.
9	Implementation	Any relevant proposals for implementing the project

7.2. The Business Case should be supported by and make reference to the following documents:

• Project Brief (PB);
• Traffic Modelling Report (TMR);
• Cost Benefit Analysis (CBA) Reports at Phase 2 Route Selection and Phase 3 Design Stage; and
• Project Appraisal Balance Sheet (PABS) at Phase 2 Route Selection and Phase 3 Design Stage

¹Guidelines for the Appraisal and Management of Capital Expenditure Proposals in the Public Sector. Department of Finance, February 2005, as amended by the Value for Money Circular of January 2006.

²Guidelines for the Appraisal and Management of Capital Expenditure Proposals in the Public Sector, Department of Finance, 2005, and Guidelines on a Common Appraisal Framework for Transport Projects and Programmes, Department of Transport, 2007.