Evaluating the outcome of a research project in an annual report is difficult because many projects run over multiple years and there is a period of time between when R&D is undertaken, completed and then adopted by end users as to when the total value of the investment is realised.
The time scale can also vary depending on the activity undertaken. While there can be an instant impact from a project — resulting in change of practices or management arrangements for example — the total outcome may take time to accrue and that can only be measured when looking back.
RDC impact assessment and performance reporting
The FRDC has put in place metrics to anticipate potential value (ex ante) and a formal measurement process to evaluate benefit cost (poste ante), which aligns with the Council of Rural Research and Development Corporations (CRRDC) evaluation framework.
The evaluation program being undertaken by the FRDC is part of the broader CRRDC work to collaboratively implement a framework of benefit cost analysis to evaluate R&D activities.
The Council of Rural RDCs benefit cost framework is based on the work of the Department of Finance in Introduction to Cost-Benefit Analysis and Alternative Evaluation Methodologies, and subsequent discussions with the department to refine the methodology.
CRRDC cross-portfolio evaluation
Over 2015-16 the FRDC, along with the other RDCs in the CRRDC, has funded a consortium of evaluation consultants to update the evaluation analysis to cover the period 2010–15 and develop an updated framework for ongoing monitoring and reporting across the RDCs. This builds on the work done by each RDC, but it is the first major cross-portfolio review in the past five years.
A stocktake of current evaluation activities of rural RDCs for the CRRDC was conducted by Lange Consulting & Software in the second half of 2015. The stocktake aimed to document existing evaluation processes and practices and inform the development of a high-level joint RDC performance framework.
From the recommendations, an opportunity was identified to standardise practices, terminology, measurement of intangibles, metrics and performance measures against Australian Government strategic research priorities for benefit cost analysis. This will better enable aggregation of performance data and provide improved visibility of the cost effectiveness of research programs across each RDC’s portfolio.
FRDC portfolio evaluation
Each year, the FRDC also undertakes an evaluation of a random selection of projects, to fulfil requirements of the Statutory Funding Agreement, and to demonstrate benefit to stakeholders. Below are the final reports for each of these assessments:
