Development of guidelines for quality assurance of Australian fisheries research and science information
Pisces Australis Pty Ltd
Keywords: fisheries science, quality assurance, peer review,
There has been a long history of international development of increasingly detailed guidelines and standards for quality assurance of scientific information used to inform government policy and management decisions. The need for such guidelines has resulted largely from various crises of confidence in government decisions relating to human health, and to sustainable management of natural resources, most notably the bovine spongiform encephalitis (mad cow disease) outbreak in the United Kingdom in the mid-1990s. This led to the publication by the UK Government Chief Scientific Advisor of the May Report in 1997, which established key principles for the use of scientific evidence in policy making. This was followed by UK government Guidelines on Scientific Analysis in Policy Making in 2005, a Code of Practice for Scientific Advisory Committees in 2007, Principles of Scientific Advice to Government in 2010 and an updated and more detailed Code of Practice for Scientific Advisory Committees in 2011.
Similar developments in the European Union were prompted by numerous environmental and health concerns, relating to acid rain, synthetic hormones, foot and mouth disease, pesticides and the dioxin crisis in Belgium in 1999, when dangerously high levels of dioxin were found in poultry and eggs, due to widespread use of PCB insecticides. The European Governance White Paper of 2001 emphasised the importance of quality of, and trust in, scientific advice, and led to the Use of Expertise by the Commission: Principles and Guidelines in 2002. Various reviews of the credibility of science used to inform EU decision making led to the European Peer Review Guide and the European Code of Conduct for Research Integrity in 2011.
Development of guidelines for ensuring quality of science in Canada was directly related to one of the most influential fisheries crises experienced: the collapse of the Grand Banks and Newfoundland cod stocks, despite scientific advice warning of overfishing. Guidelines for Science Advice for Government Effectiveness (the SAGE principles) were adopted in 1999, supported by a Framework for Science and Technology Advice in 2000. In response, the Canadian Department of Fisheries and Oceans developed a detailed Science Advisory Process, emphasising the role of peer review and providing guidance on choosing the most appropriate peer review process under different circumstances, supported by a set of Principles and Guidelines for Participation in Peer Review Processes.
Development of quality assurance standards relating to fisheries science in the United States stem directly from incorporation of National Standard 2 in the Magnuson-Stevens Fishery Conservation and Management Act (MSA), requiring that “(2) Conservation and management measures shall be based upon the best scientific information available”. Parallel requirements to ensure the quality of scientific information, particularly relating to scientific advice regarding pesticides and environmental concerns, resulted from the 'Data Quality Act' in 2001, actually a short rider inserted into the Consolidated Appropriations (federal budget) Act in 2001, requiring all federal agencies to “issue guidelines ensuring and maximizing the quality, objectivity, utility and integrity of information (including statistical information) disseminated by the agency”.
In response, the US Office of Management and Budget published Guidelines on Information Quality in 2002 and an Information Bulletin for Peer Review in 2004. In response to both National Standard 2 and the OMB Guidelines, the National Oceanic and Atmospheric Administration published comprehensive Information Quality Guidelines in 2006, informed by 2004 guidelines produced by the National Research Council (NRC) on Improving Use of Best Scientific Information. In further response to NRC recommendations, NOAA published detailed Guidelines for Best Scientific Information Available in 2014, as a rule under the MSA, prescribing requirements for improved implementation of National Standard 2. This rule currently provides the most detailed international guidance on quality assurance and peer review of fisheries-related scientific information.
Key components of the above guidelines, codes of practice, standards and rules were identified and consolidated to provide a distillation and summary of key principles and best practices for science information quality assurance. These international key principles and guidelines for effective peer review were used as the basis for Research and Science Information Guidelines for Australian Fisheries, intended to be applicable to the quality assurance of all research or scientific information intended or likely to inform fisheries policy or management decisions. The Guidelines cover the following aspects of science information quality assurance:
Key Principles for Scientific Information Quality: Peer Review, Reliability, Integrity, Objectivity and Relevance (the PRIOR principles).
Responsibilities of Research Purchasers and Research Providers: relating to requiring, supporting or implementing processes to ensure the quality of research and scientific information, as defined by the Key Principles.
Criteria for Effective Peer Review: balance of expertise, independence, inclusiveness, transparency and openness, timeliness, impartiality, management of conflicts of interest, reporting of uncertainty and risk and staged technical guidance.
Stages and Forms of Peer Review: options ranging from single expert review to appointed panels, established working groups, technical review workshops or independent expert review, to ensure that peer review is cost-effective and appropriate to the complexity, contentiousness and likely influence of the information.
Data Retention and Management: to ensure that data underpinning science used to inform fisheries decisions is retained, securely stored and potentially available to further analysis, subject to applicable confidentiality and privacy requirements.
Documentation and Communication: to ensure that research results and scientific information are clearly reported, and that the integrity of information is protected in such communication.
Implementation and Reporting: providing guidance on development of organisation-specific plans for the implementation of science quality assurance processes
Definition of Terms: providing definitions for key terms used, to aid with the interpretation and implementation of the guidelines.
The guidelines are non-prescriptive and provide for high flexibility in implementation, to ensure they are relevant across the wide range of research activities informing policy and management decisions for Australian wild capture fisheries and their impact on the marine environment. It is expected that implementation will be achieved by means of agency-specific implementation plans, tailored to the requirements, capabilities and current processes of each agency. An example draft implementation plan is provided for Commonwealth fisheries, prepared by the Australian Fisheries Management Authority.
Principal Investigator: Andrew Penney
Fisheries research and scientific information is used to inform fisheries management decisions and the formulation of related environmental policy. Government Ministers and decision-makers, stakeholders and the public need to have confidence and trust in research and scientific information used to inform fisheries management. To help achieve this, key principles for ensuring quality of science need to be adhered to, and effective science quality assurance processes need to be put in place, to:
1. Review recent national and international developments on science quality assurance principles, implementation guidelines and quality assurance processes relevant to Australian fisheries characteristics, management processes and requirements.
2. Prepare draft standard and guidelines for quality assurance of Australian research and science information intended or likely to inform fisheries policy and management decisions, including key principles for science quality, implementation guidelines and performance monitoring for science quality assurance processes.
3. Consult with fisheries agencies in other jurisdictions, as well as other relevant stakeholders, to ensure that the proposed science quality assurance guidelines are appropriate and implementable for all Australian fisheries, and potentially implementable for other science fields
4. Prepare an agency-specific plan for implementation of the science quality assurance key principles and quality assurance processes for AFMA, compatible with AFMA and Commonwealth fisheries requirements, capabilities and science procurement processes.