John A. Thies
Informatics Corporation
ABSTRACT
The U. S. Department of Energy (USDOE) will submit the Waste Isolation Pilot Plant (WIPP) Compliance Certification Application (CCA) to the U. S. Environmental Protection Agency (USEPA) in October, 1996. The WIPP CCA must demonstrate that transuranic (TRU) radioactive waste* can be safely received and disposed of in a deep geological disposal system (repository) at the WIPP site in accordance with the safety criteria defined by the USEPA in Code of Federal Regulations, Title 40, Part 191 (40 CFR 191). The WIPP TRU waste repository is scheduled to open in 1998, which will make it the nation's first deep geologic repository and the world's first deep geologic repository for long-lived radioactive waste. To comply with applicable regulations, DOE sponsored independent Peer Reviews of critical program elements.
INTRODUCTION
A Peer Review (PR) is a documented critical review performed by peers of those individuals that conducted the original work. The peers must be independent of the work being reviewed. A PR consists of an in-depth critique of assumptions, calculations, extrapolations, alternate interpretations, methodologies, acceptance criteria, and conclusions in the original work.
Six peer reviews were sponsored by the Department of Energy (DOE) Carlsbad Area Office (CAO) to comply with the requirements of Title 40 of the Code of Federal Regulations (CFR) Part 194, Criteria for Certification of Waste Isolation Pilot Plant (WIPP) Compliance. The Peer Reviews were as follows:
To facilitate conduct of the PRs the DOE CAO retained an independent private contractor to manage the PRs and to ensure that the peer reviews were conducted in accordance with all applicable plans and procedures.
REGULATORY REQUIREMENTS
The certification criteria in 40 CFR Part 194 prescribes the use of peer reviews to support certain areas of the WIPP compliance evaluation. Compliance criteria in 40 CFR § 194.27 state that peer review at the WIPP be performed for several specific aspects of the program and that they be performed in a manner compatible with NUREG-1297. NUREG-1297 provides guidance on the definition of peer reviews, the areas for which a peer review is appropriate, the acceptability of peers, and the conduct and documentation of peer reviews.
NUREG-1297 defines peer review as "a documented, critical review performed by peers who are independent of the work being reviewed." NUREG-1297 also states that a "peer review is an in-depth critique of the assumptions, calculations, extrapolations, alternate interpretations, methodology, and acceptance criteria employed, and of conclusions drawn from the original work."
The 40 CFR Part 194 Background Information Document (EPA 1996b) states that peer reviews can be used as part of "a comprehensive quality assurance program" to give "confidence that work completed, underway, or planned was, is, or will be properly performed." The Background Information Document also notes that "additional peer review is also necessary to establish the validity of procedures, methods, or interpretations which may not be addressed by a quality assurance program...ASME-NQA-3-1989...includes peer review among those activities affecting quality associated with the collection of scientific and technical information, when other established methods cannot be used to establish the adequacy of information."
NUREG-1297 states that for a repository,
"peer reviews should be used as a management tool to achieve confidence in the validity of certain technical and programmatic judgements. The intent of a peer review is to pass judgement on the technical adequacy of the work or data submitted for review, to identify aspects of the work on which technical consensus exists, to identify aspects on which technical consensus does not exist, and to identify aspects of the reviewed work which the reviewers believe to be incorrect or which need amplification. A peer review provides assurance in cases where scientific uncertainties and ambiguities exist but in which technical and programmatic judgements and decisions still must be made."
PEER REVIEW PROCESS
NUREG-1297 suggests that procedures be developed to "implement the NUREG-1297 guidance" and to "provide methods for initiating a peer review." These procedures, for any given peer review, "should require a planning document that describes the work to be reviewed, the size and spectrum of the peer review group, and the suggested method and schedule to arrive at a peer review report."
The Quality Assurance Program Document (QAPD) (DOE 1996c) establishes the minimum requirements for the WIPP quality assurance (QA) program. It provides guidance for development and implementation of QA programs for all aspects of the WIPP project. In particular, the QAPD provides general requirements for training, document control, and QA records management.
WIPP-specific plans and procedures ensure that peer reviews performed subsequent to promulgation of 40 CFR Part 194 were conducted in accordance with the criteria of 40 CFR Part 194 and compatible with NUREG-1297.
A Peer Review Management Plan (PRMP) (DOE 1996a) was developed and approved by the CAO to describe the management processes used to control the planning, implementation, and documentation of peer reviews. The PRMP defines the management approach, resources, schedule, and technical requirements for using peer reviews to confirm and/or verify the adequacy of data and/or information utilized to support the WIPP compliance certification application.
CAO Team Procedure (TP) 10.5, Peer Review (DOE 1996b), implements the requirements of NUREG-1297. TP 10.5 prescribes the responsibilities, requirements, and methodologies to be incorporated in the performance of peer reviews conducted by the CAO pursuant to the criteria of 40 CFR § 194.27. The procedure provides the criteria for determining the size and composition of the review panel and for selecting individual peer review panel members, and outlines the orientation and training to be provided for the panelists. TP 10.5 also describes the actual peer review process, provides criteria for development of peer review plans and report preparation, and defines the responsibilities of individuals involved in the process.
Specific peer review plans were developed to document the planning process for each peer review at the WIPP. It provides the scope of the peer review, a description of the work to be reviewed, the intended use of the work, the size and composition of the peer review panel, and methods for conducting peer reviews.
A desk instruction, prepared by the peer review manager, is consistent with and supplements TP 10.5 to clearly define peer review administrative processes and requirements.
SIZE AND COMPOSITION OF PEER REVIEW PANELS
NUREG-1297 states that
"The number of peers comprising a peer group should vary with the complexity of the work to be reviewed, its importance to establishing that safety or waste isolation performance goals are met, the number of technical disciplines involved, the degree to which uncertainties in the data or technical approach exist, and the extent to which differing viewpoints are strongly held within the applicable technical and scientific community concerning the issues under review. The collective technical expertise and qualifications of peer group members should span the technical issues and areas involved in the work to be reviewed, including any differing bodies of scientific thought. Technical areas more central to the work to be reviewed should receive proportionally more representation on the peer review group."
NUREG-1297 guidance also states that
"The peer review group should represent major schools of scientific thought. The potential for technical or organizational partiality should be minimized by selecting peers to provide a balanced review group."
Peer review panel size and composition was determined by a selection committee consisting of the Peer Review Manager and two members selected by the Peer Review Manager. These individuals were selected because of their familiarity with the peer review process, the technical aspects of the WIPP project, their impartiality, and their knowledge of potentially qualified peer reviewers.
Technical requirements for each peer review panel were established by the Peer Review Manager and provided to the selection committee, which then developed a list of potentially qualified personnel. Once a panel member was officially selected and had agreed to serve, the selection committee members documented the rationale for the selection of that peer review panel member on a "Peer Review Panel Selection, Size and Composition Justification/Decision Form," which is maintained as a QA record. Rejection rationale was also documented for the potential panel members who were not selected.
The number of member selected for a particular panel depended on the amount and complexity of the work to be reviewed, its importance for establishing that safety or waste isolation performance goals are met, the number of technical disciplines involved, the degree to which uncertainties in the data or technical approach exist, and the extent to which differing viewpoints are strongly held within the applicable technical and scientific community concerning the issues under review. The panel members were selected based on their collective technical expertise and qualifications so that they spanned the technical issues and areas involved in the work to be reviewed, including differing bodies of scientific thought. The technical areas more central to the work to be reviewed received proportionally more representation on the peer review panel. To the extent practical, the panels represented the major schools of scientific thought pertinent to the subject being reviewed. The selection committee strived to eliminate the potential for technical or organizational partiality by selecting peer reviewers that provided a balanced panel.
The strategy for staffing the panels was to use a combination of individuals knowledgeable of the WIPP with other individuals that had little or no knowledge of the WIPP. All of the individuals had to meet the strict independence requirements. To ensure that the data review panels had knowledge of the related conceptual models, two members of the Conceptual Models Peer Review Panel were assigned to the Natural Barriers Data Qualification Peer Review Panel and two others were assigned to the Engineered Systems Data Qualification Peer Review Panel. In addition, one individual from the Waste Form and Disposal Room Data Qualification Peer Review Panel was assigned to the Waste Characterization Analysis Review Panel to ensure the latter panel was provided with timely and in-depth knowledge of chemistry data pertinent to the waste.
TECHNICAL QUALIFICATIONS OF PANEL MEMBERS
NUREG-1297 states that
"The technical qualifications of the peer reviewers, in their review areas, should be at least equivalent to that needed for the original work under review and should be the primary consideration in the selection of peer reviewers. Each peer reviewer should have recognized and verifiable technical credentials in the technical area he or she has been selected to cover. The technical qualifications of each peer, and hence of the peer review group as a whole, should relate to the importance of the subject matter to be reviewed."
TP 10.5 specifies that the acceptability of any peer review panel member be based on the above NUREG-1297 requirements. The Peer Review Manager is required to ensure that education and pertinent experience information is verified and documented prior to the start of the peer review process. This documentation is also maintained as QA records.
INDEPENDENCE OF PANEL MEMBERS
NUREG-1297 states that
"Members of the peer review group should be independent of the original work to be reviewed. Independence in this case means that the peer a) was not involved as a participant, supervisor, technical reviewer, or advisor in the work being reviewed, and b) to the extent practical, has sufficient freedom from funding considerations to assure the work is impartially reviewed."
"Because of DOE's pervasive effort in the waste management area, the lack or unavailability of other technical expertise in certain areas, and the possibility of reducing the technical qualifications of the reviewers in order that total independence is maintained, it may not be possible to exclude all DOE or DOE contractor personnel from participating in a peer review. In those cases where total independence cannot be met, a documented rationale as to why someone of equivalent technical qualifications and greater independence was not selected should be placed in the peer review report."
NUREG-1297 allows both the work under review and the peer review of that work to be funded by DOE. It also provides the caveat that the
"independence criteria is not meant to exclude eminent scientists or engineers upon whose earlier work certain portions of the work under review is based so long as a general scientific consensus has been reached regarding the validity of their earlier work."
TP 10.5 provides that the above NUREG-1297 requirements be used in selecting panel members. Each peer review panel member is required to document his or her independence. These documents are reviewed and approved by the Peer Review Manager and maintained as QA records.
TRAINING OF PEER REVIEW PANEL MEMBERS
TP 10.5 requires that the Peer Review Manager ensure that all peer review panel members receive adequate training prior to beginning a peer review. Training consisted of reading assignments and, if deemed necessary by the Peer Review Manager or the Peer Review Panel Coordinator, briefings and classroom training. Assigned reading includes 40 CFR Parts 191 (EPA 1993) and 194, NUREG-1297, the CAO QAPD, TP 10.5, and the applicable Peer Review Plans.
TP 10.5 further requires that all panel members receive an orientation prior to the start of the peer review process. The orientation included information on the peer review process, administrative requirements, the applicable Peer Review Plan, a summary of the technical subject matter, and an overview of TP 10.5. Panel member training and orientation are documented and this documentation is maintained as a QA record.
PEER REVIEW PANEL REPORT
NUREG-1297 states that
"A written report documenting the results of the peer review should be issued. It is usually prepared under the direction of the chairperson of the peer review group, and is signed by each member individually. It should clearly state the work or issue that was peer reviewed and the conclusions reached by the peer review process... The report should include individual statements by peer review group members reflecting dissenting views or additional comments, as appropriate. The peer review report should contain a listing of the reviewers and any acceptability information (i.e., technical qualifications and independence) for each member of the peer group, including potential technical and/or organization partiality."
TP 10.5 requires that a peer review report be prepared for each peer review. Each panel member is required to sign and date the report. The report describes the work or issue that was reviewed and the conclusions reached by the panel, and it provides individual statements by the members reflecting dissenting views or additional comments, as appropriate. Finally, the report lists the peer review panel members and provides technical qualifications and independence information for each member.
QUALITY ASSURANCE RECORDS MANAGEMENT
NUREG-1297 states that
"minutes should be prepared of meetings, deliberations, and activities of the peer review process."
TP 10.5 requires that written minutes, including graphic or calculated materials used in panel meetings, be prepared for meetings, deliberations, daily caucuses, and other activities. These written minutes are maintained as QA records. TP 10.5 also requires that a QA records management system be developed and implemented to ensure that peer review documents are identified, assembled, and transferred on a timely basis and in an orderly manner to the appropriate records center.
QUALITY ASSURANCE AUDITS AND ASSESSMENTS
NUREG-1297 states that
"As a minimum, the QA organization should provide surveillance of the peer review process to ensure that the procedures conform to the guidance of this GTP and that they are followed by the peer review group."
The QAPD establishes requirements for implementing the QA program for the WIPP peer review process. The QAPD requires that assessments be conducted to ensure that all aspects of the peer review conform to the guidance of NUREG-1297, TP 10.5, and the CAO QAPD.
Internal Quality Assurance Assessments were conducted by the Peer Review Manager, his staff, and other independent professionals to assess records quality and records management processes. External QA audits and assessments were performed and documented to provide DOE-CAO with assurance that the peer review process was conducted in accordance with applicable procedures.
CONCLUSIONS
The DOE found the peer reviews very useful in that they focused the project's technical documentation on issues related to compliance. Also, aspects of the technical program with potentially weak technical arguments were identified to the DOE.
*Waste containing man-made radioactive isotopes with atomic weights greater than uranium containing more than 3,700 becquerels of alpha-emitting radioactive isotopes per gram of waste with half-lives greater than 20 years.