PubMed2001
A rigorous approach to evaluating the reliability and relevance of experimental methods and results is critical for making appropriate, scientifically sound decisions. A literature evaluation framework was modified and adapted based on criteria described by Klimisch et al. (H. J. Klimisch, M. Andrae, and U. Tillman, 1997, Reg. Tox. Pharm. 25, 1-5) and requirements of the USEPA High Product Volume Challenge Program. This simple framework was used to evaluate an appropriate selection of peer-reviewed references to assign a Study Reliability "score" to the study design, endpoints, and conclusions compared to established data quality guidelines. Subsequently, the interpretation of the data was evaluated and assigned a Relevance Index based on the overall strength of evidence of endocrine modulation potential, which was evaluated based on the Weybridge (European Commission (1996). European workshop on the impact of endocrine disrupters on human health and wildlife. Report of proceedings from a workshop held in Weybridge, UK, 2-4 December 1996. Report reference EUR 17549, European Commission, DGXII, Brussels, Belgium.) definition. This report describes the evaluation of 50 peer-reviewed primary publications. The vast majority of studies reviewed were published in the period from 1996 to the present and most examined in vivo responses of fish to natural hormones, organochlorines, or alkylphenolic compounds. Of the studies reviewed, approximately 40% received a score of "2-Reliable with Restrictions," while approximately 60% received a score of "3-Not Reliable," where reliability was interpreted to include scientific integrity, technical credibility, endpoint relevance, and regulatory compliance. The most common deficiencies were inadequate methods, lack of suitable controls, exceeding aqueous solubility, inappropriate statistics, and unsupported conclusions. Based on the Relevance Index, few of the studies examined attained the level of quality necessary to identify the study results as providing an acceptable basis for evaluation of endocrine modulation potential directly resulting from identified deficiencies in meeting the established study reliability criteria.