SEARCH · Search PubMed
Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
The National Institute for Occupation Safety and Health (NIOSH) has developed a program to evaluate portable, direct-reading instruments. The results of these evaluations for CO, SO2, combustible gas, and NO2 instruments are reported. Realistic performance specifications have been developed for each of these types of instruments.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Early in 1980 a Veterans Administration Medical Center initiated a comprehensive program evaluation project of its Alcohol Treatment Unit. The project has resulted in numerous publications detailing the major findings; however, many outcomes of interest have not been reported, and no report is extant which summarizes the outcome of the program evaluation. In this paper, many results are reported for the first time, and previously published results are briefly described. Additionally, the problems encountered in program evaluation are discussed, as are the implications of the results of the project.
Pediatric ambulatory surgery programs have grown tremendously during the past decade. However, limited nursing time places severe constraints on the care and education of surgical outpatients. Preoperative teaching, patient support, and postoperative instruction, previously conducted over days, is now completed in several hours. The ambulatory surgical program at Mott Children's Hospital was designed to provide thorough patient and family education, comprehensive patient care, and short-term follow-up. A formal evaluation of the program was conducted to ensure quality care for outpatients. This study evaluated preoperative and discharge preparation, postoperative problems and follow-up, preoperative waiting, stress, and privacy. The sample was composed of 332 families. Of the respondents, 289 (87%) felt very prepared overall for outpatient surgery; 33 families (10%) attended the preoperative tour, and felt significantly better prepared for surgery than families who did not attend; 322 families (97%) felt adequately prepared to very prepared for discharge; 245 families (76%) felt that the recovery time was just right; 40 parents (12%) felt that their children experienced more pain and 34 (10.4%) more nausea and vomiting than expected postoperatively; 54 families called a PACU nurse postoperatively with questions, and 92% rated the call as very helpful; 31 families called the surgical service with concerns, and 61% rated the surgeon as helpful; 83 families (25%) perceived the outpatient experience as being very stressful. Stress correlated significantly with preoperative preparation, preoperative wait, previous outpatient experience, and perceived postoperative pain. Privacy in the environment was rated as fair to poor by 192 families (59%).(ABSTRACT TRUNCATED AT 250 WORDS)
In 1986, the Centers for Disease Control (CDC) implemented the Model Performance Evaluation Program (MPEP) to evaluate the performance of laboratories that test for antibody directed against human immunodeficiency virus type 1 (HIV-1). The impetus for developing this program came from the recognition of a need to assess the quality of existing and changing laboratory technology and to ensure that the quality of testing was sufficient to meet medical and public health needs. To develop the program, CDC chose HIV-1 antibody testing as the first specific application for assessing the quality of laboratory performance because (a) of the importance of accurate and reproducible test results for acquired immunodeficiency syndrome (AIDS) surveillance, prevention, and treatment programs; (b) HIV-1 testing technology is new to many laboratories; and (c) HIV-1 testing practices and applications continue to evolve. Unlike proficiency testing programs, the MPEP is not limited to assessing quality in the analytical step, alone. It will also assess quality in the preanalytical and postanalytical steps of the testing process, that is, from the time a test is requested until the clinician who ordered the test takes an action based on the test result. The participating laboratories furnish the information needed for the performance evaluation program by (a) completing questionnaires designed to describe HIV-1 testing laboratories and their testing practices, (b) analyzing specially prepared sample panels for HIV-1 antibody reactivity, and (c) reporting results to CDC.
This article addresses the issue of program evaluation in the area of health services; examples are drawn from the field of mental health. Current arguments concerning the goals, characteristics, and methodologies of program evaluation are discussed and two generally useful quantitative evaluation models are presented. The models are compared and their advantages for clinicians and administrators are detailed.
An armed robbery alarm system was implemented in 48 different stores in two separate geographical areas for 6 months and 12 months, respectively. The alarms were placed in the two separate areas at different times and all alarms were eventually removed. Thus, multiple baseline and reversal strategies were used to evaluate program impact. A device planted in a cash drawer was triggered whenever "bait" money was removed from the drawer sending an alarm signal directly to police cars and headquarters. On-scene apprehensions of armed robbers within target stores were greatly increased even though the armed robbery systems did not deter robbery incidents nor influence the court disposition of the cases. There was also no crime deterrence, crime displacement, or increased apprehensions in either the immediate neighborhoods of target stores or on a city-wide basis. The cost effectiveness of the program was calculated to be poor even though the program is being maintained because of the absence of an alternative robbery apprehension technology.
A means of determining the effectiveness of public programs as a basis for justifying resource allocation to these programs is urgently needed. Program evaluation can provide the data base required for program decision-making and modification. A ten-step model for planning and carrying out an evaluation is suggested as a guideline for program personnel who may be assigned to evaluation responsibilities. Nutrition professionals must become aware of the increasing need for evaluation and of the procedures for conducting the evaluation process within programs for which they are responsible. Failure to provide documentation of adequate, on-going evaluation of effectiveness may lead to a loss of resources for the program and, ultimately, to the demise of the nutritional care program.
In May 1975 the Center for Educational Development, College of Medicine, University of Illinois at the Medical Center, conducted a workshop titled, "Evaluation in the Health Professions." Five tracks were offered at the workshop and one focused on program evaluation. The 12 participants from this track worked on their respective projects for four-and-one-half days from an earlier version of the protocol presented in this article. Much of the workshop was tape recorded, and this article is the result of reviewing the tapes and recording the problems that were encountered with the original version of the protocol for program evaluation.
Psychoeducational programs for families of the mentally ill became widespread during the 1980s as a means of providing a forum for the relevant education and mutual support of participants. While these programs are thought to be extremely useful as interventions, very little emphasis has been placed on evaluation as a means of demonstrating their effectiveness in achieving goals. There is a possibility, then, that psychoeducation will continue to flourish with little direct evidence of positive outcomes for its family participants. This article consists of a literature review of existing methods of psychoeducational program evaluation, both quantitative and qualitative, all of which may be applicable in certain circumstances. The process by which an evaluation instrument was developed for a program with families of the mentally ill is then presented in some detail.
An analysis of world class performance improvement over chronological time was used to develop mathematical curves of performance for each speedskating event. The equations were calculated using an unconstrained non-linear least squares iterative curve-fitting technique. A non-linear model was selected to satisfy the principle of diminishing returns, that is, it is more difficult to achieve a unit of performance improvement as performance approaches the theoretical limits of man. Minimum criteria of acceptance for each curve were: 1) the coefficient of determination must be 0.90; 2) the year 2000 predicted value must be less than the current world record; and 3) the curve must be progressive, i.e. satisfy the principle of diminishing returns. Satisfactory curves were calculated for seven of the eight male and female events studied. A four year Olympic cycle was noted and a definite change in performance trend was identified as occurring in the mid 1960's. The calculated mathematical curves can be used in three applications: 1) setting objective individual goals and evaluating these goals; 2) evaluating and comparing training programs on successive years; and 3) evaluating total programs.
Ensuring high quality in human immunodeficiency virus type 1 (HIV-1) antibody testing is an essential component of the organized public health response to epidemic HIV-1 infection. In 1986, the Centers for Disease Control designed the Model Performance Evaluation Program to assess and improve the analytic quality of HIV-1 antibody testing. In addition, the program was designed to gather information about HIV-1 antibody testing practices. The utility of this information is in identifying potential barriers to quality throughout the total testing process. Currently, 1405 laboratories participate in the program. Participating laboratories are located both within and outside the United States and consist primarily of hospitals, blood banks, health departments, and independent laboratories. The responses to a questionnaire completed by 1050 program-participant laboratories in September 1988 suggest that at several stages in the HIV-1 antibody total testing process, laboratory practices (including the interpretation of Western blot patterns) are variable and that standardization of these practices would improve quality.
An important component in the development of any educational program is the evaluation of its effectiveness. The program described in this article was designed in such a way that its impact on practitioners' knowledge, attitudes, and clinical practices regarding human immunodeficiency virus (HIV) could be evaluated. One hundred ten participants were asked to complete pre-conference and post-conference tests in order to measure improvement that resulted from program attendance. The author presents both a description of the conference and a summary of the evaluation results, along with suggestions for improving future educational programs.
Results from laboratories performing indirect immunofluorescence (IIF) testing for human immunodeficiency virus type 1 antibody and participating in the Centers for Disease Control Model Performance Evaluation Program in 1988 are presented. Approximately 90% of all laboratories receiving specimen panels or questionnaires furnished results to the Centers for Disease Control. In September 1988, 111 reports were received from IIF laboratories from 34 states and nine countries; most of these laboratories did IIF testing in conjunction with other antibody tests. Hospital laboratories were the most common type of laboratory participating in the program. Laboratories that performed IIF employed fewer personnel and performed testing less frequently than did laboratories that performed enzyme immunoassays or Western blot (immunoblot) tests and were likely to use a commercial test kit. Most of the laboratories that referred specimens for IIF testing sent them to the state laboratory. The analytic specificity for the Model Performance Evaluation Program specimens was 98.5% when indeterminate results on a negative specimen were considered correct (negative) and 89.6% when indeterminate results on a negative specimen were considered incorrect; analytic sensitivity was 94.8% when indeterminate results on a positive specimen were correct (positive) and 91.4% when indeterminate results on a positive specimen were considered incorrect. When indeterminate results were considered correct, all types of laboratories (blood bank, state, hospital, independent, and other) had analytic specificities over 96%, and all manufacturers had analytic specificities above 95%. All types of laboratories had analytic sensitivities over 92%, and analytic sensitivities were above 94% for all manufacturers and reagent sources except Cellular Products. Comparison of percentages of correct responses between IIF and Western blot assays on those samples for which there was good agreement on the target interpretation revealed no significant differences. Both individual donor and diluted materials were included in the evaluations; the diluted donor material presented the greatest testing difficulty. Within-survey reproducibility was about 93% overall and by specimen type. Between-survey reproducibility was about 81% for negative and indeterminate specimens and 88.5% for positive specimens, for an overall between-survey reproducibility of 84.3%. Differences in performance were noted when results were compared by type of laboratory and test manufacturer.
Evaluation of health program is necessary for more rational allocation of scarce resources. Steps in evaluation include: 1)Specification of the program goals, 2) Description of the program, 3) Definition and measurement of outcome, 4) Assessment of intervening variables, 5) Analysis of data, and 6) Implementation of findings. Examples from impirical evaluation studies are presented to illustrate one practical application of these steps. While the random controlled trial is a most powerful tool, it is very difficult to apply in complex human services programs. In the future, more use of multivariate matching of groups to approximate a quasi experimental situation will be needed. Education of health professionals in methods of evaluation is necessary to promote planned improvement in health services.
The effectiveness of a new approach to providing psychiatric rehabilitation services in community settings will be assessed using a combination of quality assurance principles and program evaluation techniques. A new Assertive Community Rehabilitation Program (ACRP) is evaluated and compared with existing hospital rehabilitation programs. Measures of service efficiency, admission, discharge and readmission rates, and service costs are made for 100 new referrals, 99 inpatients and 117 outpatients. Follow-up interviews use standardized measures of clients' quality of life, clinical status, client and staff satisfaction, and community resource utilization. After 19 weeks of operation, the ACRP has prevented more admissions, and discharged more inpatients than the comparison programs. Readmission rates have not differed. Results at the end of the one-year project using this program-based quality assurance approach will facilitate managerial decisions about the future of rehabilitation services.
The authors discuss evaluation studies of the community mental health centers program. The emphasize that program evaluation is a political activity to influence the allocation of public resources. They examine its relationship to decision-making, power conflicts, and bureaucratic goals. They point out that the evaluator is seldom totally objective. They also emphasize the increasing use of evaluation by politicians.