Search PubMed⌕ Search

SEARCH · Search PubMed

Results for “Benchmark”

Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13Linked to original sources
PubMed1997

Benchmarking of numerical models describing the dispersion of radionuclides in the Arctic Seas.

As part of the International Arctic Seas Assessment Project (IASAP) of the International Atomic Energy Agency (IAEA), a working group was created to model the dispersal and transfer of radionuclides released from radioactive waste disposed of in the Kara Sea. The objectives of this group are: (1) development of realistic and reliable assessment models for the dispersal of radioactive contaminants both within, and from, the Arctic ocean; and (2) evaluation of the contributions of different transfer mechanisms to contaminant dispersal and hence, ultimately, to the risks to human health and environment. With regard to the first objective, the modelling work has been directed towards assessment of model reliability and asone aspect of this, a benchmarking exercise has been carried out. This paper briefly describes the benchmark scenario, the models developed and used, and discusses some of the benchmarking results. The role of the exercise within the modelling programme of IASAP will be discussed and future work described.

Arctic Regions↗
PubMed2001

The benchmark approach applied to a 28-day toxicity study with Rhodorsil Silane in rats. the impact of increasing the number of dose groups.

The OECD study design, aimed at obtaining a no-observed-adverse-effect level (NOAEL), may be suboptimal for deriving a benchmark dose. Therefore the present subacute (28-day) study was carried out to evaluate a multiple dose study design and to compare the results with the common OECD design. Seven groups of 10 female rats each were intragastrically administered corn oil without (controls) or with 50, 150, 300, 450, 600 or 750 mg Rhodorsil Silane/kg body weight/day, once daily (7 days/week) for 4 weeks. From the complete dataset, two subsets were selected, one representing a study design with seven dose groups of five animals (7 x 5 design), the other representing a study design with four dose groups of 10 animals (4 x 10 design). Under the conditions of the present study, the NOAEL for Rhodorsil Silane 198 was assessed at 50 mg/kg body weight/day, based on the data of the 4 x 10 design. The benchmark approach resulted in a benchmark dose of 19 mg/kg body weight/day, based on the data of the 7 x 5 design. Comparison of the results demonstrated that the multiple dose (7 x 5) design led to a more reliable result than the OECD (4 x 10) design, despite the smaller total number of animals. The dose-response analysis showed that at "the NOAEL" the effect on relative spleen weight was larger than 10%, illustrating that at the NOAEL, adverse effects may occur.

Alanine Transaminase↗
PubMed1997

Benchmark test cases for evaluation of computer-based methods for detection of setup errors: realistic digitally reconstructed electronic portal images with known setup errors.

PURPOSE: The purpose of this investigation was to develop methods and software for computing realistic digitally reconstructed electronic portal images with known setup errors for use as benchmark test cases for evaluation and intercomparison of computer-based methods for image matching and detecting setup errors in electronic portal images. METHODS AND MATERIALS: An existing software tool for computing digitally reconstructed radiographs was modified to compute simulated megavoltage images. An interface was added to allow the user to specify which setup parameter(s) will contain computer-induced random and systematic errors in a reference beam created during virtual simulation. Other software features include options for adding random and structured noise, Gaussian blurring to simulate geometric unsharpness, histogram matching with a "typical" electronic portal image, specifying individual preferences for the appearance of the "gold standard" image, and specifying the number of images generated. The visible male computed tomography data set from the National Library of Medicine was used as the planning image. RESULTS: Digitally reconstructed electronic portal images with known setup errors have been generated and used to evaluate our methods for automatic image matching and error detection. Any number of different sets of test cases can be generated to investigate setup errors involving selected setup parameters and anatomic volumes. This approach has proved to be invaluable for determination of error detection sensitivity under ideal (rigid body) conditions and for guiding further development of image matching and error detection methods. Example images have been successfully exported for similar use at other sites. CONCLUSIONS: Because absolute truth is known, digitally reconstructed electronic portal images with known setup errors are well suited for evaluation of computer-aided image matching and error detection methods. High-quality planning images, such as the visible human CT scans from the National Library of Medicine, are essential for producing realistic images. Sets of test cases with systematic and random errors in selected setup parameters and anatomic volumes are suitable for use as standard benchmarks by the radiotherapy community. In addition to serving as an aid to research and development, benchmark images may also be useful for evaluation of commercial systems and as part of a quality assurance program for clinical systems. Test cases and software are available upon request.

Computer Simulation↗
PubMed1994

Appointment access: planning to benchmark a complex issue.

BACKGROUND: Kaiser Permanente-Southern California Region carried out an external benchmarking project to improve appointment access. This article provides practical guidelines for planning a benchmark project around a complex issue such as appointment access. METHOD: Before conducting external site visits, team members determined the goals of the project, defined internal processes, and identified 16 key elements of access (or those factors important to member satisfaction with appointment access). Once a questionnaire was finalized, external benchmark partners were identified and on-site visits conducted. CONCLUSION: On the basis of the on-site visits, team members identified innovative and best practices for each of the 16 key elements of access. Based on these recommendations, some improvements were made across the Southern California region, such as the elimination of provisional booking. Reports of the results have also been sent to Kaiser Permanente medical centers in the region and across the country. Each medical center is encouraged to use these results to internally improve its own appointment access.

Appointments and Schedules↗
PubMed1994

Combining benchmarking with other quality tools to achieve breakthroughs in admitting.

BACKGROUND: A cross-industry benchmarking study in which St Joseph's Regional Health Systems participated recommended that the admissions process (that is, preverifying insurance, obtaining demographic information) be completed before patients arrive at the hospital. To apply this best practice at St Joseph's, management established three team efforts. IMPROVING PREVERIFICATION. One team was assigned to improve the insurance preverification process. After cross-training staff to carry out preverification and piloting a electronic, on-line preverification system, the team increased St Joseph's preverification rate from 2% to 70% for scheduled patients. EXPLORING SEAMLESS REGISTRATION: After exploring solutions for an on-line system that will be integrated across the St Joseph system, the team recommended expanding the current computer system to include all entities and to integrate demographic and clinical information. In addition, a subteam reduced the number of registration forms patients must fill out from eight to one. REDESIGNING ADMISSIONS/REGISTRATION. As part of its restructuring effort, St Joseph's is redesigning six core processes, including admissions/registration. This redesign team is currently implementing two recommendations from the benchmarking study: express admitting and centralized scheduling. LESSONS LEARNED: Organizations are advised to understand the process in its current state by collecting data and working toward improvement goals--and to be prepared for changes in job functions and allocation of resources. Other recommendations are offered to organizations attempting to apply best practices from a benchmarking study.

Admitting Department, Hospital↗
PubMed2005

Benchmark calculations of reaction energies, barrier heights, and transition-state geometries for hydrogen abstraction from methanol by a hydrogen atom.

We report benchmark calculations of reaction energies, barrier heights, and transition-state geometries for the reaction of CH(3)OH with H to produce CH(2)OH and H(2). Highly accurate composite methods, such as CBS, G2, G3S, G3X, G3SX, and multi-coefficient correlation methods (MCCMs), are used to calibrate lower-cost methods. We also performed single-level CCSD(T) calculations extrapolated to the infinite-basis limit on the basis of aug-cc-pVXZ (X = 3, 4) correlation consistent basis sets. The benchmark high-level calculations give consensus values of the forward reaction barrier height and the reaction energy of 9.7 kcal/mol and - 6.4 kcal/mol, respectively. To evaluate the accuracy of cost-efficient methods that are potentially useful for dynamics studies of the title reaction, we further include the results obtained by hybrid density functional theory methods and hybrid meta density functional theory methods that have recently been designed for chemical kinetics. Results obtained by popular semiempirical methods are also given for comparison. On the basis of the benchmark gas-phase results, we suggest MC-QCISD/3, MC3BB, and BB1K as reasonably accurate and affordable electronic structure methods for calculating dynamics for the title reaction.

Journal Article↗
PubMed2005

Oxidative addition of the fluoromethane C-F bond to Pd. An ab initio benchmark and DFT validation study.

We have computed a state-of-the-art benchmark potential energy surface (PES) for two reaction pathways (oxidative insertion, OxIn, and S(N)2) for oxidative addition of the fluoromethane C-F bond to the palladium atom and have used this to evaluate the performance of 26 popular density functionals, covering LDA, GGA, meta-GGA, and hybrid density functionals, for describing these reactions. The ab initio benchmark is obtained by exploring the PES using a hierarchical series of ab initio methods (HF, MP2, CCSD, CCSD(T)) in combination with a hierarchical series of seven Gaussian-type basis sets, up to g polarization. Relativistic effects are taken into account through a full four-component all-electron approach. Our best estimate of kinetic and thermodynamic parameters is -5.3 (-6.1) kcal/mol for the formation of the reactant complex, 27.8 (25.4) kcal/mol for the activation energy for oxidative insertion (OxIn) relative to the separate reactants, 37.5 (31.8) kcal/mol for the activation energy for the alternative S(N)2 pathway, and -6.4 (-7.8) kcal/mol for the reaction energy (zero-point vibrational energy-corrected values in parentheses). Our work highlights the importance of sufficient higher angular momentum polarization functions for correctly describing metal-d-electron correlation. Best overall agreement with our ab initio benchmark is obtained by functionals from all three categories, GGA, meta-GGA, and hybrid DFT, with mean absolute errors of 1.4-2.7 kcal/mol and errors in activation energies ranging from 0.3 to 2.8 kcal/mol. The B3LYP functional compares very well with a slight underestimation of the overall barrier for OxIn by -0.9 kcal/mol. For comparison, the well-known BLYP functional underestimates the overall barrier by -10.1 kcal/mol. The relative performance of these two functionals is inverted with respect to previous findings for the insertion of Pd into the C-H and C-C bonds. However, all major functionals yield correct trends and qualitative features of the PES, in particular, a clear preference for the OxIn over the alternative S(N)2 pathway.

Computer Simulation↗
PubMed2000

An algorithmic benchmark for quantum information processing

Quantum information processing offers potentially great advantages over classical information processing, both for efficient algorithms and for secure communication. Therefore, it is important to establish that scalable control of a large number of quantum bits (qubits) can be achieved in practice. There are a rapidly growing number of proposed device technologies for quantum information processing. Of these technologies, those exploiting nuclear magnetic resonance (NMR) have been the first to demonstrate non-trivial quantum algorithms with small numbers of qubits. To compare different physical realizations of quantum information processors, it is necessary to establish benchmark experiments that are independent of the underlying physical system, and that demonstrate reliable and coherent control of a reasonable number of qubits. Here we report an experimental realization of an algorithmic benchmark using an NMR technique that involves coherent manipulation of seven qubits. Moreover, our experimental procedure can be used as a reliable and efficient method for creating a standard pseudopure state, the first step for implementing traditional quantum algorithms in liquid state NMR systems. The benchmark and the techniques can be adapted for use with other proposed quantum devices.

Journal Article↗
PubMed1997

Using partner hospitals in collaborative benchmarking..

In May 1994 the Physiotherapy Department at John Hunter Hospital received a Commonwealth Best Practice in the Health Sector grant to design a critical pathway for the treatment of stroke. The implementation of the pathway at John Hunter Hospital and the introduction of the methodology to secondary sites (The Alfred Healthcare Group, Melbourne, and Royal Hobart & Repatriation General Hospitals, Hobart) resulted in the development of a Benchmarking Consortium. This paper will discuss the importance of benchmarking in understanding clinical processes, and the methodology employed to ensure that meaningful benchmarks were achieved.

Ancillary Services, Hospital↗
PubMed2006

A method to determine precise benchmark doses for carbamate anticholinesterases.

In determining benchmark doses for risk assessment and regulation of carbamate anticholinesterase pesticides like formetanate, oxamyl, and methomyl, one needs to quantitate low levels of cholinesterase inhibition. For improved accuracy while using fewer subjects, we developed an assay based on the recognized ability of carbamates to protect cholinesterase from irreversible inactivation. This assay measures enzyme that survives diisopropylfluorophosphate exposure in vitro and then reactivates by decarbamylation after small molecules are removed with size-exclusion centrifugation. The 99% silencing of unprotected cholinesterase yields a low background. Comparisons of recovered activity with initial activity (representing carbamate-free enzyme) use each sample as its own control. As a result, carbamate-protection assays can demonstrate a statistically significant 2-3% inhibition of brain cholinesterase in a single experimental group of modest size. When applied to brain samples from formetanate-treated rats, such an assay predicted a benchmark dose of 0.19 mg/kg for 10% inhibition (BMD10), with a lower 95% confidence limit of 0.15 mg/kg (BMDL10). Protection assays should enable precise determinations of benchmark doses for other carbamates, as well as accurate assessment of in vivo inhibition half-lives under low-dose scenarios.

Animals↗
PubMed2025

Comprehensive benchmarking of somatic structural variant detection at ultra-low allele fractions.

Postzygotic mosaicism gives rise to somatic structural variants (SVs) at ultra-low variant allele fractions (VAFs), which pose challenges for detection due to the high-coverage sequencing required and noise introduced by sequencing artifacts. Although somatic SV detection has been extensively studied in cancer, these studies are not directly applicable to the study of tissue mosaicism, as they rely on matched normals, target higher VAF ranges, and are enriched for different types of SVs. We present comprehensive benchmark data and best practices for non-cancer somatic SV detection. We created a synthetic mosaic sample by combining six HapMap individuals at varying proportions, generating allele fractions as low as 0.25%. This sample was sequenced to ~2,300x total coverage using Illumina, PacBio, and Nanopore technologies across multiple sequencing centers. A high-confidence benchmark SV set containing over 21,000 pseudo-somatic insertions and deletions ≥50bp was derived from haplotype-resolved assemblies. We evaluated 12 SV discovery pipelines and identified caller-specific strengths and sequencing platform-specific shortcomings. We find that short read-based approaches show reduced recall for insertions and repeat-associated SVs, whereas long-read sequencing achieves high accuracy throughout the genome, increasing linearly with coverage. The best algorithm's sensitivity exceeded 80% for VAFs ≥4% and 15% for VAFs of 0.5-1% with 60x coverage. The publicly available benchmarking data and comparative analysis of current methods provide a foundation for robust discovery of SV mosaicism in non-cancer tissues..

Journal Article↗
PubMed1996

Clinical benchmarking: results into practice.

Describes the way benchmarking data are used in a district general hospital to influence clinical practice. Wirral Hospital Trust is a site for the Electronic Patient Record Project; as such there is a large amount of patient based data available for research and internal benchmarking. Includes working examples of internal benchmarking which have been used by both clinicians and hospital management to improve hospital effectiveness. Discusses the ways in which this information is being used to develop initiatives such as clinical pathway development.

Critical Pathways↗
PubMed2005

All are not equal: a benchmark of different homology modeling programs.

Modeling a protein structure based on a homologous structure is a standard method in structural biology today. In this process an alignment of a target protein sequence onto the structure of a template(s) is used as input to a program that constructs a 3D model. It has been shown that the most important factor in this process is the correctness of the alignment and the choice of the best template structure(s), while it is generally believed that there are no major differences between the best modeling programs. Therefore, a large number of studies to benchmark the alignment qualities and the selection process have been performed. However, to our knowledge no large-scale benchmark has been performed to evaluate the programs used to transform the alignment to a 3D model. In this study, a benchmark of six different homology modeling programs- Modeller, SegMod/ENCAD, SWISS-MODEL, 3D-JIGSAW, nest, and Builder-is presented. The performance of these programs is evaluated using physiochemical correctness and structural similarity to the correct structure. From our analysis it can be concluded that no single modeling program outperform the others in all tests. However, it is quite clear that three modeling programs, Modeller, nest, and SegMod/ ENCAD, perform better than the others. Interestingly, the fastest and oldest modeling program, SegMod/ ENCAD, performs very well, although it was written more than 10 years ago and has not undergone any development since. It can also be observed that none of the homology modeling programs builds side chains as well as a specialized program (SCWRL), and therefore there should be room for improvement.

Amino Acid Sequence↗
PubMed2006

New algorithms and an in silico benchmark for computational enzyme design.

The creation of novel enzymes capable of catalyzing any desired chemical reaction is a grand challenge for computational protein design. Here we describe two new algorithms for enzyme design that employ hashing techniques to allow searching through large numbers of protein scaffolds for optimal catalytic site placement. We also describe an in silico benchmark, based on the recapitulation of the active sites of native enzymes, that allows rapid evaluation and testing of enzyme design methodologies. In the benchmark test, which consists of designing sites for each of 10 different chemical reactions in backbone scaffolds derived from 10 enzymes catalyzing the reactions, the new methods succeed in identifying the native site in the native scaffold and ranking it within the top five designs for six of the 10 reactions. The new methods can be directly applied to the design of new enzymes, and the benchmark provides a powerful in silico test for guiding improvements in computational enzyme design.

Algorithms↗
PubMed2001

A health risk benchmark for the neurologic effects of styrene: comparison with NOAEL/LOAEL approach.

Benchmark dose (BMD) analysis was used to estimate an inhalation benchmark concentration for styrene neurotoxicity. Quantal data on neuropsychologic test results from styrene-exposed workers [Mutti et al. (1984). American Journal of Industrial Medicine, 5, 275-286] were used to quantify neurotoxicity, defined as the percent of tested workers who responded abnormally to > or = 1, > or = 2, or > or = 3 out of a battery of eight tests. Exposure was based on previously published results on mean urinary mandelic- and phenylglyoxylic acid levels in the workers, converted to air styrene levels (15, 44, 74, or 115 ppm). Nonstyrene-exposed workers from the same region served as a control group. Maximum-likelihood estimates (MLEs) and BMDs at 5 and 10% response levels of the exposed population were obtained from log-normal analysis of the quantal data. The highest MLE was 9 ppm (BMD = 4 ppm) styrene and represents abnormal responses to > or = 3 tests by 10% of the exposed population. The most health-protective MLE was 2 ppm styrene (BMD = 0.3 ppm) and represents abnormal responses to > or = 1 test by 5% of the exposed population. A no observed adverse effect level/lowest observed adverse effect level (NOAEL/LOAEL) analysis of the same quantal data showed workers in all styrene exposure groups responded abnormally to > or = 1, > or = 2, or > or = 3 tests, compared to controls, and the LOAEL was 15 ppm. A comparison of the BMD and NOAEL/LOAEL analyses suggests that at air styrene levels below the LOAEL, a segment of the worker population may be adversely affected. The benchmark approach will be useful for styrene noncancer risk assessment purposes by providing a more accurate estimate of potential risk that should, in turn, help to reduce the uncertainty that is a common problem in setting exposure levels.

Air Pollutants, Occupational↗
PubMed2003

Benchmark calculations in risk assessment using continuous dose-response information: the influence of variance and the determination of a cut-off value.

A benchmark dose (BMD) is the dose of a chemical that corresponds to a predetermined increase in the response (the benchmark response, BMR) of a health effect. In this article, a method (the hybrid approach) for benchmark calculations from continuous dose-response information is investigated. In the formulation of the methodology, a cut-off value for an adverse health effect has to be determined. It is shown that the influence of variance on the hybrid model depends on the choice of determination of the cut-off point. If the cut-off value is determined as corresponding to a specified tail proportion of the control distribution, P(0), the BMD becomes biased upward when the variance is biased upward. On the contrary, if the cut-off value is directly determined to some level of the continuous response variable, the BMD becomes biased upward when the variance is biased downward. A simulation study was also performed in which the accuracy and precision of the BMD was compared for the two ways of determining the cut-off value. In general, considering BMRs of 1, 5, and 10% (additional risk) the precision of the BMD became higher when the cut-off value was estimated by specifying P(0), relative to the case with a direct determination. Use of the square-root of the maximum-likelihood estimator of the variance in BMD estimation may provide a bias that is reflected by the cut-off formulation (downward bias if specifying P(0), and upward bias if specifying the cut-off, c, directly). This feature may be reduced if an unbiased estimator of the standard deviation is used in the calculations.

Analysis of Variance↗
PubMed2004

Effects of exposure imprecision on estimation of the benchmark dose.

In regression analysis failure to adjust for imprecision in the exposure variable is likely to lead to underestimation of the exposure effect. However, the consequences of exposure error for determination of safe doses of toxic substances have so far not received much attention. The benchmark approach is one of the most widely used methods for development of exposure limits. An important advantage of this approach is that it can be applied to observational data. However, in this type of data, exposure markers are seldom measured without error. It is shown that, if the exposure error is ignored, then the benchmark approach produces results that are biased toward higher and less protective levels. It is therefore important to take exposure measurement error into account when calculating benchmark doses. Methods that allow this adjustment are described and illustrated in data from an epidemiological study on the health effects of prenatal mercury exposure.

Algorithms↗
PubMed1996

A simulation study of the influence of study design on the estimation of benchmark doses for developmental toxicity.

The benchmark dose (BMD)4 approach is emerging as replacement to determination of the No Observed Adverse Effect Level (NOAEL) in noncancer risk assessment. This possibility raises the issue as to whether current study designs for endpoints such as developmental toxicity, optimized for detecting pair wise comparisons, could be improved for the purpose of calculating BMDs. In this paper, we examine various aspects of study design (number of dose groups, dose spacing, dose placement, and sample size per dose group) on BMDs for two endpoints of developmental toxicity (the incidence of abnormalities and of reduced fetal weight). Design performance was judged by the mean-squared error (reflective of the variance and bias) of the maximum likelihood estimate (MLE) from the log-logistic model of the 5% added risk level (the likely target risk for a benchmark calculation), as well as by the length of its 95% confidence interval (the lower value of which is the (BMD). We found that of the designs evaluated, the best results were obtained when two dose levels had response rates above the background level, one of which was near the ED05, were present. This situation is more likely to occur with more, rather than fewer dose levels per experiment. In this instance, there was virtually no advantage in increasing the sample size from 10 to 20 litters per dose group. If neither of the two dose groups with response rates above the background level was near the ED05, satisfactory results were also obtained, but the BMDs tended to be more conservative (i.e., lower). If only one dose level with a response rate above the background level was present, and it was near the ED05, reasonable results for the MLE and BMD were obtained, but here we observed benefits of larger dose group sizes. The poorest results were obtained when only a single group with an elevated response rate was present, and the response rate was much greater than the ED05. The results indicate that while the benchmark dose approach is readily applicable to the standard study designs and generally observed dose-responses in developmental assays, some minor design modifications would increase the accuracy and precision of the BMD.

Abnormalities, Drug-Induced↗