Search PubMed⌕ Search

SEARCH · Search PubMed

Results for “Software Validation”

Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 685 records · Page 38Linked to original sources

Automatic visual to tactile translation--Part II: Evaluation of the TACTile Image Creation System.

This is the second part of a two-part paper that develops a method for the automatic conversion of images from visual to tactile form. In Part I, a variety of topics were reviewed including issues in human factors, access technology for tactile graphics production, and image processing. In this part, the material presented in the first part is used to motivate, develop, and support the methods used in the development of a prototype visual-to-tactile translator called the TACTile Image Creation System (TACTICS). The specific choices made in the design of the system are discussed and justified, including selection of software platform, tactile output format, tactile image creation procedure, aggregate image processing sequences used, and principles from the discipline of psychophysics. The results of four experiments on tactile image discrimination, identification, and comprehension are reported and discussed, and future directions in this area are proposed.

Adult↗

Design and evaluation of a software prototype for participatory planning of environmental adaptations.

A software prototype to support the planning process for adapting home and work environments for people with physical disabilities was designed and later evaluated. The prototype exploits low-cost three-dimensional (3-D) graphics products in the home computer market. The essential features of the prototype are: interactive rendering with optional hardware acceleration, interactive walk-throughs, direct manipulation tools for moving objects and measuring distances, and import of 3-D-objects from a library. A usability study was conducted, consisting of two test sessions (three weeks apart) and a final interview. The prototype was then tested and evaluated by representatives of future users: five occupational therapist students, and four persons with physical disability, with no previous experience of the prototype. Emphasis in the usability study was placed on the prototype's efficiency and learnability. We found that it is possible to realise a planning tool for environmental adaptations, both regarding usability and technical efficiency. The usability evaluation confirms our findings from previous case studies, regarding the relevance and positive attitude towards this kind of planning tool. Although the prototype was found to be satisfactorily efficient for the basic tasks, the paper presents several suggestions for improvement of future prototype versions.

Activities of Daily Living↗

Controlling mouse pointer position using an infrared head-operated joystick.

This paper describes the motivation for and the design considerations of a low-cost head-operated joystick. The paper briefly summarizes the requirements of head-operated mouse pointer control for people with disabilities before discussing a set of technological approaches that can be used to satisfy these requirements. The paper focuses on the design of a head-operated joystick that uses infrared light emitting diodes (LED's) and photodetectors to determine head position, which is subsequently converted into signals that emulate a Microsoft mouse. There are two significant findings. The first is that, while nonideal device characteristics might appear to make the joystick difficult to use, users naturally compensate for nonlinearities, in a transparent manner, because of visual feedback of mouse pointer position. The second finding, from relatively informal, independent trials, indicates that disabled users prefer a head-operated device that has the characteristics of a joystick (a relative pointing device) to those of a mouse (an absolute pointing device).

Attitude to Health↗

Fuzzy EMG classification for prosthesis control.

This paper proposes a fuzzy approach to classify single-site electromyograph (EMG) signals for multifunctional prosthesis control. While the classification problem is the focus of this paper, the ultimate goal is to improve myoelectric system control performance, and classification is an essential step in the control. Time segmented features are fed to a fuzzy system for training and classification. In order to obtain acceptable training speed and realistic fuzzy system structure, these features are clustered without supervision using the Basic Isodata algorithm at the beginning of the training phase, and the clustering results are used in initializing the fuzzy system parameters. Afterwards, fuzzy rules in the system are trained with the back-propagation algorithm. The fuzzy approach was compared with an artificial neural network (ANN) method on four subjects, and very similar classification results were obtained. It is superior to the latter in at least three points: slightly higher recognition rate; insensitivity to overtraining; and consistent outputs demonstrating higher reliability. Some potential advantages of the fuzzy approach over the ANN approach are also discussed.

Algorithms↗

Gait event detection for FES using accelerometers and supervised machine learning.

Rule based detectors were used with a single cluster of accelerometers attached to the shank for the real time detection of the main phases of normal gait during walking. The gait phase detectors were synthesized from two rule induction algorithms, Rough Sets (RS) and Adaptive Logic Networks (ALNs), and compared with to a previously reported stance/swing detector based on a hand crafted, rule based algorithm. Data was sampled at 100 Hz and the detection errors determined at each sample for 50 steps. For three able bodied subjects, the sample by sample accuracy of stance/swing detection ranged within 94-97%, 87-94%, and 87-95% for the RS, ALN, and the handcrafted methods, respectively. A heuristically formulated postdetector filter improved the RS and ALN detectors' accuracy to 98%. RS and ALN also detected five gait phases to an overall accuracy of 82-89% and 86-91%, respectively. The postdetector filter localized the errors to the phase transitions, but did not change the detection accuracy. The average duration of the error at each transition was 40 ms and 23 ms for RS and ALN, respectively. When implemented on a microcontroller, the RS-based detector executed ten times faster and required one tenth of the memory than the ALN-based detector.

Acceleration↗

An information theoretic criterion for evaluating the quality of 3-D reconstructions from video.

Even though numerous algorithms exist for estimating the three-dimensional (3-D) structure of a scene from its video, the solutions obtained are often of unacceptable quality. To overcome some of the deficiencies, many application systems rely on processing more data than necessary, thus raising the question: how is the accuracy of the solution related to the amount of data processed by the algorithm? Can we automatically recognize situations where the quality of the data is so bad that even a large number of additional observations will not yield the desired solution? Previous efforts to answer this question have used statistical measures like second order moments. They are useful if the estimate of the structure is unbiased and the higher order statistical effects are negligible, which is often not the case. This paper introduces an alternative information-theoretic criterion for evaluating the quality of a 3-D reconstruction. The accuracy of the reconstruction is judged by considering the change in mutual information (MI) (termed as the incremental MI) between a scene and its reconstructions. An example of 3-D reconstruction from a video sequence using optical flow equations and known noise distribution is considered and it is shown how the MI can be computed from first principles. We present simulations on both synthetic and real data to demonstrate the effectiveness of the proposed criterion.

Algorithms↗

A statistical evaluation of recent full reference image quality assessment algorithms.

Measurement of visual quality is of fundamental importance for numerous image and video processing applications, where the goal of quality assessment (QA) algorithms is to automatically assess the quality of images or videos in agreement with human quality judgments. Over the years, many researchers have taken different approaches to the problem and have contributed significant research in this area and claim to have made progress in their respective domains. It is important to evaluate the performance of these algorithms in a comparative setting and analyze the strengths and weaknesses of these methods. In this paper, we present results of an extensive subjective quality assessment study in which a total of 779 distorted images were evaluated by about two dozen human subjects. The "ground truth" image quality data obtained from about 25,000 individual human quality judgments is used to evaluate the performance of several prominent full-reference image quality assessment algorithms. To the best of our knowledge, apart from video quality studies conducted by the Video Quality Experts Group, the study presented in this paper is the largest subjective image quality study in the literature in terms of number of images, distortion types, and number of human judgments per image. Moreover, we have made the data from the study freely available to the research community. This would allow other researchers to easily report comparative results in the future.

Algorithms↗

A meta-analysis of the training effectiveness of virtual reality surgical simulators.

The increasing use of virtual reality (VR) simulators in surgical training makes it imperative that definitive studies be performed to assess their training effectiveness. Indeed, in this paper we report the meta-analysis of the efficacy of virtual reality simulators in: 1) the transference of skills from the simulator training environment to the operating room, and 2) their ability to discriminate between the experience levels of their users. The task completion time and the error score were the two study outcomes collated and analyzed in this meta-analysis. Sixteen studies were identified from a computer-based literature search (1996-2004). The meta-analysis of the random effects model (because of the heterogeneity of the data) revealed that training on virtual reality simulators did lessen the time taken to complete a given surgical task as well as clearly differentiate between the experienced and the novice trainees. Meta-analytic studies such as the one reported here would be very helpful in the planning and setting up of surgical training programs and for the establishment of reference 'learning curves' for a specific simulator and surgical task. If any such programs already exist, they can then indicate the improvements to be made in the simulator used, such as providing for more variety in their case scenarios based on the state and/or rate of learning of the trainee.

Computer Simulation↗

Object dependency of resolution in reconstruction algorithms with interiteration filtering applied to PET data.

In this paper, we study the resolution properties of those algorithms where a filtering step is applied after every iteration. As concrete examples we take filtered preconditioned gradient descent algorithms for the Poisson log likelihood for PET emission data. For nonlinear estimators, resolution can be characterized in terms of the linearized local impulse response (LLIR). We provide analytic approximations for the LLIR for the class of algorithms mentioned above. Our expressions clearly show that when interiteration filtering (with linear filters) is used, the resolution properties are, in most cases, spatially varying, object dependent and asymmetric. These nonuniformities are solely due to the interaction between the filtering step and the Poisson noise model. This situation is similar to penalized likelihood reconstructions as studied previously in the literature. In contrast, nonregularized and postfiltered maximum-likelihood expectation maximization (MLEM) produce images with nearly "perfect" uniform resolution when convergence is reached. We use the analytic expressions for the LLIR to propose three different approaches to obtain nearly object independent and uniform resolution. Two of them are based on calculating filter coefficients on a pixel basis, whereas the third one chooses an appropriate preconditioner. These three approaches are tested on simulated data for the filtered MLEM algorithm or the filtered separable paraboloidal surrogates algorithm. The evaluation confirms that images obtained using our proposed regularization methods have nearly object independent and uniform resolution.

Algorithms↗

Comments on: A methodology for evaluation of boundary detection algorithms on medical images.

In this paper we analyze a result previously published about a comparison between two statistical tests used for evaluation of boundary detection algorithms on medical images. We conclude that the statement made by Chalana and Kim (1997) about the performance of the percentage test has a weak theoretical foundation, and according to our results, is not correct. In addition, we propose a one-sided hypothesis test for which the acceptance region can be determined in advance, as opposed to the two-sided confidence intervals proposed in the original paper, which change according to the estimated quantity.

Abdomen↗

Evaluation of JPEG 2000 encoder options: human and model observer detection of variable signals in X-ray coronary angiograms.

Previous studies have evaluated the effect of the new still image compression standard JPEG 2000 using nontask based image quality metrics, i.e., peak-signal-to-noise-ratio (PSNR) for nonmedical images. In this paper, the effect of JPEG 2000 encoder options was investigated using the performance of human and model observers (nonprewhitening matched filter with an eye filter, square-window Hotelling, Laguerre-Gauss Hotelling and channelized Hotelling model observer) for clinically relevant visual tasks. Two tasks were investigated: the signal known exactly but variable task (SKEV) and the signal known statistically task (SKS). Test images consisted of real X-ray coronary angiograms with simulated filling defects (signals) inserted in one of the four simulated arteries. The signals varied in size and shape. Experimental results indicated that the dependence of task performance on the JPEG 2000 encoder options was similar for all model and human observers. Model observer performance in the more tractable and computationally economic SKEV task can be used to reliably estimate performance in the complex but clinically more realistic SKS task. JPEG 2000 encoder settings different from the default ones resulted in greatly improved model and human observer performance in the studied clinically relevant visual tasks using real angiography backgrounds.

Algorithms↗

Standardized evaluation methodology for 2-D-3-D registration.

In the past few years, a number of two-dimensional (2-D) to three-dimensional (3-D) (2-D-3-D) registration algorithms have been introduced. However, these methods have been developed and evaluated for specific applications, and have not been directly compared. Understanding and evaluating their performance is therefore an open and important issue. To address this challenge we introduce a standardized evaluation methodology, which can be used for all types of 2-D-3-D registration methods and for different applications and anatomies. Our evaluation methodology uses the calibrated geometry of a 3-D rotational X-ray (3DRX) imaging system (Philips Medical Systems, Best, The Netherlands) in combination with image-based 3-D-3-D registration for attaining a highly accurate gold standard for 2-D X-ray to 3-D MR/CT/3DRX registration. Furthermore, we propose standardized starting positions and failure criteria to allow future researchers to directly compare their methods. As an illustration, the proposed methodology has been used to evaluate the performance of two 2-D-3-D registration techniques, viz. a gradient-based and an intensity-based method, for images of the spine. The data and gold standard transformations are available on the internet (http://www.isi.uu.nl/Research/Databases/).

Algorithms↗

A protocol for evaluation of similarity measures for rigid registration.

The accuracy and robustness of a registration method depend on a number of factors, such as imaging modality, image content and image degrading effects, the class of spatial transformation used for registration, similarity measure, optimization, and numerous implementation details. The complex interdependence of these factors makes the assessment of the influence of a particular factor on registration difficult, although it is often desirable to have some estimate of such influences prior to registration. The similarity measure used to create the cost function is one of the factors that most influences the quality of registration. Traditionally, limited information on the behavior of a similarity measure is obtained either by studying the quality of the final registration or by drawing plots of similarity measure values obtained by translating or rotating one image relative to the "gold standard." In this paper, we present a protocol for a more thorough, optimization-independent, and systematic statistical evaluation of similarity measures. This protocol estimates a similarity measure's capture range, the number, location and extent of local optima, and the accuracy and distinctiveness of the global optimum. To show that the proposed evaluation protocol is viable, we have conducted several experiments with nine similarity measures and real computed tomography and magnetic resonance (MR) images of a spine phantom, MR brain images, and MR and positron emission tomography brain images, for which "gold standard" registrations were available. We have also studied the impact of histogram bin size on the behavior of nine similarity measures. The proposed evaluation protocol is useful for selecting the best similarity measure and corresponding optimization method for a particular application, as well as for studying the influence of sampling, interpolation, histogram bin size, partial image overlap, and image degradation, such as noise, intensity inhomogeneity, and geometrical distortions on the behavior of a similarity measure.

Algorithms↗

Towards a new tool for the evaluation of the quality of ultrasound compressed images.

This paper presents a new tool for the evaluation of ultrasound image compression. The goal is to measure the image quality as easily as with a statistical criterion, and with the same reliability as the one provided by the medical assessment. An initial experiment is proposed to medical experts and represents our reference value for the comparison of evaluation criteria. Twenty-one statistical criteria are selected from the literature. A cumulative absolute similarity measure is defined as a distance between the criterion to evaluate and the reference value. A first fusion method based on a linear combination of criteria is proposed to improve the results obtained by each of them separately. The second proposed approach combines different statistical criteria and uses the medical assessment in a training phase with a support vector machine. Some experimental results are given and show the benefit of fusion.

Algorithms↗