Search PubMed⌕ Search

SEARCH · Search PubMed

Results for “Reinforcement Machine Learning”

Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

32 records · Page 2Linked to original sources

Teaching prenatal ultrasound to family medicine residents.

Prenatal ultrasound is a powerful diagnostic tool, but there has been little research on how to teach ultrasound to family physicians. The available evidence supports teaching through didactics followed by supervised scanning. Didactic topics include physics and machine usage, indications, fetal biometry, anatomic survey, practice management, ethical issues, and resources. Supervised scanning reinforces the didactic components of training. A "hand-on-hand" supervised scanning technique is recommended for the transmission of psychomotor skills in these sessions. Curricula for teaching ultrasound should include information on which residents will be taught prenatal ultrasound, who will teach them, how to create time for learning ultrasound skills, and how to test for competency. The literature suggests that competency can be achieved within 25-50 supervised scans. Measures of competency include examination and qualitative analysis of scanning. Competency-based testing needs further development because no uniform standards have been established.

Clinical Competence↗

Autonomous mental development in high dimensional context and action spaces.

Autonomous Mental Development (AMD) of robots opened a new paradigm for developing machine intelligence, using neural network type of techniques and it fundamentally changed the way an intelligent machine is developed from manual to autonomous. The work presented here is a part of SAIL (Self-Organizing Autonomous Incremental Learner) project which deals with autonomous development of humanoid robot with vision, audition, manipulation and locomotion. The major issue addressed here is the challenge of high dimensional action space (5-10) in addition to the high dimensional context space (hundreds to thousands and beyond), typically required by an AMD machine. This is the first work that studies a high dimensional (numeric) action space in conjunction with a high dimensional perception (context state) space, under the AMD mode. Two new learning algorithms, Direct Update on Direction Cosines (DUDC) and High-Dimensional Conjugate Gradient Search (HCGS), are developed, implemented and tested. The convergence properties of both the algorithms and their targeted applications are discussed. Autonomous learning of speech production under reinforcement learning is studied as an example.

Learning↗

Cortical substrates for exploratory decisions in humans.

Decision making in an uncertain environment poses a conflict between the opposing demands of gathering and exploiting information. In a classic illustration of this 'exploration-exploitation' dilemma, a gambler choosing between multiple slot machines balances the desire to select what seems, on the basis of accumulated experience, the richest option, against the desire to choose a less familiar option that might turn out more advantageous (and thereby provide information for improving future decisions). Far from representing idle curiosity, such exploration is often critical for organisms to discover how best to harvest resources such as food and water. In appetitive choice, substantial experimental evidence, underpinned by computational reinforcement learning (RL) theory, indicates that a dopaminergic, striatal and medial prefrontal network mediates learning to exploit. In contrast, although exploration has been well studied from both theoretical and ethological perspectives, its neural substrates are much less clear. Here we show, in a gambling task, that human subjects' choices can be characterized by a computationally well-regarded strategy for addressing the explore/exploit dilemma. Furthermore, using this characterization to classify decisions as exploratory or exploitative, we employ functional magnetic resonance imaging to show that the frontopolar cortex and intraparietal sulcus are preferentially active during exploratory decisions. In contrast, regions of striatum and ventromedial prefrontal cortex exhibit activity characteristic of an involvement in value-based exploitative decision making. The results suggest a model of action selection under uncertainty that involves switching between exploratory and exploitative behavioural modes, and provide a computationally precise characterization of the contribution of key decision-related brain systems to each of these functions.

Choice Behavior↗

Escalation of aggression: experimental studies.

A finding commonly obtained in research using the Buss "aggression machine" is a main effect for trail blocks, indicating an escalation in shock intensity over trails. Theoretical explanations for this effect were tested in a modified verbal operant-conditioning situation. In Experiment 1, subjects could administer any of 10 levels of positive reinforcement to a "learner" for correct verbal responses or any of 10 levels of negative reinforcement to a learner for incorrect responses. Half of the subjects were required to begin with weak, half with strong, reinforcements. Results indicated that, regardless of condition, subjects gave more intense reinforcements as the learning trails progressed. Those who administered negative reinforcements devalued the learner relative to those who administered positive reinforcements. In Experiment 2, a role-playing procedure was used in which subjects administered either positive or negative reinforcements to a learner whose performance either did or did not improve over trials. Again, in all experimental groups, subjects administered increasingly intense reinforcements over trials. The results are interpreted as supporting a disinhibition theory of anti- and prosocial behavior.

Aggression↗

A comparison of human and automated instruction of autistic children.

While it appears reasonable to assume that the autistic child might benefit from the development of programmed instruction and teaching machines to teach imitation, concepts, and receptive vocabulary skills, no systematic research to date has seriously investigated such possibilities. The purpose of this study was to compare the effects of automated versus teacher-controlled instruction in the education of autistic children. Four autistic children, each with extreme deficits in language, social, and self-care behaviors, were trained on a matching-to-sample task under three different instructional situations within an intrasubject replication design and multiple baseline procedure. Analysis of the data showed the following results: (1) The teacher, manually operating the teaching machine, was able to teach and maintain the matching-to-sample task; (2) the same autistic children did not acquire or maintain the task when taught by the machine alone; and (3) the teacher alone (without the machine) was able to teach and maintain the task. The results suggest that automated instruction may, at least, serve as a valuable aid to teachers of autistic children. However, before machines can be used without the participation of a trained teacher, further research appears necessary. Several areas, including the role of motivation in automated instructional settings, the saliency and effectiveness of reinforcers, and the importance of controlling antecedent stimulus conditions and off-task behavior are discussed as areas of primary concern in the development of automated instruction for autistic children.

Adolescent↗

An incremental approach to genetic-algorithms-based classification.

Incremental learning has been widely addressed in the machine learning literature to cope with learning tasks where the learning environment is ever changing or training samples become available over time. However, most research work explores incremental learning with statistical algorithms or neural networks, rather than evolutionary algorithms. The work in this paper employs genetic algorithms (GAs) as basic learning algorithms for incremental learning within one or more classifier agents in a multiagent environment. Four new approaches with different initialization schemes are proposed. They keep the old solutions and use an "integration" operation to integrate them with new elements to accommodate new attributes, while biased mutation and crossover operations are adopted to further evolve a reinforced solution. The simulation results on benchmark classification data sets show that the proposed approaches can deal with the arrival of new input attributes and integrate them with the original input space. It is also shown that the proposed approaches can be successfully used for incremental learning and improve classification rates as compared to the retraining GA. Possible applications for continuous incremental training and feature selection are also discussed.

Algorithms↗

Blood-based DNA methylation and exposure risk scores predict PTSD with high accuracy in military and civilian cohorts.

BACKGROUND: Incorporating genomic data into risk prediction has become an increasingly useful approach for rapid identification of individuals most at risk for complex disorders such as PTSD. Our goal was to develop and validate Methylation Risk Scores (MRS) using machine learning to distinguish individuals who have PTSD from those who do not. METHODS: Elastic Net was used to develop three risk score models using a discovery dataset (n = 1226; 314 cases, 912 controls) comprised of 5 diverse cohorts with available blood-derived DNA methylation (DNAm) measured on the Illumina Epic BeadChip. The first risk score, exposure and methylation risk score (eMRS) used cumulative and childhood trauma exposure and DNAm variables; the second, methylation-only risk score (MoRS) was based solely on DNAm data; the third, methylation-only risk scores with adjusted exposure variables (MoRSAE) utilized DNAm data adjusted for the two exposure variables. The potential of these risk scores to predict future PTSD based on pre-deployment data was also assessed. External validation of risk scores was conducted in four independent cohorts. RESULTS: The eMRS model showed the highest accuracy (92%), precision (91%), recall (87%), and f1-score (89%) in classifying PTSD using 3730 features. While still highly accurate, the MoRS (accuracy = 89%) using 3728 features and MoRSAE (accuracy = 84%) using 4150 features showed a decline in classification power. eMRS significantly predicted PTSD in one of the four independent cohorts, the BEAR cohort (beta = 0.6839, p-0.003), but not in the remaining three cohorts. Pre-deployment risk scores from all models (eMRS, beta = 1.92; MoRS, beta = 1.99 and MoRSAE, beta = 1.77) displayed a significant (p < 0.001) predictive power for post-deployment PTSD. CONCLUSION: Results, especially those from the eMRS, reinforce earlier findings that methylation and trauma are interconnected and can be leveraged to increase the correct classification of those with vs. without PTSD. Moreover, our models can potentially be a valuable tool in predicting the future risk of developing PTSD. As more data become available, including additional molecular, environmental, and psychosocial factors in these scores may enhance their accuracy in predicting the condition and, relatedly, improve their performance in independent cohorts.

DNA methylation↗

Threshold for conditioned suppression using x-rays as the pre-aversive stimulus.

Four male, 12-week-old Sprague-Dawley rats were used to determine the threshold for behavioral response to X-irradiation using the conditioned suppression technique. They were maintained at 80 per cent body weight and initially trained to stable performance on a VI 1 min schedule with 16 per cent sucrose solution as reinforcement. After a stable baseline was obtained, animals were placed in the instrumental conditioning box beneath the X-ray machine for a half-hour session each day. While subjects were actively pressing the lever for reinforcement, 15-sec X-ray exposures of 0.5 R/sec were administered, followed immediately by electric shock. After all animals had exhibited conditioned suppression, the dose-rate was decreased in subsequent sessions in an effort to establish threshold. The results indicate that X-rays at a dose-rate as low as 0.004 R/sec can be an effective pre-aversive stimulus for the rat.

Animals↗

Temporal characteristics of slot machine play in recreational gamblers.

The present study examined the temporal characteristics of play exhibited by recreational slot machine players. 12 women (M = 24 yr., range 22-28) played a computerized version of a slot machine with probability of payoff, i.e., a winning spin, ranging from .2 to .6. Intertrial intervals or the time between the termination of one trial to the initiation of the next trial varied within subjects as a function of wins and losses. Winning trials were associated with larger intertrial intervals and losing trials with smaller intertrial intervals for 10 of the 12 women. No differences were found across probability levels between subjects. Results are discussed in terms of the post-reinforcement pause that occurs on specific schedules of reinforcement.

Adult↗

Self-instruction in dentistry: a critique.

Recent activity and interest in self-instruction and programmed self-instruction emerged from studies which demonstrated the central importance of learner response with prompt consequences. Self-instructional procedures in formal education, e.g., teaching machines, were initially designed to be an improvement over conventional show-and-tell instruction in the arrangement of systematic student response, prompt consequences, and student self-pacing. A minority of self-instructional procedures in dentistry reflect that intent. The evidence discussed suggests that dental education may be the worse for that omission. Suggestions for the improvement of self-instructional in dentistry are presented.

Audiovisual Aids↗

The AcCell series 2000 as a support system for training and evaluation in educational and clinical settings.

Providing effective training, retraining and evaluation programs, including proficiency testing programs, for cytoprofessionals is a challenge shared by many academic and clinical educators internationally. In cytopathology the quality of training has immediately transferable and critically important impacts on satisfactory performance in the clinical setting. Well-designed interactive computer-assisted instruction and testing programs have been shown to enhance initial learning and to reinforce factual and conceptual knowledge. Computer systems designed not only to promote diagnostic accuracy but to integrate and streamline work flow in clinical service settings are candidates for educational adaptation. The AcCell 2000 system, designed as a diagnostic screening support system, offers technology that is adaptable to educational needs during basic and in-service training as well as testing of screening proficiency in both locator and identification skills. We describe the considerations, approaches and applications of the AcCell 2000 system in education programs for both training and evaluation of gynecologic diagnostic screening proficiency.

Automation↗

Adaptive internal state space construction method for reinforcement learning of a real-world agent.

One of the difficulties encountered in the application of the reinforcement learning to real-world problems is the construction of a discrete state space from a continuous sensory input signal. In the absence of a priori knowledge about the task, a straightforward approach to this problem is to discretize the input space into a grid, and to use a lookup table. However, this method suffers from the curse of dimensionality. Some studies use continuous function approximators such as neural networks instead of lookup tables. However, when global basis functions such as sigmoid functions are used, convergence cannot be guaranteed. To overcome this problem, we propose a method in which local basis functions are incrementally assigned depending on the task requirement. Initially, only one basis function is allocated over the entire space. The basis function is divided according to the statistical property of locally weighted temporal difference error (TD error) of the value function. We applied this method to an autonomous robot collision avoidance problem, and evaluated the validity of the algorithm in simulation. The proposed algorithm, which we call adaptive basis division (ABD) algorithm, achieved the task using a smaller number of basis functions than the conventional methods. Moreover, we applied the method to a goal-directed navigation problem of a real mobile robot. The action strategy was learned using a database of sensor data, and it was then used for navigation of a real machine. The robot reached the goal using a smaller number of internal states than with the conventional methods.

Journal Article↗

Associative learning.

This chapter reviews evidence demonstrating the essential role of the cerebellum and its associated circuitry in the learning and memory of classical conditioning of discrete behavioral responses (e.g., eyeblink, limb flexion, head turn). It now seems conclusive that the memory traces for this basic category of associative learning are formed and stored in the cerebellum. Lesion, neuronal recording, electrical microstimulation, and anatomical procedures have been used to identify the essential conditioned stimulus (CS) circuit, including the pontine mossy fiber projections to the cerebellum; the essential unconditioned stimulus (US) reinforcing or teaching circuit, including neurons in the inferior olive (dorsal accessory olive) projecting to the cerebellum as climbing fibers; and the essential conditioned response (CR) circuit, including the interpositus nucleus, its projection via the superior cerebellar peduncle to the magnocellular red nucleus, and rubral projections to premotor and motor nuclei. Each major component of the eyeblink CR circuit was reversibly inactivated both in trained animals and over the course of training. In all cases in trained animals, inactivation abolished the CR (and the UR as well when motor nuclei were inactivated). When animals were trained during inactivation (and not exhibiting CRs) and then tested without inactivation, animals with inactivation of the motor nuclei, red nucleus, and superior peduncle had fully learned, whereas animals with inactivation of a very localized region of the cerebellum (anterior interpositus and overlying cortex) had not learned at all. Consequently, the memory traces are formed and stored in the cerebellum. Several alternative possibilities are considered and ruled out. Both the cerebellar cortex and the interpositus nucleus are involved in the memory storage process, suggesting that a phenomenon-like long-term depression (LTD) is involved in the cerebellar cortex and long-term potentiation (LTP) is involved in the interpositus. The experimental findings reviewed in this chapter provide perhaps the first conclusive evidence for the localization of a basic form of memory storage to a particular brain region, namely the cerebellum, and indicate that the cerebellum is indeed a cognitive machine.

Animals↗

Beware the Medical-Industrial Complex.

". we must guard against the acquisition of unwarranted influence, whether sought or unsought, by the military industrial complex." Dwight D. Eisenhower, 34th President of the United States (1953-1961). Farewell Address, January 17, 1961. If Ike were with us today, he might well expand his views on power and influence to include modern American medicine. The corporatization of health care in the United States has moved rapidly in recent years. Physicians are now in a position that requires us to adapt to an increasingly Darwinian existence. Years of training to be "rugged individualists" pushing the frontiers of medical knowledge have not equipped us to fight corporate battles, nor to justify our treatment decisions to bean counters. When the most important consideration becomes the bottom line, then innovation, creativity, and research diminish in importance. They will, in fact, be selected against because they cost money. Up to now, these have been the hallmarks of American medicine, and we must strive to maintain our position of American leadership in biotechnology. New developments in cancer treatment include expensive technological "bells and whistles" which physicians must ultimately evaluate objectively, despite lush advertisements from companies with obvious vested interests, and authoritative testimonials from biased investigators who presumably believe in their own work to the point of straining credulity and denying common sense. The 3-D image that was created by a computer may look beautiful (and cost accordingly), but it is hard to believe that it can fundamentally change the outcome of patients when it does not add any new data that bear on basic issues. For example, where is the exact edge of the tumor? If one pays through the nose for increasing precision where there is no new accuracy, the purchase appears less attractive, perhaps, than the hype of the salesman or the enthusiasm of a neurosurgeon or a "stereotactic" radiation oncologist (showing biased data, if any at all). For radiation therapy, the 20th century has largely represented progress by creating larger, higher energy machines for treatment. Now, with the 21st century on the horizon, x-ray treatment parameters have probably been optimized over the past 10 years or so. We see no obvious advantage in an x-ray beam beyond about 18 MeV, and none for electrons beyond 20-25 MeV. Exotic particles such as protons, neutrons, and negative pions, though expensive and difficult to deliver, have not yielded yet significant gains in either local control or survival. A variety of new afterloading machines, such as pulsed high dose rate machines, have also been developed with no clear biologic advantage over more standard remote afterloaders. Thus, new equipment will be exploiting issues of convenience, efficiency, and increased throughput (translate: economic improvement, not biological superiority). Today's technology is vastly ahead of our biologic understanding of malignant cells. Our true challenge for the 21st century is to understand the biology of malignant cells and to bring our technology to bear on the biological aspects of cancer. To improve results, cellular manipulations of some sort will probably be necessary. Perhaps these will be mediated through gene therapy, although the manipulation of some genes, to the exclusion of all others, in only tumor cells and in all tumor cells may be a biological challenge beyond our limitations. One is reminded of another time, a decade or two ago, when some tumor immunologists were predicting monoclonal antibodies would soon replace other modalities. Eventually, over time, the immunologists began to appreciate the enormous adaptability that cancer cells possess; the cells are much more than passive receptacles of antigens simply waiting to be destroyed by antibodies. Drugs which affect the function of specific oncogenes, such as the farnesyl transferase inhibitor effect on ras genes, are also quite promising. Clearly, however, there are not "magic bullets" for most cancers. The effects of gene manipulation on patient outcome, if any, are likely to be found only in the setting of combined modality therapy. The most promising clinical research from the last decade or so reinforces the utility of a combined approach in treating cancers. Unfortunately, combined treatments and the development of new combined treatments are expensive. In today's world of corporate medicine and managed care, academic centers are under considerable pressures. They are perceived as being too expensive, and thus they are in danger of being shut out of contractual arrangements with third-party representatives. If these centers are to survive, they must reform themselves: One, they must establish meaningful relationships with community hospitals and community physicians. Two, the academic programs must learn to minimize charges and deliver a true multidisciplinary service to patients in an efficient way. Three, the centers must learn to invest wisely in new technologies that community hospitals cannot and will not be expected to support. This "wisdom" refers to selection of technology that truly may have impact on the outcome of patients' lives by early detection or by treatment. The euphoria associated with projected gains of some investigational treatments can be misleading: randomized prospective trials have shown in the past that postoperative radiation following a complete resection of lung cancer, breast cancer, or rectal cancer adds a major improvement to local control, but with relatively little improvement in survival. How many times will it be necessary for companies and self-impressed investigators to rediscover this particular wheel? We must remember that every new therapy costs money, so we must focus our research time and money in promising areas. If cost is allowed to be the most important mitigator of health care, research as we know it will end. The relative lack of new therapies means that some people will die prematurely because of our lack of foresight. As scientists, we must be seen as providers of a value-added product. Improvement in cancer cure rates has been frustratingly slow. We work against a clever, tenacious adversary - both in the clinic and in the corporate board room. It is our responsibility to tout our accomplishments, admit our failures, and provide progressively better basic and clinical research with an eye toward future improvements in outcome. We must not be seen as yet another special interest come to drink at the well of public spending, but as advocates for the public good. If we fail to become important to those who control medical spending, we will be unable to make any important long-term contribution to those who matter most - our patients.

Journal Article↗