Biomedical subjects
J E Staddon
Publications and source records attributed to J E Staddon.
The dynamics of operant conditioning.
Existing models of operant learning are relatively insensitive to historical properties of behavior and applicable to only limited data sets. This article proposes a minimal set of principles based on short-term and long-term memory mechanisms that can explain the major static and dynamic properties of operant behavior in both single-choice and multiresponse situations. The critical features of the theory are as follows: (a) The key property of conditioning is assessment of the degree of association between responses and reinforcement and between stimuli and reinforcement; (b) the contingent reinforcement is represented by learning expectancy, which is the combined prediction of response-reinforcement and stimulus-reinforcement associations; (c) the operant response is controlled by the interplay between facilitatory and suppressive variables that integrate differences between expected (long-term) and experienced (short-term) events; and (d) very-long-term effects are encoded by a consolidated memory that is sensitive to the entire reinforcement history. The model predicts the major qualitative features of operant phenomena and then suggests an experimental test of theoretical predictions about the joint effects of reinforcement probability and amount of training on operant choice. We hypothesize that the set of elementary principles that we propose may help resolve the long-standing debate about the fundamental variables controlling operant conditioning.
Time and memory: towards a pacemaker-free theory of interval timing.
A popular view of interval timing in animals is that it is driven by a discrete pacemaker-accumulator mechanism that yields a linear scale for encoded time. But these mechanisms are fundamentally at odds with the Weber law property of interval timing, and experiments that support linear encoded time can be interpreted in other ways. We argue that the dominant pacemaker-accumulator theory, scalar expectancy theory (SET), fails to explain some basic properties of operant behavior on interval-timing procedures and can only accommodate a number of discrepancies by modifications and elaborations that raise questions about the entire theory. We propose an alternative that is based on principles of memory dynamics derived from the multiple-time-scale (MTS) model of habituation. The MTS timing model can account for data from a wide variety of time-related experiments: proportional and Weber law temporal discrimination, transient as well as persistent effects of reinforcement omission and reinforcement magnitude, bisection, the discrimination of relative as well as absolute duration, and the choose-short effect and its analogue in number-discrimination experiments. Resemblances between timing and counting are an automatic consequence of the model. We also argue that the transient and persistent effects of drugs on time estimates can be interpreted as well within MTS theory as in SET. Recent real-time physiological data conform in surprising detail to the assumptions of the MTS habituation model. Comparisons between the two views suggest a number of novel experiments.
The choose-short effect and trace models of timing.
The tuned-trace multiple-time-scale (MTS) theory of timing can account both for the puzzling choose-short effect in time-discrimination experiments and for the complementary choose-long effect. But it cannot easily explain why the choose-short effect seems to disappear when the intertrial and recall intervals are signaled by different stimuli. Do differential stimuli actually abolish the effect, or merely improve memory? If the latter, there are ways in which an expanded MTS theory might explain differential-context effects in terms of reduced interference. If the former, there are observational and experimental ways to determine whether differential context favors prospective encoding or some other nontemporal discrimination.
Multiple time scales in simple habituation.
Habituation is the waning of a reflex response to repeated stimulation. Habituation to closely spaced stimuli is faster and more complete than to widely spaced stimuli, but recovery is also more rapid (rate sensitivity). We show that a 2-unit, cascaded-integrator dynamic model can explain in detail an extensive data set on rate-sensitive habituation in the nematode Caenorhabditis elegans. Many apparently complex properties of habituation and learning dynamics may reflect interactions among a small number of processes with different time scales.
Dynamics of waiting in pigeons.
Two experiments used response-initiated delay schedules to test the idea that when food reinforcement is available at regular intervals, the time an animal waits before its first operant response (waiting time) is proportional to the immediately preceding interfood interval (linear waiting; Wynne & Staddon, 1988). In Experiment 1 the interfood intervals varied from cycle to cycle according to one of four sinusoidal sequences with different amounts of added noise. Waiting times tracked the input cycle in a way which showed that they were affected by interfood intervals earlier than the immediately preceding one. In Experiment 2 different patterns of long and short interfood intervals were presented, and the results implied that waiting times are disproportionately influenced by the shortest of recent interfood intervals. A model based on this idea is shown to account for a wide range of results on the dynamics of timing behavior.
Differential vocalization in budgerigars: towards an experimental analysis of naming.
In Experiment 1, 3 budgerigars (Melopsittacus undulatus) were trained with food reinforcement to make low- or high-frequency calls in response to different color stimuli, C1 and C2 (a color-naming task), using a gradual response-differentiation procedure and an automatic call-recognition system. Thus, a call within a certain frequency band was reinforced in the presence of C1 ("C1 call"), and a call within a different band was reinforced in the presence of C2 ("C2 call"). In Experiment 2, all 3 budgerigars were trained in a form-to-color matching-to-sample task, alternating trial by trial with either the color-naming task (2 birds) or an identity color matching-to-sample task (1 bird). Sample stimuli for the new matching-to-sample task were forms (F1 or F2) and comparisons were the same two colors (C1 and C2). Given Sample F1 or F2, birds had to make a call to produce Comparison Pair C1 and C2. With F1 as the sample, a peck on C1 was reinforced; with F2 as the sample, a peck on C2 was reinforced. Although no particular call was specified in the presence of F1 and F2, 2 birds made the C1 call in the presence of F1 and the C2 call in the presence of F2. In Experiment 3, the bird that failed to match form and color calls in Experiment 2 and another bird were first trained in a color-to-form matching-to-sample task: C1 to F3 and C2 to F4. In this task, to produce the comparison pair of forms, a high call (or low for the other bird) was required in the presence of C1, and a low call (or high) was required in the presence of C2. Both birds were then trained with an identity matching-to-sample task in which sample and comparison stimuli were the same two forms, F3 and F4. Trials on the identity task alternated with the color-to-form trials. Although no particular call was required in the presence of Samples F3 and F4, both birds came to make the C1 call in the presence of F3 and the C2 call in the presence of F4. Our technique promises to be useful for the study of emergent vocal relations in budgerigars and other animals.
Cumulative effects model: a response to Williams (1994)
The cumulative effects (CE) model explains free-operant choice by the ratio of total numbers of responses and reinforcements, a probability-like variable. Williams (1994) argues that the model is vulnerable to experiments that disprove melioration, a local probability model. The authors note critical differences between the nonlocal CE model and local probability models that allow the CE model to handle some data with which they are incompatible. All models are simplifications of reality; hence, a model's failures are as revealing as its successes. Williams suggests that simple models may need to be abandoned in favor of a "representational" account. The authors point out that representations must be both acquired and acted on. Acquisition requires processing of responses and reinforcers; action requires decision rules. Models are simply testable suggestions for what these rules and processes might be.
The process of recurrent choice.
Recurrent choice has been studied for many years. A static law, matching, has been established, but there is no consensus on the underlying dynamic process. The authors distinguish between dynamic models in which the model state is identified with directly measurable behavioral properties (performance models) and models in which the relation between behavior and state is indirect (state models). Most popular dynamic choice models are local, performance models. The authors show that behavior in different types of discrimination-reversal experiments and in extinction is not explained by 2 versions of a popular local model and that the nonlocal cumulative-effects model is consistent with matching and that it can duplicate the major properties of recurrent choice in a set of discrimination-reversal experiments. The model can also duplicate results from several other experiments on extinction after complex discrimination training.
"Transitive inference" in multiple conditional discriminations.
We used multiple conditional discriminations to study the inferential abilities of pigeons. Using a five-term stimulus series, pigeons were trained to respond differentially to four overlapping pairs of concurrently presented stimuli: A+ B-, B+ C-, C+ D-, and D+ E-, where plus and minus indicate the stimulus associated with reinforcement and extinction, respectively. Transitive inference in such situations has been defined as a preference for Stimulus B over Stimulus D in a transfer test. We measured this and other untrained preferences (A vs. C, A vs. D, B vs. E, etc.) during nonreinforced test trials. In three experiments using a novel, rapid training procedure (termed autorun), we attempted to identify the necessary and sufficient conditions for transitive inference. We used two versions of autorun: response-based, in which the subject was repeatedly presented with the least well-discriminated stimulus pair; and time-based, in which the subject was repeatedly presented with the least-experienced stimulus pair. In Experiment 1, using response-based autorun, we showed that subjects learned the four stimulus pairs faster than, but at a level comparable to, a previous study on transitive inference in pigeons (Fersen, Wynne, Delius, & Staddon, 1991), but our animals failed to show transitive inference. Experiments 2 and 3 compared time- and response-based autorun. Discrimination performance was maintained, but transitive inference was observed only on the second exposure to the response-based procedure. These results show that inferential behavior in pigeons is not a reliable concomitant of good performance on a series of overlapping discriminations. The necessary and sufficient conditions for transitive inference in pigeons remain to be fully defined.
Pigeons' wait-time responses to transitions in interfood-interval duration: Another look at cyclic schedule performance.
Recent developments reveal that animals can rapidly learn about intervals of time. We studied the nature of this fast-acting process in two experiments. In Experiment 1 pigeons were exposed to a modified fixed-time schedule, in which the time between food rewards (interfood interval) changed at an unpredictable point in each session, either decreasing from 15 to 5 s (step-down) or increasing from 15 to 45 s (step-up). The birds were able to track under both conditions by producing postreinforcement wait times proportional to the preceding interfood-interval duration. However, the time course of responding differed: Tracking was apparently more gradual in the step-up condition. Experiment 2 studied the effect of having both kinds of transitions within the same session by exposing pigeons to a repeating (cyclic) sequence of the interfood-interval values used in Experiment 1. Pigeons detected changes in the input sequence of interfood intervals, but only for a few sessions-discrimination worsened with further training. The dynamic effects we observed do not support a linear waiting process of time discrimination, but instead point to a timing mechanism based on the frequency and recency of prior interfood intervals and not the preceding interfood interval alone.
Temporal control on interval schedules: what determines the postreinforcement pause?
On fixed-interval or response-initiated delay schedules of reinforcement, the average pause following food presentation is proportional to the interfood interval. Moreover, when a number of intervals of different durations occur in a programmed cyclic series, postreinforcement pauses track the changes in interval value. What controls the duration of postreinforcement pauses under these conditions? Staddon, Wynne, and Higa (1991), in their linear waiting model, propose control by the preceding interfood interval. Another possibility is that delay to reinforcement, signaled by a key peck and/or stimulus change, determines the subsequent pause. The experiments reported here examined the role of these two possible time markers by studying the performance of pigeons under a chained cyclic fixed-interval procedure. The data support the linear waiting model, but suggest that more than the immediately preceding interfood interval plays a role in temporal control.
The conventional wisdom of behavior analysis.
Explore the source record for details and available documents.
The conventional wisdom of behavior analysis: Response to comments.
Explore the source record for details and available documents.
The 'superstition' experiment: a reversible figure.
Explore the source record for details and available documents.
Waiting in pigeons: the effects of daily intercalation on temporal discrimination.
Pigeons trained on cyclic-interval schedules adjust their postfood pause from interval to interval within each experimental session. But on regular fixed-interval schedules, many sessions at a given parameter value are usually necessary before the typical fixed-interval "scallop" appears. In the first case, temporal control appears to act from one interfood interval to the next; in the second, it appears to act over hundreds of interfood intervals. The present experiments look at the intermediate case: daily variation in schedule parameters. In Experiments 1 and 2 we show that pauses proportional to interfood interval develop on short-valued response-initiated-delay schedules when parameters are changed daily, that additional experience under this regimen leads to little further improvement, and that pauses usually change as soon as the schedule parameter is changed. Experiment 3 demonstrates identical waiting behavior on fixed-interval and response-initiated-delay schedules when the food delays are short (less than 20 s) and conditions are changed daily. In Experiment 4 we show that daily intercalation prevents temporal control when interfood intervals are longer (25 to 60 s). The results of Experiment 5 suggest that downshifts in interfood interval produce more rapid waiting-time adjustments than upshifts. These and other results suggest that the effects of short interfood intervals seem to be more persistent than those of long intervals.
Dynamics of time discrimination.
Pigeons tracked sinusoidal sequences of interfood intervals (IFIs) by pausing in each interval for a time proportional to the preceding interval. Schedules with either long (30-90 s) or short (5-15 s) values, with variable numbers of cycles and starting phase each day, were tracked about equally well. Tracking was apparently immediate and did not improve across sessions. Experiment 2, in which long and short series were presented on alternate days, showed that tracking on long was more impaired than on short. Experiment 3 showed that occasional presentation of a short IFI in a series of fixed, longer IFIs caused a reduction in waiting time in the next IFI. These effects are evidence for a fast-acting timing mechanism in which waiting time in the IFI N + 1 is strongly determined by the preceding IFI, N. Earlier IFIs have some cumulative effect, but the details remain to be elucidated.
Deductive reasoning in pigeons.
Explore the source record for details and available documents.