This is the sixth in a series of posts based on my notes from the 2018 Art and Science of Animal Training Conference that was held in Irving, Texas on March 24-25, 2018.
While I try to take accurate notes, it is possible that there are errors or that some detail is lacking. If you post a comment or email me, I can try to clarify or provide some additional information. Many thanks to the speakers and organizers who allow me to share. To learn more about the conference, you can visit the conference website.
Many trainers who use applied operant conditioning are familiar with the work of Burrhus Frederic Skinner (B.F. Skinner or Fred), who was the first person to describe and study operant conditioning, but they may not know much about the events that led to his discoveries. In her presentation, Dr. Julie S. Vargas shared the story behind B. F. Skinner’s work, and how the results of his experiments, as well as all the problems and challenges he faced, led to the discovery of operant conditioning and shaping.
The Discovery of Operant Conditioning
Dr. Vargas started at the very beginning with B.F. Skinner’s childhood in Susquehanna, Pennsylvania. From an early age, he was a builder, inventor, and explorer. He was interested in how things worked and what he could learn by studying them. He attended Hamilton College and after graduation he tried to make a career out of writing, but failed.
In 1928, he decided to study psychology at Harvard University. He arrived at Harvard with three books that he thought would be useful. They were Philosophy by Bertrand Russell, Behaviorism by John B. Watson, and Conditioned Reflexes by Ivan P. Pavlov.
Unfortunately, the psychology department was run by Edwin Boring, who was interested in the study of consciousness and “idealistic” philosophy. In fact, many of the ideas embraced by the psychology department (at that time) were ones that had been dismissed by Watson as witchcraft and superstition. Having studied Watson, Skinner had a more “materialist” approach. He was interested in observable behavior and how animals respond to stimuli.
Following his interest in observable behavior, Skinner took a course on Pavlov in the physiology department as well as courses in the psychology department. This meant he was exposed to both the “materialist” view (physiology) and the “idealist” view (experimental psychology). He also took a course on animal behavior with Walter Hunter.
Through the physiology course he met William (W.J.) Crozier, the chairman, who was a follower of Jacques Loeb. Crozier was interested in studying whole organisms, particularly tropisms which are “involuntary orientation by an organism or one of its parts that involves turning or curving by movement or by differential growth and is a positive or negative response to a source of stimulation.”* Crozier was very disparaging about scientists who studied “muscle twitches,” preferring to look at how an animal’s whole behavior is affected by stimuli in their environment. Working under Crozier, Skinner did his first formal experiment on ant behavior.
Crozier had encouraged Skinner to look at how behavior changes based on dependent variables and Skinner decided to look at how antecedent stimuli affect behavior. While Crozier was away for the summer, Skinner started his “Parthenon” project. He set up an experiment using a rat and a special box that he called the Parthenon. His goal was to study the relationship between antecedents and responses, using Pavlov’s work as his guide.
The Parthenon consisted of a start box with a door, three steps, a 6 foot long runway, and a food dish. The rat was placed in a start box, would go through the door, down the steps to the ramp, and end at the food dish. Skinner gave a “click” – an audible antecedent – as the rat reached the bottom of the steps. The entire apparatus was inside another larger box, which contained a peephole so that Skinner could observe the rat’s behavior. He recorded data using a kymograph. He ran the experiment for a month, without any notable results – there was no significant change in the rat’s behavior. He did notice that the rats learned to ignore the “click,” but that kind of adaptation was nothing new.
In his second academic year (1929), Skinner did independent research, which meant he got to do exactly as he pleased (no direct supervision!), and he continued with his Parthenon experiments. At this time, his experiments consisted of “trials.” The rat was placed in the start box at the beginning of the trial and removed from the ramp at the end of the trial. As the experiment continued, Skinner did notice that how he handled the rats between trials affected their behavior and this led him to modify the apparatus, both to allow better data collection and to allow the rats to move from the start box, through the apparatus, and back to the start box on their own.
He made the following changes:
- Extended the runway so it was 8 – 10 feet long
- Added a food dish near the start so the rat would return to the starting position and he did not have to handle it
- Made the runway tip so the rat could not just stay at the food dish (the rat’s weight would tip the runway, so it didn’t stay parked at the food dish)
- Changed his data collection so he did not have to manually record the data. The rats did it themselves because their movement generated the printouts on the kymograph
What I found interesting was how his own behavior was driven by observing and responding to issues with the apparatus and the rat’s behavior, and was not part of some larger plan that he had made ahead of time. His progress was just a repeating cycle of observing the rat’s behavior and making some adjustments to the apparatus to either make it less labor intensive (for him) or to remove possible external variables. If you want more details on all the modifications, the article referenced at the end of this blog has a more complete list of his modifications.
Once he was no longer doing trials and the rats were recording their own behavior, Skinner started to see that there were predictable patterns to eating. There was a “standard” for behavior. Not only that, but a behavior like “eating,” which had previously been thought of as a “free” behavior – one that is not governed by rules, was clearly following some natural rules. The discovery of a standard for a simple behavior like eating was important because once we have standards, we can make comparisons.
Eventually Skinner realized he didn’t need a runway and he could just place the rat in a box and measure the rate of eating. The sound of the food apparatus served as an audible antecedent, but this had become less relevant because he had shifted his focus to the eating behavior. He had improved his method of data collection so that he could accurately record how often the rat ate and the time between eating. While he hadn’t given up on the idea that behavior was controlled by antecedents, he was no longer looking at the response to an antecedent.
His work on the Parthenon continued and he added a door so he could control access to the food. This led to the discovery that if a rat was denied access to food, he would eat more to catch up to his “normal” level when he was allowed access. This work became the basis for both his paper “On the Conditions on Elicitation of Certain Eating Reflexes” and his doctoral thesis. His thesis shows that while he was still thinking of behavior as occurring in response to stimuli – and not as operant behavior, he was questioning the need for an “initiating stimulus.”
In 1931, while working on his thesis, he replaced the door with a bar (lever). Now, instead of having the rat get food each time he opened the door, Skinner could set up experiments where the rat had to press the bar multiple times before getting food. He had also improved his data recording so that he could set up an experiment, leave, and come back later to see what had happened.
In April 1931, he set up an experiment, left, and came back several hours later to find out that the bar (or pellet dispenser) had jammed and the data showed beautiful extinction curves. He continued his experiments and found evidence of all four processes: deprivation, satiation, conditioning, and extinction. All this work was done with a continuous rate of reinforcement, but one day he was running low on food pellets and wondered if the rats would work for less food. He set up the apparatus to reinforce the first press 1 minute after the last press (Fixed Rate: 1) and was amazed that the number of responses increased.
It was five and a half months after the bar jammed that Skinner first mentions (in his notes) the idea of a different kind of science – a brand new theory of learning. He got a 5 year research grant and worked on two and three term contingencies (the ABC’s of behavior), focusing on postcedent analysis. Over this time period, he studied intermittent reinforcement, generalization, and discrimination. He noted that it was impossible to distinguish between a non-hungry conditioned rat and a hungry rat.
In 1938, Skinner published The Behavior of Organisms where he wrote about how the contingencies between actions (responses) and events determine rates of behavior. In his book, there is no mention of agencies like free will. He also does not place much importance on the underlying physiological processes. Knowing how legs work doesn’t tell you where a person will walk. There is no mention of shaping, but he was working with rats in boxes and squirrels on wheels, studying repetitive behavior that didn’t need to be shaped, and food was delivered by the apparatus.
When Hitler invaded Poland in 1939, Skinner got involved in the war effort. One of his projects was teaching pigeons to peck a screen to guide missiles. In 1940, he was waiting for a pigeon to knock a ball around in a box when, tired of waiting, he used a hand switch to deliver food for small approximations of the behavior. Soon, the pigeon was batting the ball around like a soccer player. He was amazed that it was easier to shape behavior by hand, compared to using a machine. Soon after this, trainers started exploring using operant conditioning outside of the laboratory. Some notable events were:
- 1943: Keller and Marian Breland founded Animal Behavior Enterprises (ABE).
- 1951: He had an article published in the magazine, Scientific American.
- 1952: He had an article published in the magazine, Look. For this article, he taught a dog to jump higher and higher as measured by marks on a wall.
- 1960’s: Ken Norris and Karen Pryor started using operant conditioning with dolphins and other marine mammals
Skinner described shaping as a process in which the trainer reinforces one or multiple properties of a behavior. In order to shape effectively, the trainer must be able to select out the property that he wants, a process that Skinner called “selecting by consequences.” In “pure” shaping, there is no prompting or luring. And, while we think of the process as being one where the trainer shapes the animal’s behavior, the animal’s behavior is also shaping the trainer’s behavior.
Skinner believed that shaping had many parallels with natural selection. In both natural selection and shaping:
- Variability is essential.
- The properties of existing actions are selected.
- The process is “materialistic” with no “intention” or “free will” – we may say the dog “wants the food,” but that’s not really correct. Skinner believed the dog was not making a conscious choice.
Skinner also believed that selection operates on three levels:
He did not believe there was any “homunculus” or “free agency” involved in either natural selection or operant conditioning. Conditioning is not a conscious process.
Interestingly enough, in recent years, psychology has become more and more focused on research on the brain – how the brain functions – as a way to explain behavior. But this is a questionable approach (according to Dr. Vargas), because neuronal activity is operant behavior and you always have to look at what the scientists arrange to get the neuronal activity going. This approach also lends itself to circular reasoning where scientists start looking for brain activity to explain physical responses and use physical responses to explain brain activity.
Dr. Vargas finished with the point that operant conditioning is a new science and it’s different. We have to put aside many of our previous ideas about why behavior happens and we still have a long way to go before we can replace “free will” with variables.
* this definition comes from the Merriam-Webster dictionary.
Thanks to Dr. Julie S. Vargas for her assistance with this article. An article that describes some of the same events is available here on the B. F. Skinner organization website. You may also want to read this article on Karen Pryor Clicker Training that describes his discovery of shaping.