ASAT Conference 2018: Dr. Julie S. Vargas on “B.F. Skinner’s Discovery of the Operant and of Shaping.”

skinner xsmall

This is the sixth in a series of posts based on my notes from the 2018 Art and Science of Animal Training Conference that was held in Irving, Texas on March 24-25, 2018.

While I try to take accurate notes, it is possible that there are errors or that some detail is lacking. If you post a comment or email me, I can try to clarify or provide some additional information. Many thanks to the speakers and organizers who allow me to share.  To learn more about the conference, you can visit the conference website.

Many trainers who use applied operant conditioning are familiar with the work of Burrhus Frederic Skinner (B.F. Skinner or Fred), who was the first person to describe and study operant conditioning, but they may not know much about the events that led to his discoveries. In her presentation, Dr. Julie S. Vargas shared the story behind B. F. Skinner’s work, and how the results of his experiments, as well as all the problems and challenges he faced, led to the discovery of operant conditioning and shaping.

The Discovery of Operant Conditioning

Dr. Vargas started at the very beginning with B.F. Skinner’s childhood in Susquehanna, Pennsylvania.  From an early age, he was a builder, inventor, and explorer. He was interested in how things worked and what he could learn by studying them. He attended Hamilton College and after graduation he tried to make a career out of writing, but failed.

In 1928, he decided to study psychology at Harvard University. He arrived at Harvard with three books that he thought would be useful. They were Philosophy by Bertrand Russell, Behaviorism by John B. Watson, and Conditioned Reflexes by Ivan P. Pavlov.

Unfortunately, the psychology department was run by Edwin Boring, who was interested in the study of consciousness and “idealistic” philosophy. In fact, many of the ideas embraced by the psychology department (at that time) were ones that had been dismissed by Watson as witchcraft and superstition. Having studied Watson, Skinner had a more “materialist” approach. He was interested in observable behavior and how animals respond to stimuli.

Following his interest in observable behavior, Skinner took a course on Pavlov in the physiology department as well as courses in the psychology department. This meant he was exposed to both the “materialist” view (physiology) and the “idealist” view (experimental psychology). He also took a course on animal behavior with Walter Hunter.

Through the physiology course he met William (W.J.) Crozier, the chairman, who was a follower of Jacques Loeb. Crozier was interested in studying whole organisms, particularly tropisms which are “involuntary orientation by an organism or one of its parts that involves turning or curving by movement or by differential growth and is a positive or negative response to a source of stimulation.”* Crozier was very disparaging about scientists who studied “muscle twitches,” preferring to look at how an animal’s whole behavior is affected by stimuli in their environment. Working under Crozier, Skinner did his first formal experiment on ant behavior.

Crozier had encouraged Skinner to look at how behavior changes based on dependent variables and Skinner decided to look at how antecedent stimuli affect behavior. While Crozier was away for the summer, Skinner started his “Parthenon” project.  He set up an experiment using a rat and a special box that he called the Parthenon. His goal was to study the relationship between antecedents and responses, using Pavlov’s work as his guide.

The Parthenon consisted of a start box with a door, three steps, a 6 foot long runway, and a food dish. The rat was placed in a start box, would go through the door, down the steps to the ramp, and end at the food dish. Skinner gave a “click” – an audible antecedent – as the rat reached the bottom of the steps. The entire apparatus was inside another larger box, which contained a peephole so that Skinner could observe the rat’s behavior. He recorded data using a kymograph. He ran the experiment for a month, without any notable results – there was no significant change in the rat’s behavior.  He did notice that the rats learned to ignore the “click,” but that kind of adaptation was nothing new.

In his second academic year (1929), Skinner did independent research, which meant he got to do exactly as he pleased (no direct supervision!), and he continued with his Parthenon experiments. At this time, his experiments consisted of “trials.” The rat was placed in the start box at the beginning of the trial and removed from the ramp at the end of the trial. As the experiment continued, Skinner did notice that how he handled the rats between trials affected their behavior and this led him to modify the apparatus, both to allow better data collection and to allow the rats to move from the start box, through the apparatus, and back to the start box on their own.

He made the following changes:

  • Extended the runway so it was 8 – 10 feet long
  • Added a food dish near the start so the rat would return to the starting position and he did not have to handle it
  • Made the runway tip so the rat could not just stay at the food dish (the rat’s weight would tip the runway, so it didn’t stay parked at the food dish)
  • Changed his data collection so he did not have to manually record the data. The rats did it themselves because their movement generated the printouts on the kymograph

What I found interesting was how his own behavior was driven by observing and responding to issues with the apparatus and the rat’s behavior, and was not part of some larger plan that he had made ahead of time. His progress was just a repeating cycle of observing the rat’s behavior and making some adjustments to the apparatus to either make it less labor intensive (for him) or to remove possible external variables. If you want more details on all the modifications, the article referenced at the end of this blog has a more complete list of his modifications.

Once he was no longer doing trials and the rats were recording their own behavior, Skinner started to see that there were predictable patterns to eating. There was a “standard” for behavior. Not only that, but a behavior like “eating,” which had previously been thought of as a “free” behavior – one that is not governed by rules, was clearly following some natural rules. The discovery of a standard for a simple behavior like eating was important because once we have standards, we can make comparisons.

Eventually Skinner realized he didn’t need a runway and he could just place the rat in a box and measure the rate of eating. The sound of the food apparatus served as an audible antecedent, but this had become less relevant because he had shifted his focus to the eating behavior. He had improved his method of data collection so that he could accurately record how often the rat ate and the time between eating. While he hadn’t given up on the idea that behavior was controlled by antecedents, he was no longer looking at the response to an antecedent.

His work on the Parthenon continued and he added a door so he could control access to the food. This led to the discovery that if a rat was denied access to food, he would eat more to catch up to his “normal” level when he was allowed access.  This work became the basis for both his paper “On the Conditions on Elicitation of Certain Eating Reflexes” and his doctoral thesis. His thesis shows that while he was still thinking of behavior as occurring in response to stimuli – and not as operant behavior, he was questioning the need for an “initiating stimulus.”

In 1931, while working on his thesis, he replaced the door with a bar (lever). Now, instead of having the rat get food each time he opened the door, Skinner could set up experiments where the rat had to press the bar multiple times before getting food. He had also improved his data recording so that he could set up an experiment, leave, and come back later to see what had happened.

In April 1931, he set up an experiment, left, and came back several hours later to find out that the bar (or pellet dispenser) had jammed and the data showed beautiful extinction curves. He continued his experiments and found evidence of all four processes: deprivation, satiation, conditioning, and extinction. All this work was done with a continuous rate of reinforcement, but one day he was running low on food pellets and wondered if the rats would work for less food. He set up the apparatus to reinforce the first press 1 minute after the last press (Fixed Rate: 1) and was amazed that the number of responses increased.

It was five and a half months after the bar jammed that Skinner first mentions (in his notes) the idea of a different kind of science – a brand new theory of learning. He got a 5 year research grant and worked on two and three term contingencies (the ABC’s of behavior), focusing on postcedent analysis. Over this time period, he studied intermittent reinforcement, generalization, and discrimination. He noted that it was impossible to distinguish between a non-hungry conditioned rat and a hungry rat.

In 1938, Skinner published The Behavior of Organisms where he wrote about how the contingencies between actions (responses) and events determine rates of behavior. In his book, there is no mention of agencies like free will. He also does not place much importance on the underlying physiological processes. Knowing how legs work doesn’t tell you where a person will walk. There is no mention of shaping, but he was working with rats in boxes and squirrels on wheels, studying repetitive behavior that didn’t need to be shaped, and food was delivered by the apparatus.

Shaping

When Hitler invaded Poland in 1939, Skinner got involved in the war effort. One of his projects was teaching pigeons to peck a screen to guide missiles. In 1940, he was waiting for a pigeon to knock a ball around in a box when, tired of waiting, he used a hand switch to deliver food for small approximations of the behavior. Soon, the pigeon was batting the ball around like a soccer player. He was amazed that it was easier to shape behavior by hand, compared to using a machine. Soon after this, trainers started exploring using operant conditioning outside of the laboratory. Some notable events were:

  • 1943: Keller and Marian Breland founded Animal Behavior Enterprises (ABE).
  • 1951: He had an article published in the magazine, Scientific American.
  • 1952: He had an article published in the magazine, Look. For this article, he taught a dog to jump higher and higher as measured by marks on a wall.
  • 1960’s: Ken Norris and Karen Pryor started using operant conditioning with dolphins and other marine mammals

Skinner described shaping as a process in which the trainer reinforces one or multiple properties of a behavior. In order to shape effectively, the trainer must be able to select out the property that he wants, a process that Skinner called “selecting by consequences.” In “pure” shaping, there is no prompting or luring. And, while we think of the process as being one where the trainer shapes the animal’s behavior, the animal’s behavior is also shaping the trainer’s behavior.

Skinner believed that shaping had many parallels with natural selection. In both natural selection and shaping:

  • Variability is essential.
  • The properties of existing actions are selected.
  • The process is “materialistic” with no “intention” or “free will” – we may say the dog “wants the food,” but that’s not really correct. Skinner believed the dog was not making a conscious choice.

Skinner also believed that selection operates on three levels:

  • Genetic
  • Individual
  • Cultural

He did not believe there was any “homunculus” or “free agency” involved in either natural selection or operant conditioning. Conditioning is not a conscious process.

Interestingly enough, in recent years, psychology has become more and more focused on research on the brain – how the brain functions – as a way to explain behavior. But this is a questionable approach (according to Dr. Vargas), because neuronal activity is operant behavior and you always have to look at what the scientists arrange to get the neuronal activity going. This approach also lends itself to circular reasoning where scientists start looking for brain activity to explain physical responses and use physical responses to explain brain activity.

Dr. Vargas finished with the point that operant conditioning is a new science and it’s different. We have to put aside many of our previous ideas about why behavior happens and we still have a long way to go before we can replace “free will” with variables.

* this definition comes from the Merriam-Webster dictionary.

Thanks to Dr. Julie S. Vargas for her assistance with this article. An article that describes some of the same events is available here on the B. F. Skinner organization website. You may also want to read this article on Karen Pryor Clicker Training that describes his discovery of shaping.

Recommended Reading: Karen Pryor

on behavior

“Until the age of 43, Phil Wylie was a terrible dancer. He was not just your normal bad dancer, who rocks timidly from side to side in time to the music. He was a really rotten dancer, the sort that sweats, bumps into people, doesn’t keep time and steps on your feet. After dancing with Phil, you were relieved to be able to sit down.”

That quote comes from Karen Pryor’s book “On Behavior.” Phil Wylie was her father, and yes – he did learn to dance.  The story of how he learned to dance is one of many that she shares in the book, which is a collection of essays and research. How did he learn to dance? Slowly and systematically, with careful instruction.  Karen writes:

“His teacher took the first ten lessons or so to teach him to tap his foot in time to the music. For him to master learning to walk in a straight line in time to the music required new miracles of coordination. He also needed lessons in stepping back and forth and sideways according to a pattern, a skill Phil was temperamentally unsuited for.”

Doesn’t the description of Phil’s dancing skills sound like some horses you know? Or perhaps it’s how you feel when you are working with your horse? I’ve certainly met lots of horses that started off as inept dance partners.  But, I’ve also seen how those horses can be transformed through careful training that teaches them about their own balance, body awareness, and how to coordinate their own movement with the movement of their trainers.

I chose to share these quotes for several reasons. The first one is that I think we are all inspired by success stories, especially when the stories remind us that we can achieve great things if we take our time and break the skills needed down into achievable steps. I find it helpful to read about how other people overcome difficulties. Seeing the similarities and differences between training different sports and activities can also generate some novel thinking about how I might approach any current training projects.

I also thought that the quote had some great parallels with horse training. It’s no fun to handle a horse that has no sense of his own space and lacks the coordination to respond to the trainer’s cues. I know it’s a bit of a trite expression, but I really do want my horse to be able to respond as smoothly and gracefully as a nice dance partner.  As I get into more advanced ridden work, it’s become clearer and clearer to me that I have to learn to be more self-aware, have better body control, and be more clear and consistent in my cues and aids. At the same time that I carefully work with my horse to develop her coordination, flexibility, and strength, I have to do the same thing with myself. We can learn it together, but I need to focus on me as much as I focus on her.

The third reason I wanted to share a quote from this book is that I’d like to remind everyone that there is a wealth of information out there in the form of books and articles by some of the early practitioners of applied operant conditioning. These are people who have spent thousands of hours observing and training animals. Reading about their experiences is one of the best ways to get some insight into how trainers think, plan, and implement their training. I also find it interesting to read and see how much has changed in animal training over the last 50 years. We’ve come a long way.

I chose to start with Karen Pryor because she has always been a masterful storyteller and she has such a wide range of experience both across species and for different applications. If you are interested in learning more about Karen, she now has her own website, karenwpryor.com.  If you are looking for additional reading, Karen has a number of other books.  “Don’t Shoot the Dog” is an excellent resource for new clicker trainers, but you can learn a lot about training and the history of animal training by reading any of the others. You can also find recommended books on the “Additional Resources” page for my book, “Teaching Horses with Positive Reinforcement.”

 

The Book – Thoughts on Chapter One: What is Clicker Training?

coversmallsnip

Clicker training changed my life. Usually, when I say that to people, they are surprised. They can understand how clicker training might have changed the way I train horses, but my life? That seems a little dramatic. But, it’s true.

Once I started to understand why behavior happens and how I can influence it, my whole view of the world changed. Not only did I start to see connections and associations that I might previously have missed, but I became aware of the importance of every little interaction, and how all those little interactions accumulate to create the relationships I have with my animals, friends, family, and the world.

When I started to put material together for the book, I felt it was important to include some articles about how clicker training changes the people who use it. Clicker training is not just another training method. It’s a different way of thinking about and responding to behavior, and it changes the trainer as much as it changes the individual being trained. In my case, I found it was both exciting and fascinating to be able to see more clearly what was happening and to think about ways to change behavior using reinforcement.

With this in mind, the first chapter contains introductory material that describes both the how and the why of clicker training. Not the scientific “why,” but the reason people choose and stick with clicker training. The following quote is from the article “Why Do I Clicker Train?” and was originally published in 2014, but it’s still true today.

I’ve had a few conversations recently with non-clicker trainers who want to know why I clicker train. Over the years, I have had different answers to this. I can give them the “based on science” answer or explain learning theory and the differences between positive-reinforcement based training and other training methods. I can also talk about how clicker training builds relationships and allows me to communicate with my horse.

But thinking about it today, I realized that while these are all good reasons (and they are part of why I do it), the real reason I clicker train is that I find it reinforcing to have horses that want to engage with me. There are things about the process of clicker training and training behaviors that make me want to continue doing it from a practical point of view, but if my horses didn’t enjoy the process and choose to interact with me, I doubt I would still be doing it.

This reminds me a bit of something Kathy Sdao said at my first ClickerExpo. She pointed out that many people were taking notes and asked what was reinforcing the note taking. She got the usual answers of wanting to remember information, learning better when you write it down, etc. Her answer was that note taking was reinforced by the ink in the pen. If your pen stopped working because it was out of ink, what would happen to the note taking behavior? It would stop.

So, in a way, all the little daily interactions I have with my horses are like the ink in the pen. They provide a constant stream of little reinforcers. I love that my horses are actively paying attention to me and looking for information that will tell them what behavior I might like. I like that they are choosing to interact with me instead of avoiding me or being indifferent. I don’t want to spend time with horses that are shut down or don’t expect life to be fun and full of good things.

A non-clicker trainer will argue that it’s all about the food, but I just see the food as part of a bigger picture. Anyone who has tried to build a relationship knows that you have to start by creating some positive associations. So, we start with food, but then it becomes more about wanting to interact with each other, and I believe that the horses get as much reinforcement out of having me pay attention to them as I get out of having them pay attention to me.

If you want to share why you clicker train, I always love to hear how people got started and what keeps them motivated. You can post a comment or email me (katie@equineclickertraining.com) directly.

Want to learn more about the book? You can find it on Amazon at: Teaching Horses with Positive Reinforcement

Teaching Horses with Positive Reinforcement: The Book…

coversmallsnipIn January 2016, I had an idea for a “little project.” I thought I would take some of the most useful articles from my website, blog, and Facebook page, organize them into a logical format and publish them as a set of “collected articles.” I organized the material into four volumes (beginner, advanced beginner, intermediate, and ridden work) and got to work.

But, I quickly realized that just doing the first one would be a challenge, and I had to consider how much time and energy I could spend on this project. In the end, I decided to focus on the material for beginners and save the rest for later. Even though I decided not to cover more “advanced” behaviors, my hope is that the book provides a good foundation and trainers can use the resources and references that I have included if they wish to learn more.

Now, 2 and a half years later, I am finally publishing my set of “collected articles,” which has evolved into something more like a book. I chose to publish the book as an e-book because I wanted it to be searchable and easy to update.  At the current time, the book does not contain any photos or video links, but there is a link to a page on my website where I will be adding photos and video links as I have time. Eventually, I will incorporate some of those into the book itself, but I thought it made more sense to publish the book now, instead of waiting another year for me to finish that aspect of it. I will post an announcement when an update becomes available.

My hope is that the book will be useful for new clicker trainers who are just starting, or need a few more details about how to get started successfully, as well as provide some in-depth information for people who have already done some clicker training, but are eager to dig a little deeper into it. If you are interested in learning more about the book, you can find the book on Amazon, which will let you look inside. The Table of Contents, listed below, will also give you some idea of the content.

Here’s a link to the book:

TABLE OF CONTENTS

 
INTRODUCTION

CHAPTER 1: WHAT IS CLICKER TRAINING?
An Introduction to Clicker Training for Horses
Are There Different Styles of Clicker Training?
Are We Trainers, or Teachers?
Why Do I Clicker Train?

CHAPTER 2: CLICKS, TREATS, AND WHAT HAPPENS IN BETWEEN 
Training with Food
Learning About the Clicker
The Basics of Food Delivery
Protected Contact
Mugging
Motivation
Timing
Before we Move on…

CHAPTER 3: A LITTLE BIT OF SCIENCE
Operant Conditioning
Punishment
Classical Conditioning
Reinforcement
Taking Another Look at the Click
How Neuroscience Can Help us with Training
Reinforcement and Bribery
A Little Review of Key Points

CHAPTER 4: TRAINING STRATEGIES
An Introduction to Cues
Shaping
Targeting
Capturing
Luring
Choosing A Training Strategy
A Closer Look at the Process of Shaping
A Shaping Plan to Teach Whales to Blow Bubble Rings

CHAPTER 5: TRAINING SESSIONS
Preparing for Training Sessions
Training Plans
Reinforcement Schedules and Variety

Unwanted Behavior and Errors
How to Introduce Your Horse to Clicker Training
What Behaviors Should You Train First?

CHAPTER 6: TARGETING
An Introduction to Targeting
Teaching a Horse to Touch a Hand-Held Target
Teaching a Horse to Touch a Stationary Target
Adding a New Cue to the Behavior “Touch the Target”
Teaching a Horse to go to a Target
Building Duration: Waiting at a Target
Teaching a Horse to Target Your Hand
Teaching A Horse to Follow a Target

CHAPTER 7: A RELAXED STAND
An Introduction to Teaching a Relaxed Stand
A Training Plan to Teach a Relaxed Stand
“The Grown-ups are Talking, Please Don’t Interrupt”

CHAPTER 8: HEAD LOWERING
An Introduction to Head Lowering
Basic Strategies for Teaching Head Lowering
Head Lowering: Adding Cues, Duration, and Movement
A Training Plan for Head Lowering

CHAPTER 9: MAT WORK
An Introduction to Mat Work
Basic Strategies to Teach a Horse to Put His Front Feet on a Mat
A Training Plan for Mat Work
More Advanced Mat Work

CHAPTER 10: BACKING
An Introduction to Backing
Basic Strategies to Teach Backing
Tips for Teaching a Horse to Back
A Training Plan to Teach a Horse to Back

CHAPTER 11: OTHER FIRST BEHAVIORS
Can I Touch You?
Can You Put Your Head in It?
Name Recognition and a Recall
Walking Forward
A Balanced Halt
Happy Faces: Ears Forward
Bucket Manners

CHAPTER 12: LOOKING FORWARD
Learning from Dogs: Thoughts on Science and Relationships
Find Your Troop
Making Decisions
Advances in the use of Positive Reinforcement in Animal Training
Stepping Stones

NOTES

ACKNOWLEDGEMENTS

ABOUT THE AUTHOR

ASAT 2018: Ken Ramirez on “Problem Solving.”

detective snoopyThis is the sixth in a series of posts based on my notes from the 2018 Art and Science of Animal Training Conference that was held in Irving, Texas on March 24-25, 2018.

While I try to take accurate notes, it is possible that there are errors or that some detail is lacking. If you post a comment or email me, I can try to clarify or provide some additional information. Many thanks to the speakers and organizers who allow me to share.  To learn more about the conference, you can visit the conference website.

Ken Ramirez: Problem solving: the eight causes of problem behavior

Problem solving can be a big part of a trainer’s job and is one of the main reasons he is called in as a consultant.  In this presentation, Ken shared his approach to identifying the cause (or causes) of problem behaviors.  The list he presented is one he started using when he was a young trainer, and he said “it has stood the test of time.” In all his years of training, he has not found a cause that could not belong under one of the eight possible causes that he’s going to describe.

Understanding the causes of problem behavior can aid in finding solutions to resolving challenging issues.  But, determining the cause is just one part of his normal approach to problem solving.

Problem solving as a 5 Step Process

The Basic System

  1. Identify the problem
  2. Determine the cause (hypothesis/analysis)
  3. Consider the balance of punishers/reinforcers – (motivation)
  4. Implement a plan
  5. Constant monitoring

People are usually too quick to jump to step 4 – implement a plan, but there’s no point in trying to make a plan before you have some idea what might be causing the problem.

Determine the cause:  8 causes of problem behavior

  • Environmental
  • Social
  • Psychological
  • Trainer
  • Session use
  • Regressing
  • Desensitization

Ken had videos and some great stories that accompanied each of the eight causes.  Because I don’t have that material, my notes are somewhat brief.  If you ever get the chance to see this presentation in real life, I recommend it. The stories and videos add a lot.

Environmental

  • Weather
  • Facility changes
  • Prop changes – story of how the dolphins stopped coming to their stations for the show, but instead, they stationed farther out. It turns out that he had new boots with white soles (instead of black) and that changed their behavior.
  • Public activity

Social

  • Their social interactions are important to them
  • Dominance/ submissiveness in social groups can make it challenging – every animal is stressed, not just the least dominant one
  • Competition
  • Sexual activity
  • Set up training sessions so the social interactions don’t cause more problems

Psychological

  • Boredom
  • Neurotic or aberrant behaviors
  • Stereotypic behavior
  • Frequency of these problems is small, but we tend to get called more often for them

Physical

  • Health
  • Aging – we don’t always recognize changes in our animal’s ability as they age, or from an injury
  • Sometimes something happens once or twice and we think the animal can do it, but really they were just lucky and it’s beyond their normal capabilities
  • Example of sea turtle with injury who had to have a special scale

Trainer

  • Is it me?
  • This should be the first question you ask
  • Am I working beyond my skill level?
  • Check the basics: cues, criteria, markers, reinforcers – sometimes we just get sloppy
  • Check our emotions, they impact our training, especially when we are in a high emotional state (good or bad) – story about the trainer whose boyfriend proposed during the show and how she dumped all the fish in the tank because she was so excited.
  • Attitude – story about the trainer who didn’t want to train sharks- what turned her around was presenting for a TV program and getting recognition from her colleagues

Session use

  • Planning – if you’re not prepared, the animal may lose focus
  • Number of sessions
  • Frequency of session – don’t want to do them so frequently that the animal is tired or satiated
  • Pacing – some animals work well at fast pace, others need a slower pace
  • Balance of reinforcement (story of sea otters after oil spill – they needed to collect blood samples but the otters were reluctant to be caught after a few sessions. They turned the hallway (where they did blood draws) into a fun place so that the blood draws were insignificant. They put the otters in the hall 4-5 times a day, blood draw was once a week.

Regression

  • Normal part of the learning curve
  • Ok to see a step back, or two

Desensitization

  • An ongoing process that never ends
  • You can’t desensitize them to everything

Summary

  • Pinpointing the cause gives you some things to change
  • This list is just one part of the overall system he uses
  • It’s a helpful checklist when you don’t know where to start, or want to make sure you consider everything
  • It stimulates thinking and helps clients explore ideas that they may not have considered

 

 

ASAT Conference 2018: Barbara Heidenreich on “Exotic solutions to exotic animal problems.”

BarbaraHeidenreich_wombatThis is the fifth in a series of posts based on my notes from the 2018 Art and Science of Animal Training Conference that was held in Irving, Texas on March 24-25, 2018.

While I try to take accurate notes, it is possible that there are errors or that some detail is lacking. If you post a comment or email me, I can try to clarify or provide some additional information. Many thanks to the speakers and organizers who allow me to share.  To learn more about the conference, you can visit the conference website.

 

Barbara Heidenreich: Exotic solutions to Exotic animal problems

Barbara works extensively within the zoo community, both as a consultant and as the trainer for long-term projects.  She shared some of her experiences working with zoo animals and described how the zoo community faces different challenges than trainers working with domesticated animals and pets.  Her presentation had a lot of video, which I can’t share, but I’ve included descriptions that highlight the main points.  If you want to see some training videos, she does have a large collection of videos on her YouTube channel.

What is it like to work with exotic animals?

There are many similarities but with variations depending upon the species and the environment.

  • Same principles
  • In many cases, similar challenges

Zoo consulting:

  • She needs to be able to work with many different species
  • It’s important to take advantage of modal action patterns – what is the animal “hard-wired” to do?
  • Neophobia (fear of new objects) is a common problem.
  • There can be challenges with hand reared animals – some hand reared animals have inappropriate behaviors around people.

She can’t be an expert on every species that she is asked to work with, so she presents herself as someone knowledgeable about influencing behavior. Her job is to work closely with the people who know the species well.

Zoo trainers may have a different kind of relationship with their animals -compared to trainers working with domestic animals

  • Sometimes they do have trusting relationships.
  • Sometimes it is irrelevant to the species or the behavior (sharks and rays, cobra).
  • Sometimes it is discouraged because it can adversely influence the species or the behavior.
  • Sometimes it is required that the trainer NOT interact (common with conservation projects).

Zoo trainers have to consider:

Group dynamics:

  • How to manage groups of animals (many trainers? 1 trainer per animal? 1 trainer per group?)
  • Teaching dominant animals to allow others to participate, shy ones to participate
  • Might need to teach separation both for practical reasons and to facilitate training
  • Multi species exhibits  – sometimes the group dynamics involves other species that may or may not also need to be trained.

Different environments:

  • Large exhibits where you might not see the animal (no easy opportunity to reinforce the animal)
  • Control and choice- how much can you give? In zoos, may animals are not restricted within their own exhibit space
  • Free flight – finding suitable locations, avoiding distractions, considering safety
  • Facility design challenges – many are not designed with training in mind, not set up for protective contact, often use antecedent arrangement but the facility may make this more difficult

She shared a video of teaching pigs to separate for feeding.  She had to consider how to mark and reinforce the behavior.  The pigs were in an enclosure with hotwire, so she didn’t use a click as the sound was too similar.  Pigs love tummy rubs and this can be used as a reinforcer.  The training plan had to take into account species specific behaviors, like the fact that pigs like to bite…

Video Examples:

She had a nice collection of videos that showed work with a lot of different species and with a wide variety of behavior problems.  Most of the examples were consulting work  where she was called in to solve a problem and only had a short time to resolve it.  In many cases, she uses the same tools which are:

  • targeting
  • stationing
  • training incompatible behaviors
  • avoiding triggering the unwanted behavior
  • creating positive associations with various objects
  • changes in the environment or antecedents

I’ve listed a brief description of how the behavior and how she addressed it.  While the videos made the presentation more interesting, I think it’s actually pretty interesting to read through the list and see how she had to make small changes for different species and different conditions.  There are lots of great examples of how to set up the training so the trainers and animals could be more successful.

Monkey with undesired sexual behavior: This was a monkey that was showing inappropriate sexual behavior when his keepers were interacting with him. It only happened with them (not with visitors) and did not happen immediately, so they had time to train other incompatible behaviors in the beginning of each session. Over time the behavior decreased as he learned to offer other behaviors.

  • Avoid reinforcing it
  • Avoid triggering it
  • Teach an incompatible behavior
  • Immediately engage in incompatible behaviors before he has to time to do the unwanted behavior
  • Build repertoire of acceptable behaviors

Hoof curl with Giraffe:  This example showed teaching duration to a giraffe for hoof care. The giraffe had learned to rest her foot on a hay bale (the hoof curl behavior) but would not keep it in position.  They extended the duration by adjusting the amount of reinforcement based on how long she held the position, and being consistent about not reinforcing for efforts that did not meet criteria.

  • Want her to put her foot on the bale of hay but she won’t keep it there
  • Used different reinforcement value for different levels of response, 1, 3, 5 biscuits – contingent on what she does

Leopard that won’t go to a laser target:  Laser targets are often used with the cats to move them from one location to another.  They wanted to use a laser dot to move the cat from one enclosure to another, but this cat was ignoring it.  It required a more creative shaping plan, one that didn’t require multiple repetitions, but just built the behavior one step at a time over two weeks.  It’s a great example of how you can train, even if you can’t do long sessions.  Even one repetition a day was enough to change this cat’s behavior.

  • Creative shaping plan
  • Put a large chalk circle on the wall, added some scent
  • Leopard would go in and check it out, click and treat
  • Did one repetition a day for 2 weeks
  • Move the circle to a new spot every day
  • Once the leopard had learned to go to the circle, they reduced the size
  • replace circle with laser dot

Gibbons that won’t shift with the trainer in the holding area:  These gibbons had become suspicious about going into the holding area when a trainer was present – because this usually indicated that they would be locked up.  She had to work with their current environment and slowly shape the behavior of going into the building.  The set-up is an island that is connected to the holding area by a log.

  • They do have a remote feeder on the island
  • Sound of feeder became the marker
  • Have to cross a log to get to the holding area
  • Gradually shape them to come closer to the holding area by using the remote feeder to provide reinforcement

Elephant who bites and spits out pills:  This elephant requires medication delivered orally. She will take it but if she bites into it and tastes the medication, then she spits it out.  They had to change some aspects of the pill delivery as well as teach her to swallow the pill instead of biting it.

  • They had tried various things, but if she bit the pills and tasted the medication, she would spit it out
  • Give fruit juice first – as lubricant
  • Then freeze the pills and coat in coconut oil – so they slide
  • Had to teach swallowing with a smaller object – they actually used a quarter of a peanut
  • Once she could swallow the peanut, and maybe something larger?, they gave her the pill and cued her to swallow

Elephant that eats everything:  They wanted to use the elephant in a commercial where she was waving an object around. But, this elephant liked to eat anything she could pick up, so they had to teach her to hold something without eating it.

  • Find something she won’t eat – difficult (she eats everything!) –  but they discovered that she doesn’t like eucalyptus
  • Reinforce her for giving objects back – Offer eucalyptus – reinforce her for giving it back
  • Then offer something with the eucalpyptus, – then reinforce her for giving it back
  • Eventually she will take other objects without the eucalyptus and give them back
  • They got their video of her waving with the object

Head squeezing surrogate mom:  I think this was an orangutan (or some other larger ape – can’t quite remember).  The baby was not being raised by the mom, but was allowed to spend some time with an “aunt.”  Unfortunately, the aunt was accidentally reinforced for squeezing the baby’s head.  They did make some progress with resolving this issue, but then the baby was able to go back with his mom.

  • Identify when it is likely to happen (when baby is hungry)
  • Build repertoire of more appropriate behaviors
  • High ROR (rate of reinforcement), then thin it out
  • Have some “bail out” behaviors – that they can ask the aunt or baby to do if it happens (they wanted a safe way of interrupting the behavior)
  • End sessions at any of the pre-cursors

Giraffe that won’t stay in the chute:  This giraffe would not stay in the chute for handling. First they had to reinforce her for being in the chute and build up a positive association with that location.  Then they had to teach it to accept touch.

  • Teach approximations to following a target
  • High ROR
  • Build up to R+ for staying in the chute
  • Door open when they start to touch – so she can leave if he wants
  • The word “touch” is used to tell the giraffe when she will be touched

Lemur that jumps on keeper’s shoulders:  The lemurs are in an indoor enclosure with branches to climb on. One of the lemurs likes to jump on the keeper’s shoulders. She had them teach an alternative behavior.

  • Teach target and station on a branch
  • High ROR for targeting and stationing
  • Make it harder to do the undesired behavior, easier to do the desired one
  • Reduce the reinforcement value of the undesired behavior
  • Practice by letting him wander a little and then come back – so he learns to return to the station.  Watching where he goes when he leaves the station is useful information about what is reinforcing.

Dive Bombing conures:  These birds were in a school and were dive bombing people who entered their enclosure.  They did come up with a plan to teach alternative behaviors, but she presented it as an example of being realistic about whether a training or a management solution was going to work better.

  • DRI (differential reinforcement of incompatible behavior) – stationing
  • High ROR – then thin it out
  • But realistically, the easiest thing was to adjust the environment so that the birds were not in the same space with the people
  • Be realistic about your options and resources

Fear of New Objects:   A giraffe was afraid of new objects and wouldn’t touch a target. They were working on a high platform at head level (for the giraffe).  They started by feeding the giraffe in the presence of the target and waiting until he chose to interact with it.

  • Giraffe doesn’t like target
  • Present target, feed as long as giraffe stays within reach
  • Eventually the giraffe gets closer and closer to the target and touches it
  • It’s about waiting for the animal to choose to interact with the object
  • Once he is choosing to touch the target, then they can mark and reinforce for a targeting behavior (switching over to an operant response)

Fear Response to Touch:  This giraffe was being taught to accept touch.  They had him touching a target while standing behind a barrier (leaning over a doorway).  One trainer was holding the target and reinforcing the giraffe. Another trainer was positioned so she could touch the giraffe on the neck as he reached forward to touch the target.

  • Systematic desensitization (they started with little hand movement toward the giraffe and very light contact)
  • Teach the animal to initiate contact – they wanted the giraffe to learn that he could choose when to start the next repetition and control when the person touched him
  • mark and reinforce for accepting contact (they started by reinforcing targeting).

Undesired vocalizations that had been reinforced: This bear had been accidentally reinforced for making loud vocalizations.  Over time it had gotten worse and worse because trainers had not waited out extinction bursts.  Barbara had a video showing how extreme the extinction burst had become.

  • Avoid reinforcing undesired behavior
  • R+ for silence
  • She was reinforcing for longer periods of quiet and pushed too far so the bear went into an extinction burst
  • This was a somewhat distressing video – the bear sounded horrible – but it did show how extinction bursts work, and how bear did seem to be able to turn the vocalizing on and off.

Bear outside grabbing people: This was a short clip that showed a young bear who had been allowed to interact with people in ways that were no longer safe, now that the bear was getting bigger.  Barbara had them teach the bear to station on a log instead of grabbing at or trying to climb people.

  • Young bear used to interacting with people
  • Teach stationing

Rough play with fox:  This fox had been rescued and allowed to play rough with people. He had learned to bite at hands when on a person’s lap. The zoo wanted to be able to have people pat him, without the fox getting excited and biting at people. The first step was to teach alternative behaviors that were appropriate when he was handled by people, and then introduce patting.

  • Avoid triggers
  • Teach other desirable behaviors
  • Gradually raise criteria

Spitting orangutan:  If you thought it might be fun to work in a zoo, well…maybe not. This was an orangutan that had developed an annoying habit. I think Barbara said that she did this to avoid interacting with the keeper, but I’m not sure.

  • Would slurp up urine and spit at keepers
  • Avoid triggering it

Bouncing leopard:  This leopard’s cage was at the end of a hallway and the keepers had to go down the hallway to access something on the wall opposite the leopard’s cage.  When they did that, the leopard would start bouncing off the walls – even though they were not intending to interact with him.  Ignoring the leopard made things worse so they had to teach the leopard what to do when someone was in the hallway.

  • Would bounce off the walls when keepers were near
  • Cue for appropriate behaviors
  • Keep engaged
  • Be quick with reinforcers and bridging

Summary: 

A lot of the examples did use the same basic strategies, but it was interesting to see how Barbara had to adjust based on the animal’s individual needs, the environment and whether she was addressing a problem behavior or teaching something new.

In some cases, the behavior would happen as soon as the trainer approached so it was a matter of focusing on antecedents. In others, an unwanted behavior had accidentally been reinforced, so it was a matter of teaching alternative behaviors and building a strong reinforcement history for them.  In others, it was about focusing on teaching new behaviors. There was a strong emphasis on teaching the animal to participate and  reinforcing for desirable behavior.

I think we can learn a lot from looking at how zoo animals are trained and while many of her “problem” behaviors are not the same ones we tend to encounter with horses, there are good ideas that we can take from all of them. Targeting and stationing are just as useful for horses as they are for zoo animals.  One that sticks in my head is teaching the leopard to go to the laser dot by doing a little each day. I’ve done this with some horse behaviors and found incorporating a minute or two of training as part of routine handling can make a big difference.

Barbara has two websites: https://barbarasffat.com and www.goodbirdinc.com  She is working on an on on-line animal training course at www.animaltrainingfundamentals.com

ASAT Conference 2018: Emily Larlham on ” Using multiple markers to prevent mixed messages.”

emily

This is the fourth in a series of posts based on my notes from the 2018 Art and Science of Animal Training Conference that was held in Irving, Texas on March 24-25, 2018.

While I try to take accurate notes, it is possible that there are errors or that some detail is lacking. If you post a comment or email me, I can try to clarify or provide some additional information. Many thanks to the speakers and organizers who allow me to share.  To learn more about the conference, you can visit the conference website.

Emily Larlham gave a 20 minute presentation on using different marker signals to indicate different types of food delivery. She had quite a lot of video so this report is quite short, but if you go to her YouTube channel Kikopup, you can see examples of different markers being used.

Why use multiple markers?

People learn to distinguish between multiple markers – we learn to recognize which markers signal different types of reinforcement. When you hear the microwave ding, you know to go look for your food in the microwave.  When the UPS guy arrives, you know to go to the door. Wouldn’t it be confusing if they were both the same sound?

And wouldn’t it be helpful if the choice of marker gave your dog information about how the reinforcer would be delivered?

She likes to use different markers to help create different emotional responses and outcomes.  The marker can tell the dog whether the food  (or reinforcer) is coming to him, or if he should go get it.

Different markers can create different emotional responses

Think of dogs and doorbells.  Does you dog respond to different sounds in different ways? These are conditioned responses and we can use them to our advantage. The biggest advantage is being able to use markers that promote calm vs. markers that are associated with arousal.

Calm markers are useful for:

  • encouraging relaxation or waiting in position
  • Group situations
  • Husbandry behaviors
  • Long duration behaviors

Arousing markers are useful for:

  • Behaviors that requires strength
  • building enthusiasm, speed

Here’s a video describing using a calm marker: https://www.youtube.com/watch?v=wHRhiwqrYRg

Choosing markers

People tend to think some marker signals are more suited to different emotional responses, but it’s not about the marker, it’s about how you trained it.  The click seems like it would generate an excited state, but she had a video example of a Doberman who did really well with click as his calm marker.

If you already use several markers, you can pay attention to how your dog responds to your markers and choose the appropriate one for each behavior.

With her dogs, she has markers that mean different things:

  • Stay – wait, reinforcement is coming
  • Continue – continue moving, reinforcement will appear in front of you
  • Go – release to toy or food in the environment
  • Come – release to trainer for toy or food
  • Do – release to do a specific behavior – go say hi, go play, go sniff

Video examples 

She had several videos showing her dogs using different markers.  I didn’t write down descriptions of all of them, but here are two that showed the effectiveness of different markers.

  • She showed taking her dogs to the park where they get to run off leash. In the first example, she used an “arousing release” and as soon as the dogs were released, they zoomed off out of view.  Then she showed using a calm release – “go sniff” and they stayed with her, even after she released them.
  • Teaching the terrier a behavior where she wanted him to stay in position  and wait for the reinforcement to come to him. She used “Good” which means stay -reinforcement is coming.  The terrier stayed in position during food delivery which made it easier to continue the behavior because she doesn’t have to restart after each “click and treat.”

Teach your markers separately, before you use them

You should train your markers and methods for reinforcement delivery before you start using them in training.  You don’t want the dog to get frustrated if he doesn’t understand how he will be reinforced.  She shared the B. F. Skinner quote about the importance of teaching the rat how to get food, before any training is done.

Calm markers are ideal for teaching duration

A calm marker makes it easier for the dog to stay in position and minimizes frustration. She trains duration behaviors with a calm marker and ends the behavior with a release cue.

Most long duration behaviors have a release cue, sometimes the trainer doesn’t know what it is, and then when they add a new one, it creates confusion because the dog is using the one it was using before.

She had a video showing her dog standing on two platforms (front feet on one, back feet on another).  She has marking the behavior  with her “stay” marker and feeding the dog with her head in the up position. In this case, the dog chose to keep her head up while eating, but it’s ok to allow the dog to drop her head to eat, so she can enjoy her reinforcement.

Her point was that staying in position doesn’t mean the dog can’t move at all. It just means the dog should stay in a position so that she can re-start the behavior again immediately after the reinforcer is delivered.

Emily Larlham is based in San Diego, Ca. Her website http://www.dogmantics.com and her YouTube channel have tons of free material if you want to learn more about using clicker training.