After missing Clicker Expo 2012, I was very excited that Clicker Expo 2013 was in Stamford, Connecticut, a measly 3+ hours from my home. We drove up on Thursday in light snow and Friday morning the hotel had a few inches of snow on the ground. I think it’s the first time I went to Clicker Expo in the snow so this was a new experience and we worried about there being issues because of the weather. But it seems like everyone made it (eventually) and the Clicker Expo staff did a great job dealing with last minute changes due to the weather. This year’s Expo had the most horse people I have seen yet and it was great to catch up with old friends and make new ones. I am excited that more horse people are going and hope this trend will continue.
This year I wrote separate reports for each lecture or lab I attended. There was so much information and some of the topics are getting quite complex so I thought this was the easiest way to do it. You can read the individual reports as well as my summary by going directly to the one of interest or just starting at the top and reading down.
- Ken Ramirez: Missteps, Myths and Mantras
- Dr. Jesús Rosales-Ruiz: Operant vs. Respondent
- Dr. Jesús Rosales-Ruiz: Cues and Context
- Kay Laurence: Connected Walking
- Kay Laurence: World of Targeting
- Dr. Jesús Rosales-Ruiz: Negative Reinforcement
- Eva Bertilsson and Emilie Johnson Vegh: This class is a Blast: Five Steps to phenomenal teaching success
- Panel discussion with Karen, Alex, Jesus, Ken, Julie, Kathy, Kay and Michele
- Dr. Susan Friedman: What the word PARROT reveals about teaching dogs
Ken Ramirez: Missteps, Myths and Mantras
The first talk I attended was Ken Ramirez’s talk titled “Missteps, Myths, and Mantras; Keeping your Training in Track.” The talk covered 26 different ways that trainers can hinder their own progress by falling into cultural traps, taking statements too literally, or being sloppy or in a hurry with their training. The underlying theme behind all of them was that trainers need to think about how they use information they get (even if it’s from a reliable source) and monitor their own training for some common mistakes. His point was that if you are aware of these things, you will be on the lookout for them.
Here is his list. Remember some of these are beliefs, some are cultural beliefs, some are just good information taken too literally, some are things we get sloppy about doing, or do that we shouldn’t.
Myths and Mantras (Sayings): There is often a factual component to these, but they should not be taken literally or as gospel. Usually they are descriptive information that is intended to simplify or provide general guidelines, so they are helpful if taken in context. He does also point out that some are just not helpful at all, like breed prejudice.
Dogs are wolves: Dogs are not wolves. We can learn about dog behavior by studying wolves, but this is not necessarily transferrable to how we train. This myth is usually used by people promoting dominance based training.
Positive Reinforcement means there are no rules: This is a misconception about how positive reinforcement trainers work, and is usually held by people who have not studied or are opposed to positive reinforcement training. They think that positive reinforcement trainers do not have rules and that the animals are allowed to do whatever they want. In truth, having rules makes positive reinforcement more effective. He also pointed out that discipline does not equal punishment. Discipline has multiple meanings, one being that you have a structure to the way you work.
Reinforce for position: This is part of a commonly heard phrase which is “click for behavior, reinforce for position.” There is nothing wrong with it, but it is sometimes taken too literally. There are lots of valid ways to deliver reinforcement and this is just one of many. Don’t let yourself get locked into always doing something one way.
Click ends behavior: The reason he included this in the list is that while it is often true that click ends the behavior you are training, it is not true that click ends all behavior, or that behavior between the click and reinforcement it not important. Trainers need to pay attention to all behavior.
NILIF (nothing in life is free). This is a protocol that is sometimes recommended for dealing with difficult animals. In NILIF, the animal has to work for everything so any kind of reinforcement is contingent upon performing a behavior first. It’s not a bad idea to ask your animal to do simple things before getting reinforcement, especially if you are using life rewards, but it can be taken to extremes and then it no longer puts the needs of the animal first. If you don’t know anything about NILIF, you can google it. An excellent book that explores NILIF and its effect on dogs and owners if Kathy Sdao’s “Plenty in Life is Free.”
Breed prejudice: Making assumptions about what an animal can and cannot do (or its temperament) based on breed alone is not a good idea. Yes, there are differences between breeds in terms of physical ability and genetically hardwired preferences, but as soon as we say “this dog can’t do that,” based on breed, we are limiting ourselves. He also pointed out that intelligence is not necessarily a measure of trainability so it’s important not to get caught up in comparing intelligence between breeds, which is still being studied.
Leader of the Pack: This relates back to the belief that dogs are wolves and live in a dominance based society (which incidentally is not true of wolves either, but that’s another discussion). His personal philosophy is that he doesn’t want his animals to see him as a conspecific, and also if you buy into the idea of there being a leader, then you are buying into the idea that there is competition to be the leader. Is this what you want? Kathy Sdao’s book “Plenty in Life is Free” also deals with the idea of the ‘leader vs. the feeder.”
Missteps (Common Trainer Mistakes):
Looking for the quick fix: People often take short cuts because it can take time to find the root cause of a problem and come up with a training plan to address it. Sometimes they don’t have time. Sometimes they don’t realize that the way which seems to offer a fast, easy resolution is not going to be a long term solution and may have side effects. Quick fixes often involve punishment and being in a hurry is one of the reasons that people resort to punishment.
Saying “the animal knows:” This is an assumption the trainer makes based on past performance or expectation. It often doesn’t take into account changes in criteria that might prevent an animal from responding correctly. Some trainers will interpret an incorrect response as the animal deliberately misbehaving or not trying hard. If you find yourself thinking that an animal knows the behavior, but is choosing not to do it, then you need to go back to the basics and find where the animal can be successful. Often we say this to make ourselves feel better as it shifts the focus from the trainer to the animal.
Forgetting the power of the click: This refers to trainers who use a clicker for initial training but then change to another less precise marker signal or don’t use one at all. There are times when you don’t need the precision of the click, but there are other times when it is an important part of maintaining a precise behavior. Ken finds that medical procedures are usually maintained better with the clarity that comes from clicking and reinforcing.
Forgetting desensitization is an ongoing process: Desensitization is never complete and no animal can ever be desensitized to everything. Training never ends and we need to constantly evaluate our animal’s comfort level with new things.
Getting greedy: There are different ways to get greedy. The trainer can just keep working for too long or do too many repetitions. They can also expect too much from an animal, either looking for more than the animal has been trained to do, or having unrealistic criteria. The best way to avoid getting greedy is to have a good training plan.
Learning never ends: Sometimes trainers forget that learning is always taking place. Animals learn 24 hours a day, not just when you have a clicker and treats. Be aware of passive training (non-formal training) and be aware that many problem behaviors are learned outside of formal training sessions.
It’s in the details: We tend to look at the big picture of behavior and don’t always notice all the details that are important. Experienced trainers know that behaviors improve when we focus on the small things. Things such as calmness vs. tension, focus, and weight distribution can be important details.
Using too many trainers for one task: Multiple trainers do not set an animal up for success. Even the best trainers will view behavior and progress differently. Usually multiple trainers are working with an animal to try and speed up training or for daily behaviors that may have to be done by different people. There are times when you have to do this, but it is essential to recognize it is not ideal and communicate well.
Lack of communication: This is just the follow-up to the previous misstep. If you use multiple trainers, be sure to clearly identify goals, priorities, and training plans so that you can be consistent.
Lack of Consistency: This can happen if you have multiple trainers or if you don’t keep good records so you are inconsistent. Sometimes it is just a simple case of not paying attention to details. Other times he has seen one trainer deliberately sabotage another’s efforts. This tends to happen with family members or room-mates who disagree over how the training should be done. It can also happen accidentally as cues can morph over time and multiple people using the same cues can end up with significantly different cues as they change over time.
Predictable Patterns: We are creatures of habit and we can develop habits during training that end up affecting our training without being aware of it. This happens a lot with cueing and with body language. It is easy to fall into bad habits such as moving your hand toward the treat pouch or pocket without being aware of it. We also can create keep going signals by talking during training and these can become a crutch.
Using behavior before it’s complete: If a behavior is not complete, don’t use it. For simple behaviors, using them too soon creates sloppy behavior. For difficult behaviors, using them too soon can break down trust and make the behavior harder to complete. If all approximations are not done, don’t use the behavior.
Giant Steps: Trainers need to be careful about taking approximations that are too large. Sometimes this happens when trainers are in a hurry. Sometimes it happens when trainers think the animal doesn’t need the small steps or they overestimate their own skill. He made the distinction between moving quickly (doing all steps, but maintaining progress) and skipping steps.
“My Animal Loves”: Don’t make assumptions about reinforcement. Make sure your reinforcers are effective and remember that they are context specific. Animals develop expectations about the kind of reinforcement that will be given for different activities. Often people make assumptions about what an animal likes, but don’t pay attention to whether or not it is actually reinforcing the behavior. If the behavior is not increasing, then you need to check to see if your reinforcers are effective.
Oops: Inadvertently conditioning a NRM (no reward marker): If we don’t control our reactions when mistakes happen, we can accidentally teach our animals a cue that means “no.” This is a conditioned punisher and can create frustration and problems. There is considerable debate about whether or not to use NRMs and that wasn’t really his point. His point was that if you create a NRM and you are not aware of it, that can be a big problem.
Training is easy: People sometimes make assumptions about what can be trained and the level of difficulty. There are limits to what a beginner or novice can accomplish and you need to be honest about your skill level. You also need to recognize what your animal is ready to do and seek advice if you get in over your head.
Not my fault! When things don’t go as planned, there is the temptation to blame the animal, the environment, or some other external factor. Ken included “labeling” animals in this category. If you hear someone say “It’s not my fault, he hates men, uniforms, etc…” they are not accepting responsibility for their training. While this may make you feel better at the time, it’s not going to help you in the long run, so avoid “it’s not my fault” excuses.
Jumping to conclusions: Be careful about making assumptions about other trainers when watching them work with their animals. You don’t know their history or reasons for what they are doing. I loved that Ken included this because it does seem like people love to comment on other people’s training without knowing the whole story. Often you can learn something new by asking someone why they are doing something instead of assuming they are doing it wrong.
Misunderstanding the value of relationships: Relationships are important and your relationship with your animal can have a significant impact of the success of your training. Ken sees trainers who both over-estimate and under-estimate the value of relationships. A common question that comes up related to this is whether or not the trainer should be involved in an aversive event such as a medical procedure. He has found that in many cases, the presence of a trusted person can decrease the stress to the animal. And yes, it depends…
Notes from Katie:
Ken started this talk by explaining why he wanted to focus on “mistakes.’ Positive reinforcement trainers emphasize what to do, both in their training and their teaching. This is great and we all need to focus on what works, but looking at things that can go wrong is also valuable. The idea here is to be forewarned about things that can go wrong (so you can avoid them), and provide a little perspective on how to interpret many of the stock phrases that are thrown around by animal trainers.
I think Ken’s list clearly shows why animal training is so challenging to do well. There are a lot of contrasting pairs on his list. You don’t want to be too predictable, but you do want to be consistent. You don’t to take something too literally, but you do need to recognize that there is an element of truth to it. As trainers, there are a lot of things we need to observe and monitor. Finding the right balance is important and paying attention to details can make a big difference.
This list was geared a little bit toward dog trainers, but I think most of them apply to any kind of animal training. Even the sayings that are “dog specific” have parallels in the horse world. For example:
Dogs are wolves – > with horses, there are a lot of people who put emphasis on acting toward a horse as if you were another horse in the herd.
Leader of the pack – > same thing with there being an emphasis on the importance of being “dominant” or “getting respect.”
Breed prejudice -> I see this with some trainers who expect certain breeds to act in certain ways and miss the horse’s individual needs or abilities.
I would add a few other myths or sayings that horse people encounter:
- Hand feeding makes horses bite
- A horse moving into your space is disrespectful
- Anyone want to add anything else?
Dr. Jesús Rosales-Ruiz: Operant vs. Respondent
The second talk I attended was by Dr. Jesús Rosales-Ruiz and was on Respondent (classical) vs. Operant conditioning. I have read and heard lectures on the difference between respondent and operant conditioning and assumed this was a clean distinction, although I knew in many cases both were happening at once. Jesús explained that people (behavior analysts) have been arguing for years about the difference, ever since Skinner made the distinction.
This was the first of three talks that Jesús gave and while they were on different subjects, there was an underlying similarity in that in each one he talked about some distinction that was made in the lab (this was the theory part) and then looked at what happened in application. After listening to him, I felt like behavior analysts have created these distinctions such as operant vs. respondent conditioning because it is easier to study them in the lab, but that in real life, things are messier.
He explained that part of the problem with having a distinction is that trainers approach problems differently when they think the problem can be solved by respondent vs. operant conditioning. It also turns out that a lot of things that were assumed to be controlled by respondent conditioning can actually be controlled by operant conditioning. He had some really good examples of these which included blood pressure, intestinal contractions and some cell functions. Most of those are controlled by negative reinforcement and can be manipulated by using an operant procedure. For the trainer out there working with their animal, they need to be aware that different things can be happening, and they should not get so fixated on one interpretation that they ignore the rest. Good advice for many things!
Jesús started with some history. B.F. Skinner first made the distinction between operant and respondent conditioning. In operant conditioning, the animal’s behavior is determined by the consequences and there is a more active relationship between the animal and its environment. When we clicker train, we are using operant conditioning because the animal learns it needs to do something to get clicked. In respondent (also called classical) conditioning, the animal’s role is more passive. Respondent conditioning is about making associations. The animal is not necessarily actively acting on the environment to earn reinforcement. An example of respondent conditioning is conditioning the clicker. If you condition the clicker by just clicking and feeding, regardless of what the animal is doing, you are using respondent conditioning. The animal learns that the click predicts food, but it does not know yet what it needs to do to get a click. This is why it is “passive.” Counter-conditioning where you associate something good with something that has previously been a trigger for fear or aggression is also respondent conditioning.
Changing the behavior of aggressive or fearful animals has traditionally been the domain of respondent conditioning as respondent conditioning is thought to affect an animal’s emotional response to the antecedent (trigger) and this change in emotion can then lead to changes in behavior. A scared animal will have a lot of unwanted behaviors that will go away on their own once the animal is no longer scared. The idea is that behavior is determined by the animal’s emotional state. Change the animal’s emotional state and the behavior will change.
When Jesús started working with aggressive dogs, he expected to use respondent training techniques to change their behavior. But at some point he realized that the dogs were acting more as if their aggression was an operant behavior. They were aggressive because it worked for them, meaning aggressive behaviors were being reinforced by something they wanted. In this case, the dogs were using aggressive behaviors to create distance. From this observation, he and Kellie Snider came up with the CAT protocol which is based on the idea that aggressive behavior is maintained by negative reinforcement. Understanding that a behavior is maintained by negative reinforcement changes your strategy for changing it. In this case, they found they could use negative reinforcement (removal of aversive by adding distance) to reinforce behaviors they did like (calm behaviors) and the change in behavior led to a change in the dog’s emotional state.
This experience led him to re-evaluate his own thoughts about respondent vs. operant conditioning and this was the subject of this talk. He believes that most respondent conditioning is really operant conditioning and we just don’t look close enough to see the consequences that are driving the behavior. He took another look at desensitization and counter-conditioning with more attention to what is really happening and looked at examples of respondent procedures to see if there was some element of operant conditioning that had been overlooked.
After explaining how he came to look more carefully at the distinction between respondent and operant conditioning, he spent a little time on the historical perspective. I am including a bit of this here because I think it’s important for us to have some idea that when we read about these terms in books and articles, we need to keep in mind that this has been the subject of much discussion. Part of his talk was a look at some of the past discussion on the subject with views from Pavlov, Miller and Kornoski, Skinner, Thorndike, Rescorla and Wagner, and Donohoe.
As I said, the distinction between respondent and operant started with B.F. Skinner. Prior to that, there was no distinction. Ivan Pavlov and Edward Thorndike both studied behavior and how it changed as a result of the environment, but they did not believe there were two types of conditioning. I thought this was pretty funny considering we now think of Pavlov’s work whenever we need an example of respondent conditioning. In Science, it is easy to end up with dual views as scientists often get caught up in seeing behavior from one perspective. Jesús showed some charts of behaviors events and environmental events (Donohoe and Vegas) which showed how you can interpret them as demonstrating respondent conditioning or operant conditioning depending upon where you put your focus. You can make arguments for either respondent or operant conditioning happening depending upon what behaviors and interactions you observe as well as where you enter the behavior stream. He suggested that rather than have a dual view of behavior, it would be better (and more practical) to have a “Unified Theory of Reinforcement.”
He led us through some of the developments that started to erode the distinction between respondent and operant conditioning. Originally, the two types of responses were defined as follows. Respondent conditioning was autonomic, involuntary, elicited and did not change the environment. Operant conditioning was skeletal, voluntary, emitted (or permitted) and did change the environment. Skinner did not believe that operant conditioning could control things like blushing or other “automatic” body functions. But in 1957, scientists showed that gastric skin releases, blood pressure and intestinal contractions were all controlled by their consequences, which made them operant, not respondent. Most of these are controlled by negative reinforcement and he had some really interesting examples of how the consequences could significantly change what happens. He also pointed out, and this goes back to his beginning statement, that if you don’t look for the consequences for a behavior, you can miss what is controlling the behavior and this can make it impossible to change it.
Ok, so this is all fun and interesting (at least to me), but what does it have to do with training? To answer this, Jesús shared some charts from an article about the two types of conditioning (Rescorla and Wagner – the article is titled “Pavlovian Conditioning: It’s not what you think it is”). According to Rescorla, both respondent and operant conditioning require a contingency and the spacing and order of events are important. In both respondent and operant conditioning, the response has to occur first in order for conditioning to happen. And it must be predictable so that both the antecedent and the consequence can be associated with the behavior. In short, both operant and respondent conditioning require many of the same things so when we train, we need to keep both in mind.
He used the example of transferring cues. Once a behavior is happening reliably, most trainers like to put it on cue. Adding a cue can be done by inserting the new cue before the old cue in the behavior stream. The new cue becomes a predictor for the old cue and the animal will eventually anticipate and offer the behavior before you use the old cue. This is usually explained as happening through respondent conditioning. But if we look at the whole behavior stream, then we see that the consequence is just as important. If you add a new cue but don’t reinforce the behavior, the animal will not learn the new cue. Writing this I am thinking that this is pretty obvious and why would anyone expect to add a new cue without reinforcing the behavior. But that’s not the point. The point is that we talk about adding new cues as a respondent process, and it’s not. It requires both respondent and operant conditioning in order to be successful.
Often the most challenging part of training is getting the behavior started. He has some video that showed how food can be used to change both the emotional state of the animal, and get some changes in movement. As clicker trainers we typically focus on adding reinforcement (often food) after the behavior has occurred, but perhaps we need to be more flexible about this. He has some video of Kay Laurence who starts a lot of behaviors with food and then switches to food as reinforcement once the animal has “gone operant.”
This means that depending upon where you start in the process, you can be starting with respondent or operant conditioning. In order to change behavior, you have to get some variation and you can approach it from the operant or respondent angle. Once you have some variation, and reinforcement is happening, then the animal looks for information from its environment. The animal will be looking for a connection between its behavior, the consequences, and the antecedent.
This was really the take home message of the talk. It was about looking at all parts of the conditioning process to see where you could insert a change that would take you in the direction you want to go. People tend to get hung up on the importance of that first step. If you start with food (not attached to a behavior), then it is perceived that you are using respondent conditioning whereas if you wait for a behavior and then add food (contingent on the behavior), then it is operant conditioning. But once you are past the first step, both are happening because the animal is looking for information from its environment to tell it how to get more of what it wants. It is better to think of both types of conditioning happening at once, without a clear separation as that is more close to the reality of what is happening.
He did talk a little bit about the best order in which to present the US (unconditioned stimulus) , CS (conditioned stimulus) and UR (unconditioned response) to get optimal conditioning. CS- US- UR works best, but you get conditioning in all three. You can block conditioning which he talked about a little, but my notes on that are pretty brief. The article he referenced was Kamin, 1970 if you want to look it up. What I remember is that it was possible to prevent an association from being made depending upon the timing. We do see this with training. If I have a cue to ask a horse to back which is a hand signal and I want to add a verbal cue, I have to add the verbal cue before the hand signal. If I do them both at once, the hand signal will “block” the learning of the verbal cue because they are happening at the same time.
He had some video of trainers using counter-conditioning and desensitization with dogs and if you looked carefully, you could see that specific behaviors were being selected out, even though the trainer thought they were working on the emotional state of the dog. It takes a lot of work to divide emotion from behavior and in most cases we don’t want to. He actually said you can’t divide emotion from behavior, but I’m not so sure. I always remember Kathy Sdao’s example of capturing ET’s bell sound (ET was an aggressive walrus). The sound was one he made when he was mad and they reinforced it and put it on cue. Once it was on cue, he would still make the sound, but it no longer meant “I’m going to kill you.”
I am still thinking about all this, and will have to see if it actually affects what I do. As trainers, we need to figure out how to get a behavior started, reinforce it so it occurs again, and put it under appropriate stimulus control.. If we approach most problems with the operant conditioning mindset, we will be providing reinforcement which is targeted toward increasing a specific behavior, and also contributes positively to the emotional state of the animal. Ken Ramirez told Jesús that most of the training (both training new behaviors and decreasing unwanted behaviors) at the Shedd Aquarium is done with operant conditioning. He added a section to his book on respondent conditioning after someone else pointed out that he had not included it. I personally have found that once an animal really understands how clicker training works, it is really hard to use respondent conditioning with food (or other reinforcement) without them attaching it to some behavior because that is essentially what we have taught them to do.
Bob Bailey says that when you are training “Pavlov is always on your shoulder.” Perhaps to balance this out, when you are using respondent conditioning, you should probably tell yourself that “Skinner is always on your shoulder” too…
Dr. Jesús Rosales-Ruiz: Cues and Context
Saturday morning I attended Jesús Rosales-Ruiz’s lecture on Cues and Context. A truly fascinating topic! This lecture was a look at what cues are, how they evolve and the implications for training. When I was first introduced to the idea of cues (as defined by most clicker trainers), cues were something the handler intentionally used to trigger a specific behavior. It sounds neat and clean. But pretty early on I realized that what I thought was the cue might not be what the horse was using for the cue, and that things could actually be pretty complicated. I have pretty much been fascinated by cues ever since so this lecture was right up my alley.
This lecture had a little bit of overlap with the respondent vs. operant lecture, or maybe it picked up where the first lecture left off. In any case, Jesús started by talking about how there is always something in the environment that triggers behavior. Behavior doesn’t just come out of nowhere. As trainers we look at individual behaviors or groups of behaviors, but in real life, behavior happens as a constant stream. When we “add cues,” we are changing something about the environment that makes certain behaviors more likely. Cues can “evolve” as part of the shaping process if we change the environment before or during the shaping of the behavior. Or they can be added after a behavior has been trained, if we want to add a “new cue” to an existing behavior.
One way to understand about cues a little better is to look at how the environment affects the reliability of cues. When we train any behavior, there is a set of constant stimuli, meaning the factors that remain constant during training (sound, light, location of things in the environment, etc..). If we train in the same location and under the same conditions, the constant stimuli can become part of the cue for a particular behavior. We often call these “context cues” meaning the animal guesses which behavior will pay based on a set of environmental variables. An animal can become dependent upon the presence of the constant stimuli and the correct response to the cue will be contingent upon maintaining this environment.
This is why part of the training process includes “proofing” your cues, which means asking the animal to do cued behaviors under lots of different conditions. Adding distractions and training for generalization are part of the same process. It is interesting to play around with this as you will quickly realize that things we consider irrelevant might end up being crucial to the animal responding to the cue. This works both ways. You can change the constant stimuli by taking things away or you can add things to the constant stimuli. In training, most animal trainers want to reduce the cue to one individual stimulus, but in reality this is really hard to do. In all likelihood, there are always context cues and unless you specifically train to eliminate them, some will be attached to your “cue.” I don’t think this is necessarily a bad thing. They can help the animal understand similar cues in different situations.
Jesús had some video of Kay shaping a behavior and then asking for the same behavior while holding a broom. The dogs were clearly confused by this change in her body posture and the addition of another piece of equipment. The audience got a bit caught up in the idea that the broom was aversive, but I think Jesús’s point was that any change can have an affect and you have to realize these things matter when you plan your training.
He spent some time talking about the difference between “emitted” and “permitted” behavior. Behavior analysts refer to behavior as being “emitted,” meaning that there is no antecedent control and it just spontaneously happens. Skinner made the distinction between behavior being “emitted” vs. “elicited.” The distinction he cared about was the idea that consequences could make an animal more likely to “emit” a behavior. You didn’t have to do something (“elicit”) to get the behavior. Jesús said he is now skeptical that behavior is ever just “emitted.” Something has to happen to make you change your behavior, otherwise why would you change? Having read some of Kay’s articles recently, I would say this is a case where the consequence becomes the antecedent for the next behavior.
Even the word “emitted” might not be accurate. Donohoe suggested that a better word would be “permitted,” meaning the behavior is permitted by the environment. The current stimulus situation determines what is possible. It is actually pretty easy to see how this applies to training. We use this all the time in training when we set up the environment to limit the animal’s options or make a specific behavior more likely. At chicken camp, they put the chickens up on tables so there are fewer behaviors they can do. Michele Pouliot uses platforms in her dog training to limit movement and make certain positions more likely.
The other idea that comes out of the distinction between “emitted” and “permitted” is that it changes the way the trainer works. If you believe behavior is emitted, then you are going to wait until something happens. If you believe behavior is permitted, then you are going to set up the environment so that behavior is more likely to happen. I thought this was a pretty interesting distinction and is consistent with what I see happening with clicker training as the training becomes more sophisticated. When I first started training, there was a lot of emphasis on just waiting for the animal to do something and the trainer was very passive. Now I see much more emphasis on setting up the environment to make the target behavior more likely. The includes adding antecedents (prompting, luring) which are faded after a few repetitions, but considered useful to get the initial behavior started.
The bottom line here is that it is best to assume that behavior is always under some kind of control by the environment. As a trainer, you need to ask yourself “what environment would permit the behavior I want?” This works for training new behaviors and for changing old behaviors. If you want to change an unwanted behavior, you need to find the antecedent. If it’s something you can control, then removing it will change the environment so the unwanted behavior is not permitted. If you want to get a new behavior, you need to set up the correct environment to permit new behaviors. He had a few examples including Kay demonstrating how to teach a tail wag by having the dog lie on the floor where the tail’s interaction with the floor becomes important feedback, and Alex’s use of the “slouch” as a cue for the horse to offer behavior in a micro-shaping session. He had a line here which I loved. He said “this is not your mother’s differential reinforcement.” Everything is becoming more sophisticated.
He had some other interesting video clips showing horses being wormed and teaching a dog left and right. In the worming case, they were looking at what antecedents were cueing evasive behavior on the part of the horse. It turned out that there had to be number of conditions (presence of bucket, lead rope, handler’s movement, etc…) and the actual wormer was not one of them. Worming with an “invisible” tube created the same problem. They had to look quite far back in chain (looking for antecedents) in order to find out what triggered the first evasive response. I see this with horses that are reactive to shots. They are often reactive to position and body language, more than a syringe in your hand. With the dog, he showed how the trainer inadvertently created a cue which was going to make her long term training project harder. The take home message was that problems with cue control are often about consequence control.
Another important point about cues is how understanding how to add a cue. A lot of what he had discussed before this was about a new way of looking at cues, how cues evolve or are created as part of the training process, what can interfere with correct responses to cues, and how cues can be accidentally created. Now he talked about the specific process of adding a cue to an existing behavior. This goes back a little bit to the talk on respondent vs. operant conditioning because cues are taught by association. The animal learns that doing a behavior under certain conditions (presence of the cue) means reinforcement is likely.
In order for an animal to learn a new cue, the cue must become associated with the behavior. In a finished behavior, we cue the animal which responds correctly and is then reinforced. There are different methods for adding the cue. Some trainers don’t add the cue until they can predict when the animal is going to do the behavior. They can use the “new cue” “old cue” method or just add the cue once the animal is offering the behavior in a predictable pattern. Other trainers will add the cue while the animal is doing the behavior and slowly change the timing so that the cue precedes the behavior.
In both cases, the cue must be added in such a way that it is salient to the animal and becomes attached to the behavior AND the consequences. Sounds simple? Well, it turns out the timing matters. He used the Kamin experiment on blocking to show that when you add the cue, you need to make sure that there is not another cue in the environment that will prevent the animal from learning the new cue. In the Kamin experiment the rats were conditioned to freeze (yes, yet another experiment shocking rats…), and they looked at whether or not a new cue could be conditioned if an existing cue was in place. If the new cue (light) is presented at the same time as the old cue (tone), no conditioning occurs. They did several combinations (differences in timing of light and tone), but conditioning only occurred if there was a discrepancy between the presentation of the new cue and the old cue. It may appear as if both are part of the stimulus condition for the behavior, but unless you try them separately, you don’t know. This is why you need to identify possible stimuli that are part of a context cue and also why you need to add a cue by doing “new cue,” pause, “old cue” so that the animal pays attention to each one individually.
The last part of his talk was perhaps the most interesting as it drew together a lot of these separate threads. And it led to his new “JRR Principle of Learning.” The JRR Principle of Learning says that in order for learning to occur, there has to be something in the environment to which the behavior can become attached. If you can’t predict when something is going to happen, then you can’t learn from it.
He showed this with some video of rats being trained to take vodka out of syringes. The rats were trained to move between two stations on a desk and take apple juice out of a syringe. Once they were doing this reliably, the syringe was sometimes filled with vodka instead. The experimenters replaced the apple juice with vodka so that it was about 4 syringes of juice to 1 syringe of vodka. The rats hated the vodka and their body language clearly showed their disgust, but they continued to take it because they could not tell when a syringe would contain vodka vs. when it would contain apple juice. There was no antecedent to indicate when it was vodka, so they could not learn to avoid it. Over time their body language did change so that they seemed to be “getting used to” the vodka and it seemed less aversive.
Then they made it so there was a “cue” for vodka which was a paper plate around the syringe. Within 2 or 3 trials the rats no longer drank from the syringe with the vodka. So even though their body language suggested they didn’t mind the vodka, they still chose not to drink it. And even if they were reinforced for drinking vodka (with a syringe of apple juice), this was not enough to maintain the behavior. The reinforcement value of the apple juice was not enough to counter the unpleasantness of the vodka.
Jesús’s conclusion from this was that it’s not enough to just read the body language of an animal that might be asked to do something it doesn’t like. You don’t know how the animal really feels about until you give them a choice with a clear cue to indicate what they are choosing. I think this has a lot of ramifications for some of the husbandry work we do with horses. I realize that some things are not optional, but I also think that we should be careful about assuming a horse is ok with something based on its body language alone. In some cases, it might be a good idea to set up a little experiment to see how the animal really feels about it.
Kay Laurence: Connected Walking
Kay Laurence had both a lecture and a lab on “Connected Walking.” Kay started off by saying that this is a protocol that meets the needs of the people and the dog and that it works for 95% of the dog/handler teams she meets. The other 5% require some minor tweaking. While this was about walking with dogs, I thought a lot of her ideas would work well with horses. I see a lot of cross-pollination between her work and Alex’s and this is one area where there are similarities.
Connected walking is about walking with connection between you and your dog. It is about more than the physical connection. A person can be physically connected to a dog, but not emotionally connected or moving together. In Kay’s connected walking, she is looking for the person and the dog to be sharing the experience of being out together.
The idea behind connected walking is that dogs seek connection with their owners and that this connection is reinforcing to both parties. Connection is inherent, reinforcing, underpins all communication, changeable (don’t have to have the same level of connection all the time), a 2 way street, and can be challenging (it varies from breed to breed and there is probably some genetic component). Connection comes from sharing positive experiences, creating trust and reliability, security in each other, and being comfortable with each other. Building connection takes effort from both sides as it is not just about training the dog to accommodate you. It is also about the owner learning to recognize the dog’s needs.
Most people walk their dogs using some kind of equipment. There is nothing wrong with this (safety is important), but you should recognize that all equipment is inherently punishing. Dogs are not born to wear anything so even mild equipment like a flat collar takes getting used to. Also, a lot of equipment is designed to suppress behavior, not to teach the dog what to do. The wrong choice of equipment can prevent you from building connection. Equipment should be chosen carefully and used in such a way that it doesn’t have a negative effect on the experience of being together. Along those lines she talked a little bit about how dogs can learn to make associations such as being on leash = bad stuff is going to happen vs. being off lead = freedom and fun. If you want your dog to enjoy being on leash, then you need to make sure fun stuff happens when the dog is on leash.
How does equipment suppress behavior? Some of the ways this happens are obvious, as many items are marketed as preventing certain behaviors (no pull harnesses, head halters, etc..). Other ways are just a by-product of what the equipment is intended to do. Equipment can make dogs feel trapped and frustrated. It can prevent them from behaving in normal ways and force them to put up with unwanted advances by strangers (both people and dogs). It can create behavioral resistance by actually strengthening behaviors you don’t want. And if the equipment is aversive enough, it can make the shared experience of walking together so unpleasant that this spills over into other areas of your relationship.
In addition to looking at the effect of equipment on walking with our dog, we need to look at the physics of how we walk when we walk with our dog. Kay has spent some time studying dog/human pairs that can walk comfortably together and comparing them to dog/human pairs where the dog has to do a lot of stopping and starting, or ends up pacing. It turns out that if you compare normal walking speed (and stride length) for most people and the normal speed (and stride length) of dogs at the walk and trot, you can see that there are only certain combinations that are going to mean that both the person and the dog are moving at a comfortable pace.
If the person’s walking speed is between the dog’s walk and trot speeds, then the dog is going to have to start and stop to keep pace, or it is going to have to find some in-between speed which often ends up causing the dog to pace. Pacing is very hard on a dog and can lead to physical issues so it is not something you want your dog to be doing. This means the person has to adjust by either walking slower so the dog can also walk, or walking faster so the dog can trot. Otherwise the dog has to compensate and you get pulling, starting and stopping or unnatural movement. She believes that a lot of pulling happens when the dog’s trot is faster than the person’s walk and the dog doesn’t compensate by starting and stopping or pacing. If you can the dog can find a comfortable speed for both of you, then the leash should be able to be loose or have a soft connection.
When I got home I walked our dogs to see how we are doing. It turns out that my casual walk is a nice speed for the shi-tzu to trot. If I take the border collies out, I have to walk slower to accommodate their walk or walk faster so they can trot. I thought it was pretty funny that the most comfortable dog for me to walk is actually the shi-tzu, considering I am about 5’7″ and have long legs.
She talked quite a bit about why people walk dogs vs. why dogs want to go on walks. People and dogs go for walks for different reasons. People go to get exercise, exercise their dogs, see a new place, as a social event, etc… Dogs go for walks to sniff, check out the environment, and do other natural doggy behaviors such as chase animals, roll in things, etc… It’s important that when you take your dog for a walk, you allow the animal to enjoy itself and meet its own needs. Dogs also do not usually choose to walk at the same pace and in the same direction for long periods of time. So some kinds of leash walking are asking the dog to behave contrary to its nature.
Her connected walking protocol has three basic components which are connected walking, transport, and parking. I am going to start by describing connected walking because the goal is to work towards doing connected walking most of the time. Connected walking is about walking together, but in a more companionable sense than the usual idea of “taking the dog for a walk.” It is more about enjoying being outside moving through the environment in a protocol where both you and the dog are allowed input in what you choose to do. The dog is allowed to sniff, look at things, stop, etc.. within reasonable bounds that you can set. The way you establish these boundaries is by using the leash to limit the dog’s behavior and teaching the dog to reconnect to you so you can move on.
This element of the dog choosing to reconnect with you is central to the idea of connected walking. You can teach it by using a clicker and treats (or some other reinforce), but the clicker and additional reinforcement should become less necessary and eventually faded out as the connection between you and your dog builds. With many dogs, she doesn’t use a clicker and treats at all. The moment when the dog reconnects is a “visual click” and continuing the walk functions as a reinforcer too. She had some video of a woman walking her dog with a nice connection. They were moving together at a relaxed pace. If the dog wanted to stop and look at something, the woman waited until the dog reconnected with her, which in this case was when the dog looked at her and moved toward her. If the dog took the slack out of the leash to pull in a direction she didn’t want to go, then she just waited holding the leash steady at the point of contact (sound familiar?).
Connected walking does take some skills. The person walking the dog has to learn how to hold the leash (she recommends a trapeze hold), adjust the length of the leash as needed, and wait in a stable position if using the leash to set a boundary. They have to learn to assess the environment to see what’s coming and make changes as necessary. They have to learn to walk with the right speed, energy, and rhythm. They also have to make decisions about knowing what and when to reinforce and when to change strategies, especially if they need to go into “transport.”
“Transport” is the second component of connected walking and it is pretty much what the word implies. It is a method of walking where you are moving the dog from one place to another with efficiency. She recommends that you use transport in situations where you just need to move the dog quickly either away from something or to a new location. In transport the leash is shortened and you are using pressure on the collar to direct the dog in the direction you want. Dogs pick up on our intent so if you walk with a sense of purpose, the dog will too. Most dogs learn the difference between connected walking (loose leash or soft connection) and transport. If the dog is uncomfortable with transport, you can click and treat every few steps until the dog understands the protocol.
When you are out and about with your dog, you are going to be using a mixture of connected walking and transport. For younger dogs, or dogs new to training, or in some environments, you are going to use a lot of transport. Over time you will be able to add in more and more periods of connected walking until transport is only used when something unexpected happens.
The third component of connected walking is “parking.” When you park your dog, you step on the leash so that the dog has limited room to move (more on this below). Parking is about teaching the dog what to do when you are not moving. It’s a useful behavior to do in dog training class if you need to stop working and listen to the instructor. It’s a useful behavior to do if you want to stop and chat with someone while walking your dog. It’s also a useful behavior to do if you want to prevent someone else from having access to your dog. Kay is a big believer that our dogs do not need to be touched by strangers. She spent part of the lab showing people how to teach their dogs to park and also had them practice “repelling” strangers by stepping forward and blocking access to the dog.
So how do you park your dog? While the dog is near you, you are going to take the dog’s collar and hold it, then carefully step on the leash so that the dog has less freedom to move. The dog should have enough length of leash to sit or lie down next to you and still have slack in the line. With small dogs, the dog may have enough room to stand or move around a little, but there should be a slight downward feel on the collar if the dog moves too much. This will encourage the dog to choose a more stationary (and lower to the ground) position. Kay does not want the handler to interact at all with the dog when it is in park. This is “down time” for the dog, so no talking, no treats, no patting. If you are consistent about ignoring the dog, it will learn that it is just supposed to take a break and settle down. When you want to move off again, you carefully step off the leash by stepping back, wait for the dog to connect and move off.
In the lab, Kay demonstrated parking, repelling strangers, connected walking, and transport. She showed how you can move easily between connected walking and transport. The dogs and their people went for a little walk through the lobby and around the mezzanine and learned how to move at a pace that was appropriate for the dog. In one area where there was a lot of activity, the dogs needed a little time to think before moving on and in other areas Kay had them use transport to get past an obstacle.
At the end of the lecture, Kay shared these thoughts. Connection is about investing quality time into a relationship and enjoying your dog for being a dog. A lot of dog training puts the emphasis on “fixing” your dog. But part of what we like about dogs is who they are. Rather than “fixing them,” dog owners should learn to dismiss the need for external social approval and do what is right for them and their dog. This means having realistic expectations, being together in the now, and enjoying the reinforcement that comes from connection.
Kay Laurence: World of Targeting
On the last day I attended Kay Laurence’s World of Targeting Lab. Kay’s use of targeting is different than any other trainer I have seen. Instead of using targets to get behavior started, she adds a target once the animal is already doing the behavior she wants, or has a good start at it. She also uses different targets for different things. So rather than using one target for left and right paw touches and having different cues, she uses one target for left paw and one target for right paw. When the animal sees the target, it does not have to guess which behavior it is going to be asked to do. The target itself provides that information. In this way, the targets function more as cues than as objects to help start the shaping process. What she has found is that using different targets and presenting the target later in the shaping process creates stronger and cleaner behavior, and the training is more error-free because she never puts the dog in a position where it has to guess.
The lab was designed to show how this actually works. She started off by talking a bit about the advantages of using targets this way. She finds you get less variability in the behavior and this makes it more reliable long term. This is partly because in addition to adding the target later, you also don’t start clicking until later. You don’t start clicking until the behavior you want (for that step in the process) is happening on a consistent basis. This means you have only clicked responses that are closer to your final criteria.
Kay likes to keep things simple. She doesn’t use body parts (hers) as targets. She said if you use your hand as a target, the dog will sometimes become confused about what to target and this can create a “stutter” in the behavior. You don’t need to use modifiers (left, right etc..) as the target object alone tells the animal what to do. If you want to use the same target to create different effects such as a high five vs. wave with a paw target, you can change how you present the target to get the different behaviors.
In the lab she showed how to train paw touches, going to a mat (the mat is the target), and following a target to get the dog trotting out. The first exercise the attendees did was to teach the dog to cross its paws when lying down. This is one behavior that can be taught using a paw target. She gets the behavior started with food delivery. With the dog lying down, she had the trainer feed the dog (no click) by pushing the food under one front leg (starting from the midline of the dog.) The dog will move the leg to the side to get the food and then return it back to a comfortable position. This returning leg movement is the action that Kay wants.
With some of the dogs it took a while to figure out exactly where to put the food to get the correct movement. The food has to be placed so the dog moves the leg out, and the leg has to move far enough out that the dog wants to move it back after eating. Once the “return” leg movement is happening regularly, Kay has the trainer start clicking. The sequence is as follows: place the food under the leg, the dog moves the leg out to get food, dog returns the leg to center, trainer clicks, and places the food under the leg to set up for another repetition.
Once this part has a nice rhythm to it, you can start clicking. Then you can add a target so that the target is in the path of the dog’s leg when it moves it back toward center. Once the target is in place, the clickable moment is touching the target. When the dog is intentionally putting a paw on the target, you can start to move the target towards the other leg to start to shape the paw crossing over the other one. She recommends training left and right on the same day, or within a few days so the dog doesn’t develop a preference. This is one way to teach a paw target which can then be used to teach other behaviors such as shake, wave, jazz steps, Spanish walk, etc…
That’s just one example of a typical shaping sequence using food delivery to get the behavior started and then adding a target. She also used food delivery (throwing the food) to start the behavior of going to the mat. The trainer starts without the mat and throws food out away from where she is standing so the dog gets in a pattern of running out to get the food and then running back to the trainer. Once the dog is doing the behavior “trot back to owner” consistently, then she adds the click. After that is working well, she adds the target (mat).
When you start by using food, the dog is not necessarily aware of what behavior it is doing to get reinforced. This is ok as you are just trying to create a pattern of movement. But it is important that when you add in the click and/or target, you start to see some deliberate action on the dog’s part. When training “go to the mat,” Kay wants to see the dog look at the mat, as opposed to just running back to the person without paying any attention to what is under its feet. The dog in the lab was not paying attention to the mat until after a few repetitions and this is pretty normal for a dog that has not been trained using this sequence (food, click, target). I think animals learn about different training “strategies” if you use the same ones for many behaviors. My guess is that after you trained a number of behaviors using food delivery and then adding the target, the dog would be quick to recognize when you were clicking for some interaction with the target and this transition would be smooth and quick.
While Kay was showing how to train the “go to mat” behavior, she talked about how important it is to define the behavior before you start shaping. What should “go to mat” look like? Do you care how fast the dog goes (walk vs. trot)? Do you care what the dog does when it gets there (stand, sit, lie down)? She has some mats that she uses for resting or taking breaks. She has other mats that she uses for “hold” positions. Those are used to send a dog to a location and ask it to wait before it goes off again. They are often shaped so the dog has two legs on the mat. If you have a variety of different mats (different shapes, sizes, colors), you can have mats for lots of different things.
She also shared a lot of the details that make this exercise work. How you throw (underhand is better than overhand), where you throw (throw in different directions so the dog learns to approach the mat from different angles), and how far you throw (farther away will get more movement) are some of the things that matter. But more importantly, this attention to detail pointed out the obvious…. which is that details matter. The more things you can decide up front and the more careful you are about observing what affects the dog’s behavior, the more successful you will be in your training.
The last behavior she worked on was “trotting out” which is getting the dog to follow a target in motion. She uses a “scent target” which is a dog treat in a cup on the end of a stick. The dog orients toward the cup and this encourages a certain body posture. The “scent target” also creates focus. At home she will reinforce the dog by throwing the treat that is in the cup. I think she just tosses it by tipping the cup slightly and swinging the stick. This didn’t work very well here with the set-up she was using, so she just threw a treat as if it had come from the cup.
This led into a bit of a discussion about why one needs to use a clicker at all? What is the purpose of the click? I think we all know the standard answers to this question which are that it adds precision and allows you time to deliver the reinforcement. The click marks a specific behavior so by selecting it out of a stream of behavior. This adds precision and removes uncertainty and increases focus. The click also allows you to separate out the click from the reinforcement.
This is the part Kay was interested in. She wanted to show that this gives us a lot of flexibility in how we handle the reinforcement. If you always do the click -> reinforcement part in the same way, you are missing out on opportunities to increase the power of the reinforcement or the clarity of the click. She did a little demo of the difference between holding the food and throwing it right after the click as compared to getting the food from the table right after the click. The little delay as she walked, from where she was standing when she clicked, to the table made the dog focus back on her and built anticipation. This also had the benefit of teaching the dog to go away from the reinforcement (on the table) so that it could do the behavior that would earn the next reinforcement. This builds power and reliability.
I learned a lot of things from watching this lab. Her approach is different than what I usually see where the target is used to start the behavior. This made it easier to identify how each element of training could be ‘tweaked” to get different results. If you start using food delivery and nothing else (no target, no click), you have the ability to really play around with food delivery and see what happens when you change things. This is great information about what works for your dog and how to use food delivery in the future to get different results.
Adding the click a little later in the process made it easier to see how the click was working. Was the timing right? Did the animal really understand what was being clicked? Adding in the target as a separate piece also made it easier to make sure the dog knew that the click was about the target. Separating out these elements made it simpler for the dog and also for the trainer to identify what was working and what was not.
I realized again how important it is to have a good shaping plan. You need to be able to clearly define the behavior you want. You need to plan how to use the food, when to add the click, and when to add the target. You need to have good mechanical skills. Delivering food so that it supports the training is important and does require some skills.
You also need to know a lot about the natural behavior of the animal you are training. Kay has spent so much time with dogs that she knows how to position the dog so it is more likely to do the behavior she wants. There were details she shared about how to position the dog to make the cross paws behavior more likely, including such things as which breeds have looser elbows (which affects how you place the food). She also knows how to take advantage of a dog’s natural tendencies so you work with the dog’s nature instead of against it. All three of these exercises used the dog’s natural response to something as a way to get the behavior started.
Dr. Jesús Rosales-Ruiz: Negative Reinforcement
Sunday afternoon I attended Jesús Rosales-Ruiz’s talk on negative reinforcement. Yes, there was a talk on negative reinforcement at Clicker Expo! It’s quite amazing really when you consider how controversial the subject is in any gathering of clicker trainers. When I first met Jesús in 2006, I asked him about using negative reinforcement and he said it was not a good way to train, although he did add that Alexandra Kurland seemed to be able to use it well, and I should talk to her if I wanted to learn more about it. His answer was consistent with a lot of the material I had read, which said that even though negative reinforcement is reinforcement (as opposed to punishment), most behavior analysts and a lot of trainers seem to put it in the “don’t use” category.
In his lectures, Jesús often includes a bit of history of the topic he is discussing. This glimpse into the evolution of an idea or way of thinking is useful because it shows how ideas develop and change over time as new discoveries are made. It also makes it easier to figure out why people might believe something at one time based on their understanding of how things work, and then change as advances in technology allow them to see new details or learn more about how a system works.
His discussion of negative reinforcement started with B. F. Skinner who first made the distinction between two types of reinforcing events. He defined positive reinforcement (+R) as a situation where you add something, and negative reinforcement (-R) as one where you take away something. The only thing they have in common is that both increase the probability of the behavior.
This seems pretty simple, but in many cases, it is not clear what is motivating the change. Is it something being added or something being removed? If I am cold and put on my coat, am I removing cold (removing an aversive) or adding heat (adding something I want)? Perhaps it depends upon how cold you are? This is kind of a silly example, but there are lots of situations where you can argue that something is positive reinforcement or negative reinforcement, depending upon how you look at it.
Negative reinforcement results in an increase in one behavior (or maybe a related set of behaviors), but this can manifest itself in different ways. They can be broadly classified into either “escape” or “avoidance,” but Jesús broke them down farther into the categories listed below:
1. It can lead to the reduction of some event as a consequence of behavior. An example of this would be using the reins to ask for the horse to stop. If the horse stops on a slight change in tension, it can reduce the likelihood of being pulled hard in the mouth (punishment).
2. It can lead to elimination or removal of something aversive. Taking aspirin for a headache is another behavior that is maintained by negative reinforcement.
3. It can lead to postponement of an event. Some kinds of evasive action are maintained by negative reinforcement. Procrastination is a great example of this. People procrastinate even though they usually do end up having to do the job they are avoiding. Even a temporary delay is reinforcing so procrastinating is maintained by negative reinforcement.
4. Reduction of the event’s frequency or occurrence. Sleeping more is maintained by being sick less often, feeling less tired etc…
5. Reduction of intensity. People diet to reduce their blood pressure. If that is why you are dieting, then dieting is maintained by negative reinforcement.
As you can see, negative reinforcement is a little more challenging to identify than positive reinforcement. An additional complication is that negative reinforcement and punishment (positive punishment/+P) are often confused. Because the word “punishment’ was in use before Skinner defined the quadrants, people often use the word punishment when they really mean negative reinforcement. And because negative reinforcement contains the word “negative,” people often say negative reinforcement when they mean punishment.
As if this wasn’t bad enough, negative reinforcement and punishment are often paired in real life. Skinner said that punishment set up a condition under which negative reinforcement could work. This is because in order to remove something (-R is removing something), there must be something present to remove. If it’s not present, you have to add it (+P…maybe) so you can remove it. When I was first learning the quadrants, it was hard for me to get my brain around this, especially since I used negative reinforcement in my training and it didn’t feel like punishment (in the cultural sense). So just remember there is the cultural definition of punishment, then there is the scientific definition which is the addition of a stimulus that decreases the likelihood of a behavior happening in the future.
So out of this, how do we reconcile using negative reinforcement with clicker training? Jesús said we have to look at how we use it in our training. One of the basic ideas in clicker training is that we ignore errors (or deal with them otherwise). This is a “constructional” approach because we are focusing on what we want. When we come up with a training plan that includes negative reinforcement, we need to put the emphasis on increasing a desired behavior, not on using it to manage or control an undesired behavior.
An example he shared was training loose leash walking. A trainer who is focused on negative reinforcement in a constructive way is going to be focused on using negative reinforcement to increase the likelihood of a slack leash. They will not be focused on using it to make the dog do less of something (pulling). This may seem like a subtle difference, but in training the intent of the trainer is very important.
He talked a little bit about the importance of choosing your reinforcer and having a good training plan. To make a good training plan you need to look at what you have (component behaviors), decide where you want to go, decide how to get there, and choose the type of reinforcement you are going to use. You are better off with a good training plan and mediocre reinforcers than you are with excellent reinforcers and a poor training plan. He also does not recommend mixing types of reinforcement. If you use both positive reinforcement and negative reinforcement, then you don’t really know what is motivating the animal.
A constructive approach to training, including how you handle errors, is very different than the more traditional ways to handle errors. He had a list of 9 ways to deal with errors that is published by a traditional dog trainer. Every one of the nine involved some kind of punishment. I can’t remember exactly why he shared the list, but I think it was partly to show how negative reinforcement has been used by dog trainers as part of punishment and that this is why many trainers choose not to use it at all.
At this point he talked a little bit about jargon. He had already mentioned how negative reinforcement and punishment are often confused, but now he expanded upon the idea that the jargon that is used by behavior analysts actually makes it harder to understand what the quadrants do. He shared some other examples of commonly held beliefs about the quadrants, and suggested that perhaps we need to rename the quadrants to get rid of the confusion. Other behavior analysts have suggested the same thing. Lindsely suggested the following alternative names: positive reinforcement= reward, negative punishment = penalty, positive punishment = punishment, and negative reinforcement = relief.
While renaming the quadrants might be one step toward removing some confusion, the reality is that what really matters is your procedure. Your procedure needs to creates a positive learning experience for the animal. You also need to be using your reinforcement (positive or negative) to build behavior by focusing on what you want. If both of these are true, then you are using your reinforcement in a constructive way and that is consistent with the principles of clicker training.
That led to the next part of the lecture which was a closer look at different ways that negative reinforcement is affecting our behavior on a daily basis, whether we know it or not. To understand this, it helps to remember that negative reinforcement can result in escape or avoidance. Jesús mentioned earlier when he was listing the ways that negative reinforcement can operate. In escape, the animal learns to that it can remove (or get away from) an aversive by doing another behavior. In avoidance, the animal learns that it can avoid the aversive by doing another behavior.
Whether negative reinforcement leads to escape vs. avoidance can change over time. In many situations, negative reinforcement starts as escape and then becomes avoidance. This can happen if there is a stimulus that precedes the addition of the aversive. This stimulus becomes a cue and the animal can avoid the aversive by responding to the cue. If there is no cue, the animal can’t learn to avoid the aversive and is stuck in escape (remember the rats drinking vodka in the “Cues and Context” lecture?).
Once you recognize the many ways that negative reinforcement can operate, you realize that it is everywhere. A lot of the information we get about our world is in the form of cues that allow us to avoid aversives. Many of the behaviors that we do when driving a car are maintained by negative reinforcement and they are cued by things like traffic signs (stop, yield, merge, etc…), traffic lights, speed limit signs, warning signs (“bridge freezes before road,” “windy road,” etc..).
A lot of our “choices” are maintained by negative reinforcement. Why do students study? Some students study because they are afraid to fail. This is a different kind of motivation than studying because you are interested in the material. Why do people to work? Do they go because they like it, or because they are afraid of getting fired, not having money, or some other reason that is more about avoiding a consequence?
Why does this matter? It matters because motivation matters. If you are avoiding, escaping, or running away from something, you are going to be in a different mental state than if you are moving towards something.
Addictions are maintained by negative reinforcement. Withdrawal feels bad so even if you have some desire to stop, you might be unable to do so because the urge to do it again is too strong. Incidentally, Jesús pointed out that if you do give in when you have the “urge,” you are making things worse than if you gave in at some other time.
These examples may make it seem like negative reinforcement is always bad, but this is not the case. If you think back to the lecture on operant vs. respondent conditioning, Jesús noted that a lot of biological feedback loops (in our body and other places) are controlled by negative reinforcement. I always remember Kathy Sdao’s example of urinating as being controlled by negative reinforcement. When we urinate, we feel a sense of relief. Jesús used the example of a crying baby. Babies cry when they need something and we respond because the sound is aversive. This is a good thing! We need to take care of crying babies.
So if negative reinforcement is prevalent in the environment and so effective, why is its use in training controversial among animal behavior analysts? There are a number of reasons including:
1. motivation – why you are doing something is important as it affects your emotional state
2. it needs the application of an aversive stimulus (this can generate unwanted emotions)
3.. incompatible responses make it difficult to use (you may get more “garbage” or “baggage” attached to your behaviors)
Jesús quoted Iwata (1987) as saying “In at least one sense, -R might be considered to be more intrusive than punishment because with -R, presentation of the aversive stimulus is contingent on the presence rather than the absence of behavior.” He also quoted B.F. Skinner who said “If we knew as much about -R as we do about +R, we might find it good for shaping, but at the moment it’s ineffective and has negative by-products.”
So where does this leave us with regard to training with negative reinforcement? On one hand, it is in the environment and clearly affecting our behavior so it could be an effective training tool. On the other hand, there are clearly disadvantages to using it because of its close association with aversives and punishment. He spent the last part of the lecture showing how trainers can use negative reinforcement in a positive or “constructional way.” This is really an important piece. It is about recognizing what is controlling the existing behavior and setting up conditions where reinforcement works best (whichever you choose). If you start with negative reinforcement, you have the additional step of switching over to positive reinforcement.
He shared two examples. One was the CAT (constructional aggression treatment) protocol which uses negative reinforcement to shape friendly behavior in dogs. For a fearful dog, the best reinforcer is distance. Distance is more important to them than food, toys, or other things that could be added with positive reinforcement. He showed a video clip of a shelter dog who was fearful about being approached. The trainer would approach the dog and retreat when the dog showed some sign of calmly accepting his presence. This could be a change in posture or movement. They look for behaviors that are seen in “friendly” dogs.
Over a series of approaches, the dog becomes less and less fearful until the dog is comfortable with the trainer approaching. Once the trainer is close, the dog will indicate when it is ready for an interaction that involves positive reinforcement. So they start with negative reinforcement, but then change over to positive reinforcement. (If you’ve never heard of CAT and want to learn more about it, google CAT + Jesús Rosales-Ruiz).
An important point here is that aggressive behavior in dogs is often maintained by negative reinforcement. The dog is using aggression to get the aversive (other dog, person, etc..) to move away. By using negative reinforcement to shape the dogs behavior to from aggressive to friendly, Jesús is using the same motivation. He finds that behaviors that are maintained by negative reinforcement are often easier to solve by using negative reinforcement.
His other example was the rope handling that Alexandra Kurland uses. In her rope handling and use of pressure, Alex teaches the horse to accept physical contact (either through a rope or body contact) as information. She starts out with adding a small amount of pressure and waits for the horse to respond. There is no escalating of pressure and the amount used is individual to each horse. The release of pressure is marked with a click and she reinforces the horse for the correct response. Over time she can use less and less pressure and the contact becomes more of a cue.
This is the same idea of starting with negative reinforcement and then moving toward cues and positive reinforcement. I’m not sure one can say that a cue taught this way does not have some component of negative reinforcement, but you can always add a new cue once you have the behavior. The poisoned cue video showed that the behavior is not poisoned, it is the cue that is poisoned. Adding new cues is one way to remove any emotional fallout from using negative reinforcement. If you do that, then you don’t have to wonder if your cue is a conditioned aversive stimulus or a cue with positive associations.
There was a little discussion here about if the animal is “learning” when it is being directed by negative reinforcement. Jesús indicated that in order for real “learning” to happen, the animal needs to learn to do the behavior without guidance. I have to say that I’m not sure I agree with such a narrow definition of learning. I think animals are learning all the time and one thing I want my horses to learn is to accept various kinds of guidance from me. Yes, if I want them to do the behavior independently, I need to slowly remove the guidance (like taking training wheels off a bike), but that doesn’t mean that they are not learning important things while they are still in the “guidance phase.”
These examples show that negative reinforcement can be used in a “positive” way if applied with discretion in a positive training environment. Jesús said that if negative reinforcement is already controlling a behavior, you can often change it by using negative reinforcement (as in CAT). If an animal is not offering behavior, you can use negative reinforcement as information to help the animal find the right answer (as in Alex’s work). He added that even with these examples, the goal is to move towards positive reinforcement because for most animals positive reinforcement is more fun.
Eva Bertilsson and Emilie Johnson Vegh: This class is a Blast: Five Steps to phenomenal teaching success
This was the last lecture I attended and it was a nice one to hear at the end of the expo because it was less about minute details of technique and more about how to help others learn clicker training by creating a fun, but practical learning environment. In their notes they state that this lecture is about “developing competent, confident, and creative trainers – setting up for successful learning in class.”
In the beginning of their talk they shared their thoughts on working with students including setting goals and developing independence. They want the students to learn both what to do (knowing and understanding) as well as how to do it (mechanics). Then they can learn how to develop stimulus control and fluency. Over time students should learn to be work independently so that they can make decisions, solve problems, and ask for help when needed.
Yes, the last one is important. Working independently is not the same as working alone and interestingly, although their emphasis is on creating independent trainers, they make the trainers work in groups. Groups actually encourage trainers to have to work things out in their own little groups, which makes people become more involved. When people work on their own, they tend to wait for the instructor if they get stuck. Working in groups adds another level of support between being working alone and working under direct supervision of the instructor.
Working in groups also allows them to rotate jobs, spend time observing, get regular feedback (a group member is always observing), have discussions which help with understanding, and learn to explain and answer questions so that their helpers know what to do. Of course it also helps that with a group there are people available to be timers, record keepers, and watch dogs, etc…I talked to Eva after the lecture about how people feel about working in groups and she said that initially some people would rather work alone, but they get used to working in groups and eventually see the benefit.
They talked quite a lot about developing trainer mechanics. One of the key points was that it is better to build a few skills to fluency rather that to try to be perfect at everything. They wrote “We can only choose to do what we actually have the skills of doing.” So they concentrate on teaching fluency in selected skills, which can be used to train a variety of behaviors. As part of teaching trainer mechanics, they use clearly defined exercises that show what to do and then when to do it.
They make everyone go through a four step process each time they train a new behavior. Each step allows the trainer to focus on different components that they need to work out before they actually work with the dog. The steps are to practice the skill with no dog, practice with a pretend dog, practice with a human, and practice with a dog. Getting people to practice without their dog is important so that they work out the details before training the dog, but also…it is important because people get used to working this way. Then if it is not going well with the dog, they can go back to one of the previous steps, but without the feeling that they have done it “wrong.” Instead it seems natural to go back and find an easier step, just like they do with the dogs.
To teach understanding and planning, they teach the students to be more specific about their goals. What does the finished behavior look like? How does this feel from the dog’s perspective? Do they know what to do, when to do it, and why? What does the dog need to know before they start? What do they need to know or be able to do (mechanical skills) before they start? They also show how you can use flow charts, checklists, feedback lists, and writing out instructions to helpers to help come up with a good plan.
They had some good video of classes and training sessions and one thing I saw that I really liked was that they created an environment where people were clearly learning, but there was a bit of a light hearted atmosphere. In some of the dog-free activities, people were encouraged to get into their roles and some of the video of the people pretending to be dogs showed how much fun they were having. Training can be so serious that I thought it was nice to see people loosen up. I am smiling as I write this because anyone who knows me will know that I have a hard time loosening up, but I have good memories of Alex clinics where we pretended to be horses and all ended up laughing. Laughing is part of good training too.
Panel Discussion with Karen, Kay, Alex, Julie, Kathy, Ken, Jesus, and Michele
Every year they have a panel discussion with some of the faculty members. I always enjoy this as it’s a chance to listen to different viewpoints on the same topics. The panel discussion featured Kathy Sdao, Michele Pouliot, Ken Ramirez, Karen Pryor, Alexandra Kurland, Jesús Rosales-Ruiz, and Julie Shaw. The questions for the panel discussion are submitted by expo attendees and Aaron Clayton chooses which ones to ask. This year Aaron chose three questions.
The first question was “does clicker training bring out qualities in animals that we would not otherwise see?”
Kay started off by saying that she thought clicker training brought out qualities in people that we would not otherwise see (compassion, empathy, etc..). There was general agreement on this. Kathy said that she finds that people learn to welcome behavior instead of being afraid of it. They learn that behavior follows rules and that they can change behavior. This leads to recognizing what is natural for the animal and accepting it or coming up with solutions that meet the needs of both the animal and the trainer.
Jesús and Karen both expanded on this idea that clicker training makes animals offer more behavior and that it is more varied. They said that, with clicker training, animals learn to experiment and communicate with us in new ways. Ken said he sees happier, more comfortable animals. The animals are not afraid to make mistakes and are more persistent about working through problems or training challenges.
Almost everyone shared a story about an animal doing something unexpected. I liked Michele Pouliot’s story about how the service dogs sometimes help with finding the crosswalk button if the person keeps missing it. The dogs will actually push the person in the right direction so their hand can find the button. This was not a specific behavior that the guide dogs were trained to do. The dogs just figured it out on their own. Alex talked about how her horses now make sounds that she didn’t even know horses could make.
There was a little discussion here about how clicker training changes the relationship between people and their animals. Karen said that by teaching animals about cues and other concepts, we teach them to think and make choices. This allows us to have a different type of conversation that is different than working with non-clicker trained animals. Kathy talked about encouraging creativity and how it changed the trainer’s perception of what the animal could do. Kay thinks clicker training changes relationships because people are listening to animals instead of just telling them what to do. There was general agreement that clicker training should empower the animal.
The second question was “how do you train for generalization?”
Ken said generalization is part of his everyday training program. They want an animal to be able to work in new environments, cope with distractions, and handle minor changes without becoming anxious or being unable to respond to cues. He said it is easy to become stagnant in your training and it’s important to expose the animal to new things all the time. Kathy shared how they worked with the dolphins to prepare them for the open ocean work. They trained behavior to fluency in a few locations and found that if they had done that work well, the dolphins could work in a new environment.
In the course of this discussion, the role of the environment came up several times. It was stated by several faculty members that an important thing is for the animal to be comfortable in any new environment where it is expected to respond to cues. They take time to get the animal used to different environments as part of their training program and this is different than taking the animal to new environments to train. It is more informal. Kathy made a distinction between socializing and training.
Jesús says that animals generalize more than we think. The traditional view has been that animals don’t generalize well, but in some situations they generalize too well. There are lots of examples of animals that become afraid in one situation and then generalize to similar situations. For example, a dog that is mistreated by a man can quickly become afraid of all men. Julie mentioned that Temple Grandin has written that animals are more likely to generalize a behavior if it is associated with a strong emotion. Kathy agreed and said that fear is really “sticky.”
So it depends. According to Jesús, if the animal does not generalize, it is likely that the new environment is competing with our cues, or we don’t realize that our training environment is part of our cue. He said it takes at least 3 examples to generalize and it helps if you can keep as many things constant between new locations. You need to be pro-active about it instead of just expecting it to happen.
This led to a slightly different aspect of generalization which is not so much about adding distractions, but more about building cues so that they are not environment dependent to start with. When I say “environment dependent,” I mean that they are not using some aspect of the physical environment (that includes the trainer) that was not intended.
Someone (I can’t remember who) shared a story about a dog that learned a behavior in a basement and used the mirror behind the person as part of the cue. When it was asked for the behavior in another location, it did not respond because part of the cue was missing (no mirror.)
This is one reason you have to be very pro-active about teaching generalization. It is important to move around while training. Karen recommends just changing compass directions (north, south, east, west) relative to the dog when adding a cue. Kay said she makes a point of moving around and changing her own position so the dogs learn what is and what is not part of the cue. She said her dogs also generalize generalization.
The last question was “why do animals require such clean and discrete cues?”
Aaron posed this as a question from a frustrated student who didn’t see why the animal had to be given such precise cues. Why can’t the animal just figure it out even if the cue is a little sloppy?
In some ways the answers to this question were a continuation of the discussion they just had, but looking at it from a different angle. If I rephrase the question to “why don’t we want animals to generalize our cues?” or “wouldn’t it be easier if the animal generalized cues?,” then it is clear that you can’t talk about discrimination without talking about generalization about discrimination and vice versa.
In generalization we want the animal to recognize specific cues, even when the environment is different from the training environment. In discrimination, we want the animal to recognize discrete cues because we want the animal to distinguish between potentially similar cues to figure out which behavior to do. Part of the challenge of training is teaching an animal when to generalize and when to discriminate. This goes back to teaching the animal which details matter and which don’t.
Kathy had a very practical response to Aaron’s question which is that the more specific your cues are, the more of them you can have. She used the example of hand cues. If you use your left hand to cue “sit” and you are sloppy about it, you can train your dog that any cue with the left hand means “sit.” But then you can no longer use your left hand to cue other behaviors. However, if you have a very specific hand motion that means “sit,” then you can use another hand motion for “down,” and the animal will understand the different cues. You can have lots of hand cues with one hand as long as you have good mechanics (make them obviously different to the animal) and train the animal that each different motion is cueing a different behavior.
Ken shared a story about how they let guests train at the Shedd. They allow them to ask for a few basic behaviors and they usually have one behavior that is cued with the right hand and one behavior that is cued with the left. The animals (dolphins?) have learned that guests are sloppy, so they will spin for almost any right hand movement because they have learned that to ignore the differences in how different people give the cues. But when you take the same dolphins and put them in a training session with a Shedd trainer, they go back to distinguishing between multiple right hand cues. Their generalization is context specific.
The panel ended with this thought from Kathy. She said that sloppy cueing puts the cognitive burden on the dog (animal). This can create a lot of stress on an animal that is trying to figure out what we want. One of the best things we can do for our animals is to clean up our cues, so that the animals we love don’t have to wade through a lot of static to figure out what we mean.
Dr. Susan Friedman: What the word PARROT reveals about teaching dogs
The Saturday night talk was titled “What the word PARROT reveals about teaching dogs. In the acronym PARROT,
Before she talked about what each of these means, she talked a bit about conventional wisdom vs. scientific knowledge. I thought this was a really good place to start because so many of us have had to learn to look beyond conventional wisdom. And even if we have changed our own view of animal training, we are still encountering people who treat conventional wisdom as fact, and do not recognize the difference between conventional knowledge and scientific knowledge.
At the same time, our job is even more challenging because it turns out that scientific knowledge is not fact. It is only the most reliable information we have at the moment. As she pointed out, science self corrects, so today’s scientific knowledge may not be tomorrow’s scientific knowledge. She said “A fact only stands until it is replaced by a better one.”
I actually found this sort of a radical view. I was trained as a biologist and spent years memorizing facts about how biological systems work. Yes, I know that with new advances we learn more and can add details or change our understanding, but to me the word “fact” still implied some kind of absolute certainty. Or maybe it seems radical because our culture values knowing the answers and here is someone saying that the nature of scientific knowledge is that it changes. Sounds obvious when you put it that way, but the idea that facts change, well… I am still working on it. Moving on to the word PARROT…
Behavior is the engine that gives animals power to exert control over their environment. That is what behavior has evolved to do. She had some examples of very young animals (babies, rats, rhesus macaques) who show that being able to control their environment is very reinforcing and the ability to do so has an effect on their physical and emotional development. As dog trainers we need to learn to empower our learners more and impose on them less. She now considers power to be on the same level (a basic need) as food and water.
Approximations are the key to teaching new behavior. As clicker trainers we are all taught that shaping is the use of successive approximations and that behavior can be built by selecting out those variations that lead to our end goal. Ehen we are shaping and allow the animal to participate in the process, there is a tendency for the training to become one-sided. Dr. Friedman suggested that shaping should be a conversation. Any shaping plan is a loose suggestion and changes in criteria should not be imposed by the trainer, but come about by observing and taking advantage of natural variations in behavior.
The ability to learn (i.e. to change what we do), based on our experience, is our nature. We are designed to learn from the consequences of our actions. They are essential feedback about how to behave in the future. Is it important to think about consequences as more than just reinforcement or punishment. They are information. Using reinforcement as a consequence is not manipulation and reinforcers are not bribes. They are tapping into a natural system that is set up so we can learn.
She talked a bit about schedules of behavior here, which ties in with her talk on Blazing Clickers. She is a proponent of one click=one treat and also of a continuous reinforcement schedule for new behaviors and to maintain the strength of mastered behaviors. In some rare cases, intermittent schedules may be important if you need the behavior to persist in very low reinforcement conditions, but she suspects that behind every behavior problem is an intermittent reinforcement schedule.
Note that this does not mean you have to maintain the same type of reinforcement you use to teach a behavior. She recommends using secondary reinforcers and naturally occurring reinforcers. She also mentioned the Matching law (reinforcement value is related to effort) and looking at what is reinforcing all behaviors. She doesn’t recommend using extinction as part of your reinforcement (or non-reinforcement) strategy.
People tend to underestimate how much repletion is needed. You need a lot of repetitions to get behavioral fluency and generalization. It doesn’t have to be time-consuming as a few repetitions a day over a long period of time can be very effective. A high rate of repetition builds big trust accounts between trainer and trainer. Trust accounts are important and a less skilled trainer can be just as effective as one with more experience, if they take the time to build a relationship with their trainee and take time to build behaviors to fluency.
For an objective understanding of behavior, focus on what you see – what the animal actually does. Behavior is what animals do, not what they are. She talked about the ABC’s of behavior. The model she uses is of behavior as a 3 term contingency.
- A= antecedent (distant and immediate predictors)
- B=behavior (define it so it is unambiguous and observable)
- C=consequences (what is their purpose, outcome, feedback?)
If we want to control behavior, then we can do so by controlling the antecedent and consequences. Positive reinforcement trainers are very aware of the consequence part of this, but arranging the antecedent can be just as important. You want to arrange the antecedent to make the right behavior even easier.
See teaching opportunities where others see punishment opportunities. Misbehavior often comes from lack of information, lack of motivation or lack of practice. It can also be affected by weak or competing reinforcers. Animals learn from every interaction so we need to consider what the animal could do instead (of an unwanted behavior) and teach it.
She closed with Kay Laurence’s quote (which I think Kay got from one of her students) that it’s not about teaching impressive behaviors, but about impressive teaching of behaviors. If you want to learn more about Dr. Susan Friedman, her web site is www.behaviorworks.org.
Dr. Friedman also talked during the closing ceremony on Sunday. The Sunday talk was on “Blazing clickers.” I don’t need to repeat it here because you can read her article of the same name at the following link: http://behaviorworks.org/files/journals/Blazing Clickers.pdf
I hope you enjoyed reading the Clicker Expo reports. If you have any questions or comments, feel free to contact me.
Katie Bartlett, 2013 – please do not copy or distribute without my permission