A Training Strategy for Building Duration: Backchain it by using a terminal behavior.

Duration with a short chain.jpgBuilding duration is often a sticky point for many trainers. I know that I find it fun and easy to get behaviors started, but when it comes to building duration and adding stimulus control, it starts to feel a bit more like work. Part of this is because those two processes often involve withholding reinforcement for an existing behavior (which seems slightly unfair) and keeping better track of what you are doing (which requires better record keeping).   This is a little different than the shaping part where I have a training plan, but my job is to work with what the learner offers and the learner typically has more freedom and choices about how we do the training.

But that doesn’t mean that building duration has to be difficult. It’s really just a matter of having a variety of options for ways to build duration and choosing the right one for the animal and the behavior that you are training. I was originally taught to build duration by slowly delaying the click so the animal has to do the behavior for longer before it gets the click and treat. There are lots of additional details and tips that can make this successful and if you are new to building duration, you might want to read the article on building duration on my website.

While the method I described above is one I still use at times, I’ve also learned some other ways that can make building duration easier for certain types of behaviors. One of my favorite ones is to increase duration by using a short chain of two behaviors.  The first behavior in the chain is the one that I would like to get more of (increase the duration) and the second behavior is what Kay Laurence calls the “terminal behavior.” It’s the behavior that tells the animal when it is done with the first behavior. You can also do this with more than two behaviors, but your chain will always include the duration behavior and the terminal behavior.

I like this method because it means that I can set up the training so the animal learns a predictable end to the behavior, and this often works better than having it be the more open ended “do it until I click.” It also takes advantage of the power of backchaining which is when you build a chain by starting with the last behavior and lengthen the chain by adding new behaviors to the beginning.

I first started playing around with this idea when I was teaching young horses to pick up and then continue to hold up their feet. Since they have often gotten a lot of reinforcement for picking the foot up and immediately putting it back down, it’s a big criteria shift to ask them to hold it up for longer. I always felt like they were a bit confused about why they didn’t get clicked for picking it up, and also when they did hold it up, they were waiting anxiously for the click. Since relaxation is important when handling feet, I started looking for ways to give them information about what I was doing so they knew when they were getting close to the end of the behavior.

Why does this matter? I think it matters because it changes the horse’s focus. I think of it as creating a “forward thinking” horse so that instead of the horse thinking “Am I done? Am I done? Am I done?” while it is doing the behavior, and looking for any sign it might be done, the horse is waiting for, or moving toward the next behavior. It might seem like this is the same as waiting for the click, but in practice I find that there is often a difference in the emotional state of a horse waiting for a click vs. a horse waiting for the next behavior.

An additional advantage is that using a terminal behavior can make the behavior clearer to the horse by providing a concrete end point. I have always felt that Rosie does better when she can see the point of what we are doing or there is a clear goal behavior.   I have also found that horses will usually try and look for information that predicts when they are getting close to the click. By choosing and deliberately using a terminal behavior, they are less likely to start to use other behaviors (that I might be unaware they are using) and try to end the behavior early because they are using information that I didn’t intend for them to use.   Not every advantage applies to every use of this technique, but hopefully you’ll get a feel for how and when to use it as I share some different examples.

Back to hoof handling. It turns out that I am pretty systematic about how I teach hoof handling and once I started thinking about it, I realized that I was already set up to create a little chain to build duration for holding the foot up. My usual progression is:

  • Can you pick up your foot and hold it up for 1 second?
  • Can you pick up your foot and hold it up for 2 seconds?
  • Can you pick up your foot and hold it while I brush it with my hand?
  • Can you pick up your foot and hold it up while I brush it with my hand for longer? (I do a few repetitions of this until I have enough duration to match one stroke with the hoof pick)
  • Can you let me use the hoof pick once?
  • Can you let me use the hoof pick twice? And so on… until I can pick out the entire hoof before putting it down.

Normally I progress through these steps, but what if I continued to do the same behavior right before I put the foot back down? Would that provide some consistency so that the behavior was “hold up your foot until I do this behavior” and not “hold up your foot for a variable amount of time.” So I tried it using the behavior of “brushing the foot with my hand” and it worked very well. Even once I was up to cleaning the entire foot, I always finished by brushing it with my hand before I put it back down. It looked like this:

  • Can you pick your foot up hold it while I brush it with my hand?
  • Can you pick your foot up and hold it for one swipe of the hoofpick followed by a brush with my hand?
  • Can you pick your foot up and hold it up for two swipes with the hoofpick followed by a brush with my hand?

The horses learned that I might do various thing with their feet but that I always ended with brushing the foot with my hand. I found this made them calmer while I was doing the other hoof behaviors because they were not waiting with such anticipation for the click. This made it easier to build duration by slowly increasing the number of things I did before I brushed the hoof with my hand. I chose to use brushing the hoof with my hand because it’s easy to do, regardless of why I have the foot up, and it’s the first thing I usually do with babies, but any simple behavior that includes holding the foot quietly will work as well.

Now one obvious question would be “Isn’t putting the foot back down a good terminal behavior?” and in a more advanced horse, it could function as one. But what I have found is that when using a terminal behavior as part of the teaching process, it is important to choose a terminal behavior that I can control. By control, I just mean that I can decide when to ask for the behavior or set up the environment so that the behavior happens when the horse has met the desired criteria for duration. Yes, the horse can stop or cease doing the behavior at any time, so this is not about making the behavior happen, but if he wants to get clicked and reinforced, he has to complete the chain with no shortcuts. It’s worth remembering that the downside of chains is that animals will rush through them to get to the end, so if I want to build duration using a chain, I need to set it up so that it’s clear when the criteria for duration has been met and that there is no reinforcement for jumping to the end of the chain.

This might be clearer with another example of how to use a short chain to build duration in movement. Here’s one way to increase duration at the walk by adding the terminal behavior of going to a mat, target, or other destination.

Let’s say I have trained my horse to walk forward on cue and he will walk for a few steps (3) before I click. Now I would like him to walk forward for more steps, building up to the ability to send him forward and having him keep walking until he gets another cue. I can do this by setting it up so that he is doing a short chain: walk forward -> stop at destination (mat, target, specific location). The destination should be indicated by a behavior the horse already knows so if my horse doesn’t know matwork or targeting, I need to teach that first.

I start by placing the destination object, or positioning the horse, so he has to walk the number of steps he already knows (3, in this example) to arrive there. Once he has learned to do this, I can start to build more duration for walking forward by changing the distance to the destination. I still want to follow the general guidelines for building duration by increasing the distance slowly and maintaining some variability so the horse has some easy reps mixed in with some harder reps, but it’s easy to do this because I get to decide how to set up each repetition. Here’s an example using walking forward to a mat. It might look like this:

  • walk 3 steps to mat – click/treat
  • walk 4 steps to mat – click/treat
  • Walk 6 steps to mat – click/treat
  • walk 4 steps to mat – click/treat
  • walk 5 steps to mat – click/treat
  • walk 7 steps to mat – click/treat

If my goal is to fade out the mat, at some point I can just start clicking before the horse gets to the mat and he will learn that he is getting clicked for walking forward and not necessarily for going to the mat. I can do the same thing with targets or specific locations. I’ve done a lot of work where Rosie gets clicked for arriving at a corner.

Once you get the idea of using short chains, it becomes easy to think of ways to use terminal behaviors to improve preceding behaviors. Some examples of other ways to use them to build duration are:

    • Backing – In the same way that I can use a destination to build duration for walking forward, I can use a destination for walking backwards. I’ve played with using targets and mats to help a horse learn to back for more steps or with better mechanics (diagonal pairs, relaxed topline).
    • Any other movement behaviors – Giving a horse a destination can be incorporated into a lot of ground and ridden work to make it easier to build duration.
    • Husbandry behaviors – A lot of husbandry behaviors require the horse to stand or hold still while I do something. If the horse starts to anticipate the end of the behavior and end it too early, it can be helpful to add a specific terminal behavior that teaches the horse to maintain the behavior for slightly longer.
    • Leg lifts – I’ve used a touch as a terminal behavior for leg lifts. The horse learns to hold his leg up until I touch it. This has the nice advantage that the touch can also be used to reinforce the leg being in the desired position because touching the leg when it is in a certain position also reinforces that variation on the behavior.

General tips:

    • Choose a simple, easy, and short behavior that you can use to mark the end of the behavior. It can be a change in your behavior (something you do to the horse), or the environment, or something you set up as a goal behavior for the horse. I like to think of the terminal behavior as the period at the end of the sentence.
    • Consider whether or not you will want to continue to use the terminal behavior as part of the final behavior. If you just want to use it as part of the teaching process, then you need to plan how you will fade it out.
    • Using a terminal behavior does not mean you never click for the duration behavior.  You can (and probably should) sometimes click and reinforce the duration behavior at times. This is because some chains can start to break down if individual elements are never clicked and reinforced. I usually start by clicking the duration behavior to get it started, then I add a terminal behavior, and start working toward my goal duration. At various points along the way, I may choose to click for the duration behavior instead of cueing the terminal behavior. I tend to do this either to confirm that the duration behavior is the right answer or if I want to introduce the idea that I might not always ask for the terminal behavior.
    • Remember to choose a terminal behavior that cannot be offered prematurely by the horse.  One of the downsides of chaining is that animals tend to rush to the end of the chain if they think they can. I try to think carefully about how to set up the training so the horse can’t take a shortcut and expect to get reinforced.

I’ve put together a little video that shows the use of short chains to build duration. You can watch it at: https://www.youtube.com/watch?v=3mjLuUpWijw&feature=youtu.be. Please ask questions if you have them. Here is a brief description of what we were doing:


Foot pick up and hold. Aurora was learning to hold her feet up for longer. I am showing how I use the behavior of brushing the bottom of her hoof before I put it back down.


Backing to a mat. Red has several cues to walk backwards, but over time his default response to his back cue had become to back 2 steps and then stop. This was often all I needed around the barn, so he lost the idea of backing for more steps. Since I didn’t want to build duration by withholding the click if he only backed two steps, I set up a new exercise using a mat so that he learned to back for more steps as I increased the distance to the mat.

I have to say that when I started this project, I underestimated how hard it was for him to back 5 steps in a straight line. So, in addition to getting 5 steps of backing, we spent a lot of time working on symmetrical and even steps. This was great for him as it helped him work through some crookedness in his body, but it did make the behavior more complicated to teach. You’ll see me checking the position of, and resetting the mat because I wanted to make sure it was directly behind him so he would land on it only if he backed in a straight line. If I hadn’t wanted to work on backing in a straight line, I could have used a wider mat.

I also want to note that I wanted a square halt on the mat since this is a behavior he knows and this exercise was part of set of related exercises that were focused on straightness and stopping in balance. If you are just using the mat to mark the end of backing, there’s no need to be that particular about how the horse lands on the mat.


Backing while I’m sitting in a chair. This was just a fun project that came out of the backing to the mat. I wanted to see if he really understood that the back cue meant to keep going and if he would do it without me accompanying him (I usually walk toward him as he backs).   I taught this by slowly shifting the position of the chair (relative to him) so he learned to back up until his nose was even with the chair. I did add a pole to help him stay straight since my position makes him inclined to want to orient toward me. When I have time to play with this again, I could fade out the pole if that was important to me.

Using Targeting to Build Confidence When Riding Outside the Ring

dsc00371Do you have a horse that is not very confident riding out alone? If so, then this story about Rosie may be of interest to you. It’s about how I used clicker training to build her confidence and give her a way to communicate with me so we could ride out together.

My main riding horse is my mare Rosie. We do a lot of ring work, but I also like to ride out in the fields and woods around our house. I think it’s good to have some variety and for her to learn to be ridden in different places.

When she was young, I started riding her out of the ring by slowly adding new areas to our normal routine. I often introduced these areas by hand walking her in them a few times before riding there. On our property it’s pretty easy to set up larger and larger “loops” so I started by riding down to the house from the barn, and then down and around the house (out of sight of the barn) and then around the house and into the front field.   I mixed things up a bit until she was comfortable being ridden around on our property in lots of different ways.   I say “comfortable,” but of course, there was some variation, depending upon what was going on, the weather, her mood etc.

However, for the most part, I felt she was relaxed and interested being out and about. I wanted riding out of the ring to be a positive experience so I incorporated some familiar exercises or behaviors like going to a cone or mat, putting out buckets with treats, and allowing her to stop and graze. Sometimes I could allow her to choose which way we would go. When I could, I rode with another horse so she had company, and on those rides we would often go a little farther than normal. It’s great for young horses to go out in company as they have the more experienced horses for support and as role models.

I continued to ride her out in the fields and trails near my house for a few years, but one summer she was badly frightened by a group of mountain bikers and after that she became very anxious on the trails. She has always had the habit of stopping and freezing when something catches her attention and now she did this for longer periods of time and also started spooking more. After a while, it became clear to me that I needed to rebuild her confidence and I stopped riding her out of the ring because neither of us was enjoying it. I did continue to ride her in the yard around my house, but that was the farthest we went.

After about 6 months, I decided to try again. That was 3 or 4 years ago and while we have not gotten back to riding out as much as we did at one point, we have worked out a way that she can be ridden out and we can both have a nice ride. The first step was to go back and start again from scratch, paying more attention to her level of tension and expanding her comfort zone much more slowly. I also made a few other changes and one was that I started riding with a target stick. I am sure there are lots of ways to use a target stick while riding but I thought I would share how I use it because it ended up being such a nice way for her to communicate with me.

I started carrying a target stick because I wanted to be able to cue a well-known behavior while riding, one that she could do even if she was a little stressed and one that was not dependent upon location, or that required me to put objects out ahead of time. Carrying a target stick seemed like a good idea because she loves targeting, it’s easy to do from the saddle, and I can present the cue and she can touch it when she is ready.

The last point is important because it means I don’t have to worry about what to do if she doesn’t respond to the cue. I’m always a little cautious about using a cued behavior in a situation where an animal is distracted or upset. I don’t want to use the cue if the animal is not likely to respond, but sometimes giving a cue reminds the animal that there are other options and can be an important step in teaching the animal that there are alternative behaviors.   With the target stick, I can present it and Rosie can respond when she is ready. I didn’t realize it when I started using the target stick, but giving her this ability to tell me when she was ready was probably the most important thing I could have done for her.

Just riding around my property, her big issues were that she would stop and freeze or walk very slowly. When I started riding with the target stick, I started out by just having her stop and touch the target at various times so she knew I was carrying it and that this was a behavior she could do while I was riding. My initial thought was that I would use it to get her attention when she froze and looked at things.

But I soon found that if asked for her to touch the target before she was ready, she would ignore it, or she would touch it and then snatch her head away or refuse to take the treat.   It was better to wait until she oriented back toward me a little bit or to present it so she could see it out of the corner of her eye, so she knew it was there, but it was more of a suggestion than a request.

What this means is that when she stops and looks at something, she can tell me when she is ready to go forward by touching the target stick. She touches the target, I click and treat, and then she walks forward or I cue forward. I’d like to say that this was all planned out in advance, but the reality is that this is what evolved and worked for us. One reason I wanted to share how we use the target stick is to show that sometimes training is about starting with an idea and seeing where it takes you. You don’t always have to have all the answers when you get started.

This use of targeting sounds simple, but along the way I did have to make some rules. One is that touching the target stick is always followed by going forward. This means I do not ask her for, or allow her to do, multiple touches. She touches it once and then she either walks off on her own, responds to my cue to walk off, or we stand and wait until she’s ready to walk forward.   This doesn’t mean she has to walk far. In the beginning, I was happy with one step and would click for it. Then we might do a few steps before I clicked. Usually once she was past the “scary spot,” she would go back to walking nicely for longer periods. Now that we have been doing this for a while, she usually walks off nicely after her target touch and treat.

Do I ever break this rule? Yes, of course. Sometimes she seems ok and will target, but then something else happens (the bush wiggles) and she has to look again. So I try to be aware of times when she is still concerned and compromise a bit. So there is definitely a component of reading her body language, but in general, I have found it works better if she does move forward between target touches. In some cases, I do end up waiting until she is ready to walk off instead of cueing it, but I don’t offer the target again until she has moved at least a step or two.

I also discovered that she sometimes associates targeting with specific locations. So if something startles her in one location on our ride, she will want to stop and target the next time she approaches that spot, even if there is nothing there. I don’t really have a problem with doing this once or twice, but I will sometimes ask her to go a little farther if she stops for no obvious reason and it’s a location where she has stopped before. Usually once she realizes that the location is not a cue to target, she’s fine the next time.

This summer we had some really nice rides out in the fields near my house.  She walked along at a nice pace and seemed to enjoy being out and seeing what was going on.  I also noticed that she wanted to stop and graze at more places along the way, which is something she doesn’t do when she is anxious.  I think that with time she will become even more confident and we will be able to go for longer rides again. But even if we don’t, I’m happy that we now have more options for what to do when it’s a nice day and we want to go riding.

About the picture:  I don’t have a picture of her touching her target stick on the trail so I have used one of her meeting the Easter Bunny.  Teaching horses to target novel objects is another way to build confidence and was something I did with her when she was younger and we were walking around exploring new areas.


Aurora gets a bath: Some tips on how to train your horse for husbandry behaviors


Using clicker training for husbandry and medical procedures is a great way to teach your horse to accept some of the routine, but less fun, aspects of handling as well as to prepare her for procedures that might involve some discomfort.

We recently had a hot spell and I decided to spend a little time with Aurora in the wash stall, continuing with some work with the hose and bathing that I started last year. When I got her in the fall of 2014 (she was a weanling), I knew she had some previous experience with hoses and bathing and didn’t particularly like it. I learned this on the day I bought her, when her breeder insisted on hosing off her muddy legs. She said she didn’t want to send me off with a dirty horse. Aurora did stand for it, but her posture indicated that she was very uncomfortable with the whole process and I had to intervene and say a little mud was not a problem.

I put hosing on my list of things to do, but since it was November, I didn’t plan on starting right away. Then, a few weeks after she got here, she cut her leg and I had to hose it off daily for about a week.   That didn’t exactly fit in with my plan which included being able to do some careful training to prepare her, but we did get through it. And, on the plus side, I did discover that she would stand still (mostly) as long as I fed her treats. So I managed to make it a more positive experience by keeping her busy eating and taking breaks as often as I could.

Last summer, I did a little introductory work to see how she really felt about the hose and getting wet. I waited to do this until she was comfortable in my wash stall. I had spent the previous winter getting her used to grooming in her stall and then in other locations, working up to standing in the wash stall for grooming. While I do a certain amount of work at liberty, I also like to teach my horses that we sometimes work in a halter and lead and to be tied in different locations and configurations (cross-ties as well as single tie).  In many cases I can use a target or mat to ask the horse to stand still, but I do like to have the option of limiting the horse’s movement if I think it will make a procedure easier for both of us.

Working at liberty vs. using a halter, lead rope, being tied etc…

This question of whether or not go use any form of restraint in husbandry work is one that comes up a lot. A lot of trainers like to work completely at liberty (horse loose and wearing no halter) so the animal has the choice of leaving the session if they are uncomfortable. I think this is important, but there are many things that have to be considered when choosing whether or not use restraint. And there are many levels of restraint.

I have chosen to use the word “restraint” here because it does seem to be the most common way of describing training with some kind of equipment that limits an animal’s choices. But I am using it with some reservations because it implies the horse is being held against her will and I don’t think that has to be the case.   Horses can learn that the presence of certain pieces of equipment are good indicators of what kind of behavior will be reinforced and while the same pieces of equipment can certainly be used in a very restrictive or punishing way, that does not always have to be the case.

For me, restraint rarely means anything more than choosing to put a halter (with or without lead) on a horse or tying her. I do use cross-ties, but I have very stretchy ones and they allow a certain amount of forward and back movement, as well as freedom in the head and neck. My horses can reach back and bite at flies on their shoulders and backs while they are on them. Some cross-ties are overly restrictive and I don’t like ones that hold a horse in a very rigid position

Regardless of what equipment I am using, my goal is for the lead rope or tie to remain loose during the training process. I am not using them to prevent the horse from moving, but more to indicate that I would like the horse to stand still. If I am training with a halter and lead and the horse starts reacting so she is putting pressure on the lead, then I need to re-evaluate my training plan. This may mean breaking the behavior down into smaller steps, removing or changing the equipment, location, or some other element of my training plan.

There are some horses that shut down (offer no behavior and seem depressed) when they are haltered or tied because of past experiences, and this should be taken into account. They may benefit from doing a lot of work with complete freedom as well as making new positive associations with halters and other equipment that needs to be used for routine handling.

I don’t have set rules for whether or not I put a halter and/or lead rope on a horse. It depends upon each individual horse, what I am training, and where I am in the process. I find that I often go back and forth between the different variations. I think it’s fine to start with a halter and lead and work toward doing the same behavior at liberty (if appropriate) and it’s also fine to start at liberty, teach the behavior and then show the horse they have choices even when they are wearing their halter, on a lead, or tied.

Here are some things I consider when choosing whether or not to use restraint:

  • How does the horse feel about restraint in general? Some horses panic if they feel trapped and it’s important to be very careful so that no one gets hurt. My horses all know how to tie, but I have one that panics a bit if something unexpected happens. I never tie her up for anything where I think she might move quickly.
  • How much time do I have to train it? If I am dealing with something unexpected and my horse has not been trained to handle it, I will put the horse on a lead rope or tie it.   If I have a lot of time to train a behavior, I may choose to train the entire thing at liberty.
  • What is my training space like? Is the space one where I can safely work at liberty? Or is it safer if I limit my horse’s movement? It’s good to have a few options so that the horse can learn behaviors under different conditions. I have spaces where the horses are used to being loose, being tied, or put on cross-ties.
  • How do I do the rest of my training? If I do most of my training at liberty, then it makes more sense to do husbandry behaviors at liberty too. The same goes for working on a lead. If I do most of my work on a lead rope, then I think it’s fine to do husbandry behaviors with a lead rope too. I don’t want my horse to think there is anything different about husbandry behaviors. I’m not going to say that a horse can learn to enjoy standing for a shot as much as it enjoys pushing a ball around, but the more similar I can make the training for both, the less likely it is that the horse will perceive shot training as “bad” and ball training as “good.”
  • Is the procedure one where it is ok if the horse moves? With some husbandry behaviors, it doesn’t matter if the horse leaves before I am done or moves around a little bit. With others (particularly some medical procedures), the horse does need to stand still for a period of time.
  • How important is the timing of my click? It can be hard to click and treat with good timing if the horse is moving around a lot. The more structured I can make the procedure, the easier it is to click precisely. Using targets can help a lot with this. I can also teach a horse to accept the use of my hands to hold a body part in position. I routinely teach horses to allow me to manipulate their legs and hold them in certain positions, and I do the same thing with their heads. This can be done through a combination of teaching body part targeting and teaching the horse to accept prolonged contact. For some behaviors, I do want the horse to wait until I click or release them, so the teaching process may include moments when the horse wants to move and I ask him to wait a moment longer and then release him.
  • Do I have a helper? If I am working alone, I am more likely to use restraint. If I have a helper, I can have the helper click and treat the horse for a simple behavior like staying on a target.  Or I can click and she can treat if I am in the best position to decide when to click.

Passive Restraint vs. Active restraint

Another thing to keep in mind is that there is a difference between “active restraint” and “passive restraint.” Ken Ramirez’s book “Animal Training” has some great information on training for husbandry behaviors and it includes this article about teaching a dolphin to accept passive restraint for daily injections. This work was done at the Brookfield Zoo and the article describes why they decided to use passive restraint and how they taught it.

“After much discussion, we realized that we had two options. We could continue to catch and physically restrain Akea for the next several months or teach her to participate in a behavior called “passive restraint.” Passive can be defined as “inactive, while acted upon” and “receptive to outside influences” while restraint is defined as “to hold back from action!” This technique has been used at other facilities and with stranded animals with success, but we had not trained it in recent history at Seven Seas.”

“It was important that the staff realize that we would not be able to condition this behavior fully using only positive reinforcers, and the challenge would be to balance the reinforcement using a minimum of negative reinforcers. Precise implementation of both positive and negative reinforcers would be needed to condition the passive restraint.”

The article goes on to describe how they started with several trainers physically holding the dolphin in place, but were able to condition the dolphin to accept restraint and over time they were able to decrease the number of trainers. The dolphin eventually learned to stay in position for injections with fewer people (few enough she could have left if she wanted) and the behavior was maintained mostly with positive reinforcement. They listed the components that contributed to the success of this training. They included starting with basic restraint (having enough people that they could hold her), positive reinforcement, a companion animal, fading the aversive (they used a net to contain her at times), minimizing time (coming up with ways to make the procedure quicker) and minimizing staff.

This work was done in 1997, so it’s a slightly older article, but I think it shows how you can use restraint as part of a positive training program.  My guess would be that most zoos now have training programs in place to prepare the animals for medical procedures, but I thought it was interesting how they did it and that they got good results.  I found myself in the same situation when Aurora first cut her leg and I didn’t have any time to teach her to stand for hosing. While it would be nice to always be prepared and train behaviors before you need them, there are going to be times when your training is not completed or you have a time constraint and can’t take the amount of time you would like.

Different Approaches to Training Husbandry Behaviors

With this in mind, let’s look at some different approaches to training horses for husbandry behaviors.   I am including training strategies that require a long and carefully thought out training process as well as those that you can use in an emergency or when you have limited training time.   I find that in most cases, it is possible to make an unexpected procedure a positive experience for your horse, or at the least, you can avoid setting yourself up for future problems because you had to do something that caused discomfort, before you had time to prepare your horse.

There are different approaches to husbandry behaviors. Some of them focus on teaching the animal to be comfortable with and accept what needs to be done, others put more focus on finding ways that the animal can actively participate. It’s tempting to rate some of these approaches as being better than the others, but I think it’s more appropriate to recognize that any one of them might be the best option at the time and most of us will use each of them at one time or another. Yes, it’s fun to have a horse get excited about and actively participate in husbandry behaviors, but that may not be appropriate or possible for all horses, and all behaviors, all the time.  Teaching a horse to stand quietly for husbandry behaviors using positive reinforcement is a big step in the right direction and anyone who does it should feel pleased with their training.

Steady click and treat during the entire procedure (or just treat):

If I don’t have time to prepare the horse and something needs to be done, I can try maintaining a high rate of reinforcement for the procedure.  In this situation, I am feeding the horse while something is done to it. Some horses are better if I click and feed. Others are better if I just feed. I may have to experiment to see what works. The food is keeping the horse busy, and while the food is not necessarily being used in the optimum way for classical conditioning, the use of food usually makes the whole process more acceptable to the horse. This is easier to do with two people.

One advantages to this approach is that no previous training is required. If I can’t hand feed, I can use a bucket. When Aurora came and I had to trim her feet, I had someone feed her while I did it. This was a better option than trying to use a more operant approach with a weanling who had no understanding of the click and treat and was worried about having her feet handled. I used the same strategy with her the first time she cut her leg and I had to hose her. I fed her a steady stream of treats while I was hosing her leg.

There are a few disadvantages. It may not work in all cases, especially if the procedure is painful or the horse has to stay absolutely still for a period of time. If I am working alone, I may need to restrict the horse’s movement, either by holding the horse, tying her, or putting her in a small space. It also will not work if the horse won’t eat or the food is not enough to reinforce the behavior I want.  In addition, I have to be careful that the horse doesn’t get satiated before I am done.

The other point to keep in mind is that this approach is more about management than it is about training. A horse that learns to stand and eat during husbandry or medical procedures is not learning what it should do. The food is acting more as a distraction or a way to prevent misbehavior and if I want to be able to do the same thing with less food, I will need to start to make the reinforcement contingent upon behavior, which is the next option.  Having said that, there’s nothing wrong with doing this as a first step or until you can do some more training.

Click and treat for cooperation: the horse accepts the procedure

If I have time to prepare the horse or your horse is already familiar with clicker training, then I can take a more operant approach. This would include teaching the horse the appropriate behavior ahead of time, taking the time to desensitize the horse to any equipment and preparing them for new stimuli (feel of needle, water, etc.).  I would be looking very specifically for behavior to reinforce that was consistent with my end goal for the husbandry behavior that I was teaching.

Even under unexpected conditions, I can do a shorter version of this so the horse is introduced to equipment and any new stimuli in as gradual a way as possible. If the horse has previously been clicker trained for some husbandry or medical procedures with positive reinforcement, it is likely that he will be able to accept a new one as long as I take a little time at the beginning and make use of already trained behaviors.  A horse that has learned to target for one husbandry behavior will often be willing to target for a new one, as long as I set up similar conditions and take a little extra time at the beginning.

This approach has some advantages over the first one. The first one is that the focus is on training behavior, not on managing behavior. Because I am using the click and treat to mark specific behaviors, I can shape behavior if needed, and I can build some duration. Over a few sessions, I can educate my horse about exactly what I want him to do and he will start to offer behavior and be actively working to earn a click and treat.

Using this method, horses can be taught to accept lengthier procedures with fewer clicks and treats which avoids the problem of satiation or having the click and treat interrupt what is happening. Honestly, I find that the latter is usually only an issue if I am working alone and I have to interrupt what I am doing to feed the horse. I’ve only encountered this for things like wrapping legs or hooves where it’s hard to stop in the middle to feed the horse.

Click and treat for participation: the horse gets to make some decisions about how the procedure is done.

The difference between this approach and the previous one is the horse’s level of participation. Both approaches include taking time to desensitize the animal to any equipment, teach appropriate behaviors,  and prepare him for what needs to be done.

In the second approach, the horse is usually being trained to stand quietly while something is done to it. In this approach, the horse is trained to do behaviors that allow her to more actively participate or she can be taught how to communicate with the trainer about when sshe is ready to start, when he needs a break, or some other aspect of how the job is done. Many animals feel more comfortable when they are allowed some control over what is happening and even though we can’t always let them choose whether or not something is done to them, we can let them make some choices about some aspect of how and when it is done.

There are lots of variations on this kind of training and it goes by a lot of different names. Dr. Susan Friedman calls it “dialog training.” Alexandra Kurland calls it “cue communication.”  Jen Digate calls it using “barometer behaviors.” And Eva Bertilsson, Emelie Johnson-Vegh and Peggy Hogan call it using “start button behaviors.”

The biggest advantage to this method is that it allows for more communication between the horse and the trainer about what is happening. It’s always important for the trainer to be observing the horse for body language that indicates discomfort, but this takes it to another level by giving the horse a clear way to ask for something specific.  Some horses do much better if they are given more choices about how husbandry behaviors are done and they will participate willingly in a procedure that that would not tolerate if they felt trapped or restricted.

I hate to say there is any disadvantage to this approach as I think we should all work towards giving our horses more voice about when and how things are done to them, but I will say that it takes a little more careful planning, good observation skills, and it requires that the trainer has complete control over the session, which is not always possible with some veterinary procedures. So I think it’s something to work toward and some elements can be incorporated from the beginning, but it is a more advanced way to teach husbandry behaviors.

Going into the details of how to teach husbandry behaviors this way is beyond the scope of this article, but I’ll share a little story that gives an example.

Many years ago I had a foal who was terrified of clippers. I don’t use clippers much but I do like to introduce them in case a vet needs to clip around an injury. I started by running the clippers while he was eating, touching him with them while they were turned off, and then holding them in my hand while they were running, etc., all the standard desensitization ideas that people often use with clippers. He was ok with this, but he was very upset when I touched him with them while they were running. I was working at liberty and as soon as they made contact with his skin, he would flinch and move away.

So I taught him that he could turn them off by touching them. I would turn them on and ask him to target them. If he touched them, I clicked, turned them off and treated. Once he understood that, I would stand next to him, touch him with them, click and treat. Then I would hold them out to him. If he touched them, I would turn them off and give him a break. If he turned and tried to touch them while I was working, I would offer them and let him tell me if he wanted me to turn them off. If he kept his head forward and I was able to use them for a short time (I was actually only pretending to clip), then he got clicked and treated for that. Over time he started asking to touch them less and less and I could keep them running for longer. Since I didn’t actually need to clip him I never built up a lot of duration, but I clearly remember how much more comfortable he became with them once he learned he could ask me to turn them off.

Husbandry Training using Aurora’s bath as an example

I am going to go through the steps I used to introduce Aurora to the hose, related equipment, and getting wet because the general process is one that can be used for any husbandry behavior and it’s a good example of how you can break a behavior down into many small steps.   I have found that taking the time to go very slowly, even if the horse seems ok with it, pays off in the long run. It takes much longer if you move too fast and the horse gets worried, because then you have to take a step back and rebuild the behavior again.

Aurora’s Bath Plan

I followed the tips in Ken Ramirez’s book “Animal Training.”

“The important steps are:

  • Plan Carefully
  • Progress slowly
  • Desensitize all stimuli
  • Bridge precisely
  • Maintain trust
  • Apply proven operant techniques”

Plan carefully, Progress slowly, desensitize all stimuli, maintain trust

I wrote out a general training progression and then broke it down into steps for each session. Here’s my general plan and some notes about how it worked out, or any changes I needed to make:

  • Let her see other horses being hosed or bathed. Her stall is across from the wash stall so this was easy. I just made a point of using the hose to clean or spray off one of the other horses every few days so she could see how the other horse reacted. This isn’t necessary and is not always possible, but horses do learn from watching each other.  Seeing her friends have baths and being exposed to some of the sounds of the bath from a distance was an easy way to introduce the idea to her.
  • Desensitize her to all possible noise and movement. This includes the sound of the water turning on and off and the sound of the spray nozzle. My spray nozzle can make some odd sounds so I spent time spraying the floor, spraying into a bucket and spraying my hand. I have had horses that were upset by the spray nozzle in the past, so if she had seemed concerned I would have just removed it and added it back in later when she was comfortable with the rest of the process.I spent time moving the hose around, so she saw it going in front of her and behind her. I also lifted it up and down and just moved it around on the floor. I wiped her with a dry sponge and scrubby so she got used to the feel of them on her body. I did all this over a period of weeks a little bit at a time. I could have done it much quicker but I would still have wanted to give her some time between sessions so I didn’t try to expose her to too many things too quickly.
  • Start with the part of her body where I think she will be most comfortable. I always start with water that is lukewarm or slightly warm, even if it’s a hot day. With many horses I start by gently hosing the shoulder area. They seem less reactive to water there and it’s easy to stand in that location and click and treat without having to move the hose around.However, with her, I decided to start by hosing off her hooves.   This might not be a good choice for some horses because a lot of horses move when they feel water on their legs, but I have done a lot of work handling her legs and she is very comfortable with me working around them. She also had some previous experience and reinforcement history for standing while I hosed her legs.

    One advantage to hosing her hooves was that I could introduce the sensation of water on her skin very gradually. So I sprayed her hooves and a little water misted up on her lower legs. Then each day I went a little higher.   I found it was easy to move from foot to foot and I would spray one foot for a few seconds and then do another one. This allowed me to slowly build some duration without staying in one spot and made it easier for me to click and treat without accidentally clicking when she moved.

  • My plan included an option for what to do if she was reactive to water coming from the hose on to her body. I have worked with a few horses that were uncomfortable being sprayed directly by the hose. One option is to remove the spray nozzle and just set the hose on a slow drip. Once the horse is comfortable with this, I can use my thumb over the end to create a little bit of a misting effect and slowly build up to using a spray nozzle.Another option is to use a sponge as an intermediate step. When I do this, I use the following progression:

    Can I touch you with the dry sponge?
    Can I touch you with the wet sponge?
    Can I touch you with the wet sponge while holding the hose?
    Can I touch you with the wet sponge while the water from the hose is going on to the sponge?
    Can I touch you with the water from the hose going over the sponge so some is hitting your body directly?
    Can I touch you with the water from the hose going over my hand (no sponge) so some is hitting your body directly?
    Can I touch you with my hand and water from the hose?
    Can I spray you directly with the hose?

    This progression prepares the horse for the sensation of the water hitting her skin directly. I think many horses react because they are surprised by the water coming from a distance. They are more comfortable when I use the sponge and my hand because that tells them where the water will touch them.

  • Work in short sessions, focusing on one new element each day. She is general pretty accepting of new things so I had to make myself keep the sessions short and not assume that acceptance was the same as really being ok with it. I find she’s sometimes harder to read than a more reactive horse and I have to make myself go slow, even though it may not look like she needs it. I think she does, she just doesn’t make it very obvious. I was able to move pretty quickly through the desensitization steps, but made a point of only using the hose on her for a minute or two in the first few sessions.There has been some research, looking at physical indications like heart rate and breathing, that shows that a quiet horse is not necessarily less stressed than a horse that is reacting to a procedure. This is more likely to be true with horses that have been punished in the past, but I think you have to be careful with any horse to make sure that you are not asking too much of them. Learn to read your horse’s body language and allow her to let you know how she feels about what you are doing.
  • Look for opportunities to make it more fun for her. Some horses like to put their noses in the hose or enjoy being hosed in certain areas. My gelding Willy always liked having the underside of his neck hosed off and he would stretch his neck out and try to rub against my hand when I hosed him there. She seems interested in the water so I sometimes hold the spray nozzle near her nose and let the water run into my hand and she will stick her nose in it. Another option is to mix in other favorite behaviors such as targeting or interactions with objects.

Bridge Precisely

The plan covers most of Ken’s steps (plan carefully, progress slowly, desensitize all stimuli, maintain trust) in detail. It also mentions the importance of clicking at the right moment. One of the challenges of training husbandry behaviors is that a poorly timed click can set the training back considerably, especially if it happens in the early stages of training. I find that clicker trained horses like to be clicked for doing things, so a mis-timed click, that happens during movement, can often lead to the horse offering more movement.  One way to make this less likely to happen is to try and set up the training so that the clickable moments are long enough or for previously learned behaviors so that they are easy to mark.

There are a couple of strategies I can use to help with this:

  • Use a target – If the horse has been trained to hold on a target, that gives both of us a focus and makes it easy to decide when to click. A nose target can be useful if I want the horse to hold his head still. A foot target might be a better choice if I want the feet to stand still. I put Aurora on a mat for hosing to encourage her to stand in one place.
  • Click and treat for standing quietly when I am doing something to the horse and also when I am not. It’s a good idea to mix in some clicks and treats for standing quietly between repetitions of the husbandry behavior or between procedures.  When I was hosing her feet, I clicked and treated when she was standing still while I was hosing. I also clicked and treated when she was standing still when I was not hosing. Those were great opportunities to click and treat for the behavior I wanted, but with less risk that she would move just as I clicked.
  • Teach the horse to voluntarily participate.   It’s easier to time the click if the horse is the one that initiates contact or movement or the horse is responsible  for maintaining contact. For example, if I train my horse to target my hand or an object with his ear or eye, it’s easier to time my click correctly because I can see the horse’s intention. If I just put my hand out toward his ear, I might not be able to tell if he is going to allow the touch or not, so there’s more risk of clicking at the wrong moment.

Apply Proven Operant Techniques

The last point I want to mention as part of the plan is Ken’s comment about using “proven operant techniques.” The operant technique I use the most is positive reinforcement (the click and treat) and it’s an integral part of my training plan. I am going to mark and reinforce Aurora for moments when she is standing still or doing other desirable behaviors. During bathing, I sometimes click for a change to a more relaxed posture. So if her head is up, I can click if she drops it to a lower position. I don’t necessarily want her to take her nose all the way to the floor, but I can encourage a drop to a more neutral position.

I can also click for standing in good balance or for interacting with me in appropriate ways. I don’t need her to stand like a statue, so I sometimes click for natural behaviors that show she is aware of what I am doing and wants to see more or check in with me. I do have to be careful that she doesn’t think these behaviors are required, so I try to respond to her but without creating any expectation or setting up a pattern. I can also mix in simple behaviors to give her a break or additional opportunities for reinforcement.

With a husbandry behavior like bathing, positive reinforcement is not the only operant technique that I am using. Negative reinforcement also comes into it. Does this mean what I am doing is aversive? No, it just means that there will be moments, especially in the beginning, where she finds it reinforcing when I stop using the hose or washing her. She may just be tired of standing while I work on one location or need to take a break. So I can time when I stop hosing to reinforce behaviors I like.

I should point out that I while I am aware that taking the hose away can also reinforce undesirable behavior (fidgeting, moving away), I don’t like to leave it on if the horse is clearly uncomfortable. Waiting for a “good moment” when a horse is already stressed is not a good training strategy. It’s better to listen to what the horse has to say, remove the hose and try again with a slightly modified training plan.

In the beginning, if she stands nicely while I spray one location, I can remove the hose, click and treat. Later I can reinforce the same behavior by taking the hose away for a moment and giving her a break. I don’t necessarily have to click and treat for every area I hose. I actually find that if I am doing something with shampoo, it can be tricky to click and treat a lot during bathing as my hands get covered with soap and the horses don’t like that. So I try to build in other types of reinforcement like a scratch or pat, removal of the hose, or some praise. That way I can click and treat less often but still reinforce good behavior in between.

Final Thoughts

Most horses can be clicker trained to accept and participate in husbandry behaviors if the trainer takes time to prepare them and works at the pace at which the horse is comfortable.  Teaching your horse really strong targeting behaviors and introducing equipment gradually will make the process easier and more enjoyable for everyone.  Planning ahead and doing some training before you need it is always a good idea. I try to mix in a little practice here and there as I have time so that it’s not a big deal if I suddenly need to hose a leg, put in eye medication, or wrap a foot.

A lot of the information I have shared in this article can be used for other husbandry behaviors. The key to success is breaking it down into tiny steps and observing your horse carefully to make sure that she is comfortable with what you are doing.   Since Aurora is only 2, I’ve been doing a lot of husbandry behaviors with her and I hope to share more tips as we work on each one.

Superstitious Behavior

black  cat leavesLast Friday was Friday the 13th. I don’t consider myself a superstitious person, but I do have to admit that I am unlikely to do anything risky on Friday the 13th. I mean, why take a chance? Interestingly enough, on last Friday I was feeling a bit perplexed over a problem  with one of my training projects and it had to do with a “superstitious behavior” that had crept in and was proving to be a little difficult to remove.


What is a superstitious behavior?

In psychology, a superstitious behavior is a behavior that an individual does because he thinks it is associated with reinforcement, but in reality it has no direct effect on whether he will be reinforced or not.

There are lots of examples of superstitious behavior in people including things like wearing your lucky socks, buying lottery tickets at the store where you won before, and rituals associated with games or competitions.

In animals, superstitious behaviors are those behaviors that have become part of the animal’s repertoire (or have increased in frequency) because they are associated with other behaviors that lead to reinforcement. The animal doesn’t actually need to do the superstitious behavior to earn reinforcement, but it doesn’t know that, so it continues to do it, either as a separate behavior or in combination with other behaviors that do lead to reinforcement.

In the laboratory:

The term “superstitious behavior” was coined by B.F. Skinner who noticed that pigeons would end up performing a specific behavior, even under conditions where reinforcement was delivered at random.   The behavior was selected and then further shaped because the pigeon just happened to be doing it when the random reinforcement was delivered. I would assume that the selection of a behavior happens quite soon after reinforcement starts, but I can’t find any documentation on this, it just makes sense to me. Anyway, the reinforcement made the pigeon more likely to repeat the behavior, which increased the probability that it was doing that behavior when reinforcement was delivered, and so on, until that behavior had been reinforced so strongly that the pigeon consistently repeated it.

Iver Iversen talked about superstitious behaviors in his talk, at the Art and Science of Animal Training conference in 2016, when he explained that as soon as reinforcement is available, some behavior will be selected. He had a video example of a superstitious behavior that a rat “learned” when it was put in a Skinner box which he had set up so that food was delivered on a random basis. When he came back later, the rat was doing a full roll between reinforcements. This behavior had been shaped by the random delivery of food and the rat continued to do it, even though it could have done nothing and food would still have been delivered.

This same type of experiment has been done with people and produced similar results.  In “Learning and Behavior,” Paul Chance describes several experiments. In one, thhe experimenters put an individual in a room with a device that would intermittently provide reinforcement.  It was set up to reinforce at random intervals, but the person was not told that information. In most cases, the person would start trying to figure out what he had to do to be reinforced. He would come up with some behavior, start to repeat it, it would get reinforced, and he would repeat it more.  This pattern would continue, with every reinforcement confirming his belief that he had found the behavior that led to reinforcement.  The subjects in the experiment were sure they had figured out what they had to do to get reinforced and could not believe the reinforcement had been totally random.

In real life and training:

What happens in the laboratory is not always a good predictor of what will happen in real life, but it turns out that animals do learn superstitious behaviors in real life and during training sessions, and this can happen for a few reasons.

  • In real life, it can happen for the same reasons it happens in the laboratory where random reinforcement ends up selecting for a specific behavior.
  • In training, it can happen because an animal can be doing several behaviors at once or there can be overlap between behaviors. Behavior rarely happens in discrete little units and while the clicker(or other marker) allows the trainer to mark a specific moment, there can still be several behaviors happening in that exact moment.
  • It can also happen because animals will often connect together several unrelated behaviors (do this and then do that) or fall back on previous associations when they are looking for patterns that lead to reinforcement.  So even though the trainer is trying to select out one behavior, another one can “tag” along.

This why it’s important to set up your training so that you can isolate out individual behaviors and to be watching for unwanted connections that the animal might be making.

While there are similarities between superstitious behavior and unwanted behaviors that have become attached to other behaviors, I don’t think it’s accurate to say that any behavior that contains both wanted and unwanted components contains superstitious behavior.  Sometimes animals offer the “wrong behavior” because of bad timing, cueing issues, or other trainer errors.  But these incorrect responses will disappear if they are not reinforced.   A superstitious behavior is a little different because it is one that was never intentionally reinforced and the animal continues to offer it over a long period of time, even though it may never receive any direct reinforcement.

My personal thought is that, like in people, animals continue to do superstitious behaviors because they think they work and the difference between an unwanted behavior and a superstitious behavior is the animal’s own belief that this behavior is part of what it needs to do to earn reinforcement.  Of course, I have no way to know what exactly what an animal is thinking, but I have found that animals can be very committed to superstitious behaviors, and changing them can take quite a bit of effort.

Here are two examples of superstitious behavior that come from my own training.   In one case the superstitious behavior was reinforced as part of a chain. In the other case, it is not as clear where it came from, but it became associated with the “hind feet on the mat” behavior even though it was never deliberately reinforced.

An example: Willy learns to nicker on cue

My first encounter with a superstitious behavior was many years ago when I wanted to teach my older horse Willy to nicker on cue. I knew that if I went into the barn at certain times of the day, he was more likely to nicker.   I started by reinforcing him for nickering and he did start to nicker more often, but it was not predictable enough that I could add a cue. So I decided it was important to get a few repetitions in a row.

What I had been doing was clicking and treating him for the nicker when I walked in the barn, and then continuing with whatever I was out there to do. My idea was that if I walked out of the barn after reinforcing him for nickering, and then came back in, I could do several repetitions in a row and then I could add a new cue because the current cue (walking into the barn) was not terribly useful.  And, even though he was nickering more often than when I had started reinforcing the nickering, I was not sure he really knew what was being reinforced.  I thought clicking and treating the same behavior a few times in a row would have the additional benefit of helping him understand that it was the nickering that was being reinforced.

Guess what? It worked great. I got him to repeat the nickering a few times in a row and after a few days, I added a cue. But… at that point I realized I had a problem. When I cued the nicker a few times in a row (while standing in front of him), he would walk in a circle in his stall, then come to the door and nicker. It turns out that when I was teaching the behavior and left the barn between each repetition, he would make a little circle and then when I came back in, he would nicker. So the behavior I taught was “walk in circle -> nicker,” not just “nicker.” He thought that walking in a circle was part of the behavior, so even when I stood in front of him and cued him several times, he continued to circle  before each nicker.

I learned a few things from this. One was that it’s really a good idea to keep your horse within view while training (ha!) and the other was how easily behaviors can get linked together so that you get the one you want, but you also get some other behaviors coming along for the ride.

Another example: Red and the resting hind leg

My more recent challenge with superstitious behavior happened for a totally different reason. A few summers ago, I taught Red to back up on to a mat. He had already done some hind foot targeting on the mat, but I had taught this by walking him forward on to the mat. Now I wanted to train him to back up on to it.  Prior to this, I had taught this behavior to other horses by asking them to back and clicking for the hind feet hitting the mat. Usually I start by clicking for backing up and then I add the criteria of backing until they step on the mat and over time I add distance, if I want.

In his case, I had started as usual by clicking for backing and when he got to the mat, I would click and reinforce him in place a few times.  But I had been doing some reading about backchaining and how important it was to practice the final behaviors in a chain, as well as to repeat a behavior to build muscle memory.  Backing to a mat is actually a chain of backing steps followed by a halt. He already knew the halt on the mat so I decided to put the mat directly behind him and click and treat for a single foot stepping back on to the mat. If he stepped on to the mat, I clicked and treated and then took him one step forward off the mat. So I had one hind leg stepping on and off the mat. I practiced both sides and he did great. I did also reinforce for both feet solid on the mat so I thought I was practicing the final two steps in the chain.

The next day when I brought him out and positioned him with the mat directly behind him, he stepped back and rested his foot by balancing on this toe on the mat. Hmm… ok? That was not a behavior I had clicked. The best I could guess was that he was resting his toe so he was prepared to move either way.  From that position, he could lift it up and go forward or continue to step back. Or perhaps I had reinforced the foot in that position by clicking for moving forward and back.

Either way I had not specifically clicked for a resting toe, but that was what he had learned. Now whenever I backed him on to the mat, instead of standing solidly with both feet, he would rest one. I did eventually get him to land on the mat and stand without resting a hind foot but it took quite a few sessions to eliminate the toe resting behavior. At this point I didn’t consider it a superstitious behavior, just an odd by-product of how I shaped foot movement on to the mat.

Fast forward to this spring when I have been  doing some hind foot targeting with Red and on the third or fourth session, the toe resting came back in. He had been doing a nice solid “two hind feet on the mat” behavior and then one day he started to rest the toe again.  So I had to go through the process again of cleaning up the hind foot targeting to get a solid stand.  At this point I did look at it and think “hmmm.. a superstitious behavior” because it was a behavior that he was clearly offering in hopes of reinforcement, even though it had never been reinforced directly and was not a necessary part of the final behavior.

What do you do about superstitious behaviors?

I think the challenge with decreasing the incidence of superstitious behaviors is that they are often being indirectly reinforced and it can be hard to determine why the behavior is happening. The best way to change behavior is by changing the antecedents or the consequences, so when they are both murky, well…..that makes things a little complicated.

And it seems, in many cases, that the animal is SURE that what he is doing is part of the behavior the trainer wants.  Red’s not going to give up resting that hind leg any more than someone is going to give up his lucky socks or rabbit’s foot unless I convince him that something else is a better option.

The good news is that while it might take a little more time and creativity to get rid of superstitious behaviors, I can do it using the same strategies I would use for any unwanted behavior. In some cases, especially if the superstitious behavior is occurring on its own, I  can make it less likely to happen by setting up conditions where it is not associated with any reinforcement and training an alternative behavior instead.   In other cases, it’s better to just start again and be more careful about the set-up and what is reinforced. Now that I know Red might think about resting that toe or Willy might circle, I could plan better next time.

As a final note, I have to say that superstitious behaviors are a good example of how persistent an animal can be about continuing to do a behavior if the animal believes it needs to do it to earn reinforcement. And while that might be annoying if you are dealing with an unwanted behavior, I think it says something great about how committed our horses can be if they think they know the right answer.

Notes from ClickerExpo: Kay Laurence, “Hocus Focus: Helping your dog determine what’s important.”

distractions1These notes are from a lab that was taught by Kay Laurence. I took notes on each dog and then looked for overall trends or comments that would be useful and organized them into this report.

This means my report is a little different than some of my past reports because it is more of a summary of what she shared, and not a point by point account of the material. It also means that some of the content is based on my understanding of what she was doing. If you have questions, let me know and I can ask Kay for a clarification.


There were four main things that she covered in this lab. They were:

  • Distractions vs. relevant and irrelevant details. Does it matter what we call them?
  • Who determines relevant vs. irrelevant?
  • How do we teach our dogs to focus?
  • How do we teach our dogs the difference between relevant and irrelevant details?

Distractions vs. relevant and irrelevant details. Does it matter what we call them?

Kay started by telling everyone present that they were not allowed to use the word “distraction.” The word distraction comes from traditional training and has an implied meaning that the trainer must be more important than other things in the environment. Instead she prefers to talk about relevant and irrelevant details. Relevant details are the things that we want the dog to notice. They are useful information. Irrelevant details are those details that we want the dog to recognize as not providing useful information about what to do.

Here are some examples of details that we usually want the dog to consider relevant:

  • Our cues
  • Context cues that provide useful information about how to do a behavior
  • the equipment we are using (target stick, platform, etc…)
  • The click
  • The placement of the food after the click

Here are some examples of details that we usually want the dog to consider irrelevant:

  • Our posture – it’s important to be able to cue behaviors while standing in different positions (arm and leg placement and/or movement)
  • The location of the food reserve
  • our clothing, appearance, etc…
  • presence of other people, dogs, animals, etc… in the training area
  • many stimuli that dogs normally orient to (smells, trash, etc…)

Note: I made these lists based on what we did in the lab. They did not come from Kay and are not intended to be complete, just to get you thinking.

Who determines relevant or irrelevant?

We can make decisions ahead of time about whether we want something to be relevant or not, both for a particular behavior and in a particular training session. It’s important to make a list of possible significant stimuli and ask yourself what you want the dog to do if she notices them. If there are possible stimuli that we want to make sure are irrelevant, then we need to teach that distinction as we train the behavior. To do this, we set up conditions where we introduce changes and give the dog time to learn which ones are relevant and which ones are not. Kay had some exercises that showed us to do this. I’ve described them later in this report.

So it is important to plan ahead and teach your dog about relevant/irrelevant details as part of your training. On the other hand, while we are training, unexpected things might happen and we need to recognize that in those circumstances, the dog needs to be allowed to assess the change and decide if the new stimulus is relevant or not. Some things are more relevant than other things and we have to wait for the dog to decide if something has gone from relevant to irrelevant. Do we wait for the relevance to be assessed? Yes.

One example she used here was what would you do if someone walked into the room carrying a gun? Most of us would stop what we were doing and assess the situation. If we determined that it was a Marshall or a policeman who was visiting, then we would probably go back to what we were doing. The gun went from relevant to irrelevant after our assessment.

What about if you were in the room and you couldn’t see the man with the gun very clearly, but were told he was ok. Would it matter who told you? If you don’t know someone’s assessment skills, should you trust them? I think most of us would be tempted to take a quick look ourselves, just to be sure.

It seems unfair to expect dogs to accept our assessment unless we have proven to them that we can be trusted. But we can build that trust over time.  We do that by teaching them that we can give them useful information about whether something is relevant or irrelevant.   One part of the teaching process for this involves asking for already trained behaviors under varying conditions.  Another part of it comes from introducing irrelevant details so they get to practice assessing changes in the environment.

For example, you may be out walking your dog and he sees vomit on the pavement. Most dogs are going to want to check it out. If this happens to Kay when she is walking her dog, she cues the dog to “walk on.” This is a behavior the dog knows well and is incompatible with sniffing vomit. This is preferable to saying “Leave it,” which like saying “no,” because it doesn’t tell the dog what to do. Leave it and do what? Better to just say “walk on.” Leave it is another example of a word/behavior that we have inherited from traditional dog training, but that is not the clearest way to communicate with our dogs.

Let’s leave irrelevant vs. relevant for a moment and think about the underlying skill which is learning to focus.

How do we teach focus? Using a Target stick cup to teach focus:

Focus is a learned skill. It requires a lot of mental effort so it’s important to remember that focus disappears when you get mental fatigue. Some jobs require a tight focus where you are constantly filtering out irrelevant details (working in a busy office, air traffic controller). Other jobs or activities may allow for some loss of focus without it causing significant problems. It’s important to teach your dog to focus, but you need to be aware that it takes energy and you may need to alternate or vary between tasks that require a tight focus and others that do not.

One of the ways that Kay teaches a dog to focus is by taking advantage of the dog’s natural ability to follow a scent trail. She teaches this using her own style of targeting stick, which is a wooden stick (dowel) with a small measuring cup attached to one end. She likes using the metal measuring cups with slightly squarish bottoms as that helps to keep the food in the cup during movement. The cup can be attached by tape or heat shrink tubing.

The dog is taught to follow the movement of the cup. When I first saw this a few years ago, she called it a “scent target” because the dog learns to follow the scent of the food, which is a very natural behavior for a dog. It could be considered a form of luring, but since dogs were designed to smell, she doesn’t feel it’s a problem. And she finds it is very easy to use the target cup stick to prompt natural behavior which you can reinforce and put on cue.

Here are some additional points/tips on using a target stick cup:

  • Practice with the target stick cup first (without the dog). Practice moving it without the food falling out and then practice deliberately placing the food where you want it. To feed, you lower the cup and tip it out so the food is placed on or drops a short distance to the ground.
  • Slightly wet food works well as it is less likely to fall out.
  • You need to be able to load the cup while the dog is not looking. Load it while the dog is eating.
  • Teach the dog to follow the cup in movement and while standing still.
  • Using the target stick cup maintains focus better than a regular target stick because the dog is not watching your food hand or checking in with you after getting the treat. You load the cup and put it right in front of the dog so she can start again.
  • You can elicit natural behavior by how you move the cup. For example, if you lift the cup up, the dog will stop and stand because the cup movement is similar to a bird going up and triggers standing still.
  • Teaching a dog to circle with a target stick cup is easy because the dog can follow the scent trail.
  • Do short sessions, about 10-12 treats and then take a break

She demonstrated how to introduce and start working with the target stick cup to one dog. First she got him to look at it, clicked, lowered the cup and placed the food on the floor. Then she got him following it a few steps before she clicked and reinforced him. She mixed up asking for a few different behaviors so she might click a few times for following the cup and then clicking for orienting to the cup while standing or using the cup to ask for stop. She played with movement of the cup to ask for turns and stops. Her rate of reinforcement was high enough that the dog became very interested in following the cup.

She introduced the target stick cup early in the lab to show how to teach focus and then moved into exercises showing how to teach relevant/irrelevant using the target cup stick and when cueing other behaviors.

So how do teach our dogs what is relevant and what is irrelevant?

This is really about teaching the skill to the dog: Do I pay attention or not? This is something you can set up in your training sessions by introducing new stimuli in a thoughtful manner and allowing the dog to learn to make the decision relevant/irrelevant?

She had a nice example from Simon Gadbois. One of his projects is training dogs to identify ribbon snakes by their scent instead of their appearance. Ribbon snakes look very similar to garter snakes, but they smell different, so teaching the dogs to recognize and discriminate between the scent of ribbon snakes and other snakes is a more effective way to teach the dogs to find them.

He starts by putting out two samples that have snake scent on them. Everything is identical, but the sample with the garter snake scent has been frozen, which makes it less strong. He teaches the dog to choose the ribbon snake scent, which is easier to find because it is stronger. But, over the course of the training, the sample with the garter snake scent gets stronger (or “fades in”) as the sample defrosts. After 8 hours, the scents are of equivalent strength, but because the dog initially learned that the smell of the garter snake was irrelevant, it has no confusion when the smells are of equal strength. The dogs continue to find and be reinforced for choosing the ribbon snake smell, even as the garter snake smell gets stronger.

This should give you some ideas for how we can teach dogs what is relevant and irrelevant. You want to start with something very small or at a distance, and teach the dog that it is irrelevant. Then, as the stimulus becomes stronger, the dog should assess but continue to regard the stimuli as irrelevant. Think about how you could do this with a person with a toy. Start at a distance and let the dog decide if it is relevant or irrelevant.

Exercises:  Target stick with Cup work:

The cup work contains a nice example of teaching relevant vs. irrelevant. Is left arm movement relevant?

In the first part of the session, she carefully put food in the cup while the dog was eating. But after a few clicks and treats, she started to move her left arm around a bit at random moments during the training. Once he was ignoring it, she could then start to use her left arm to put food in the cup and the dog was not distracted by it. It was a good example of first teaching him that her left hand movement did not require his attention, and then being able to use it without distracting him. This gave her a little more flexibility in the timing of when she reloaded the cup.

Is a person walking relevant?

Once the dog was following the cup, she had a person start walking slowly around the outside of his working space. Once he was ignoring the person on the outside, she had the person change her pattern so that she was sometimes walking through his workspace and the dog continued to ignore her. If the dog was distracted by the person, she had the person stop and wait until the dog was done assessing and then walk off when he focused on the cup again.

Exercises with “trained” behaviors: what is relevant?

Most of the rest of the dogs did a different exercise. She had the owner choose a behavior that the dog knew well and she structured the exercise so that there was an opportunity for the owner to change something about their behavior between food delivery and the next cue. She suggested the owner start with a minor change that was a familiar action. Pick simple things like adjusting your glasses, touching your face, fixing your shirt, etc… Then she or he could do more obvious things like picking up one foot, waving an arm, and larger movements like that. The exercise looked like this:

  • Trainer: cues “sit” or “down”
  • Dog: sits or lies down
  • Trainer: clicks and tosses food away toward the edge of the training area
  • Dog: goes to get food
  • Trainer: adds in “something new” so she is doing it as the dog is returning.
  • Dog: if dog returns as it normally would (ignores irrelevant detail), the owner cues again and repeats, doing something else the next time. If the dog returns and seems unsure, the owner stops the extra movement, waits for the dog to orient to her and cues the behavior again. Kay did mention it was better to cue the behavior than to click the dog for re-orienting to you after the hesitation.

She pointed out that the trainer’s goal is to TEACH the dog, not to TEST the dog. You really want to set it up so the dog is successful and you choose small changes so that the dog can quickly go “oh, irrelevant, what are we doing next?”

She worked with several dog/trainer pairs on this exercise and it was interesting to see how some types of movements were more distracting than others and also how some dogs would offer default behaviors if they got confused. This is not a bad thing, but indicates that there needs to be more work done on teaching the dogs about relevant and irrelevant details.

Here are a few other notes from the session:

• At the beginning of one session, she had the trainer put her treats away on a side table. This changed the dog’s focus from the trainer to the treat location. It actually allowed the trainer to be able to tell when the dog was ready to focus on her and not on the food.

• Remember you are teaching the dog what is relevant/irrelevant under these conditions. Some of this will transfer to other conditions, but you can’t count on it.

• Some dogs were confused when the trainer added hand motions that were similar to those used in food delivery. Kay said that hand cues need to be very distinct from food delivery. If you’re going to use a hand signal, it has to start from the shoulder.

• There was one little dog who was being cued to down and he was clearly using the edge of the marked area as a mat. He would down and then scoot into location. Rather than continue to cue “down,” she had the trainer use her “mat” cue since that was what he was actually doing.

• You can practice this on your own without the dog before you start your training. Think of what you want to teach the dog and plan the order (start with small easy irrelevant details) and how you will do them. This can be harder than it seems and is yet another example of the gap that often exists between understanding and doing.

• Can something that has been determined to be irrelevant now become relevant? That is one question Kay has been asking herself. If the answer is “no,” then it explains why it is better to add a new cue if the dog has learned to ignore an existing cue because it was not added at the right point or in the correct way to be meaningful.

A few reflections:

I first encountered the idea of irrelevant details (by another name) when I was learning about cues. It was not uncommon to find out that the cue I had been using was not exactly the same as the cue the horse was using. Often there were environmental components that I had not considered and when I changed them, my cue no longer “worked.”  It was quite an effort for me to learn to train so that I varied those details that I didn’t want to have attached to the cue.  But it has made a difference in my horse’s understanding of what each cue means and what information they need to consider before responding.

One aspect of relevant/irrelevant details that I think is worth consideration, especially for horse people, is how you handle novel objects in the environment. When my horses were younger, I encouraged them to stop, look and approach novel objects and see them as potential sources of reinforcement. This worked great but I found that sometimes I did want the horse to ignore the object (accept it as irrelevant) and that by encouraging interaction, I was making any new object relevant.

So over time my thinking has shifted a bit and there are times when I do allow interaction, but there are also times when I set up my training to make it clear that I do not consider the novel object relevant and neither of us need to pay attention to it.   I still allow assessment, but I try to set up my training so that I can communicate to the horse that we are going to continue what we were doing by providing clear information about what I want her to do.

Notes from ClickerExpo: Dr. Jesús Rosales-Ruiz, “The Quadrant Quandry: Clarity and Perspective on an Icon.”


Untitled-2_edited-1For all you theory geeks out there (that would include me…), and maybe a few others, here are my notes from Dr. Jesús Rosales-Ruiz and his talk titled “The Quadrant Quandry: Clarity and Perspective on an Icon.”

The quadrants…. what a topic!

I’m so glad that Dr. Rosales-Ruiz put together a presentation on the quadrants. When I first started looking more into the science behind clicker training, I found a nice neat little diagram describing the ways that behavior could be increased or decreased.  Cool!  Then I started reading more, and thinking more, and observing more, and the question of “which quadrant” became one of the more confusing, mind boggling, illuminating and tangled up things I have ever tried to sort out.

Training in real life was not neat and tidy like the diagram suggested. Often two things seemed to be happening at once and even those were sometimes subject to interpretation. Trying to understand the quadrants generated a lot of thinking and forced me to look more carefully at what I was doing.  Most traditional horse training is based on negative reinforcement. So now I had to ask myself “is this ok?” Where is the line between negative reinforcement and punishment?

At the same time I realized I couldn’t assume that I was using positive reinforcement just because I was feeding treats. I had to learn to ask myself what the horse really wanted.   Was I using food in such a way that I was trying to “cover up” other issues, or was I using it to address behavior at the level that was most important to the horse.

I spent a lot of years asking myself these kinds of questions. Using the quadrants as a reference point was helpful in that it gave me a framework for my thinking, but in the end I realized that while it was important to understand them and recognize how different kinds of reinforcement and punishment affect behavior, the most important thing to consider was how the horse felt about what I was doing and whether or not we were working together toward behaviors that would benefit both of us.

So… should you learn about the quadrants? Definitely.  Should you use them as an absolute guide?  No.  But learning about them will make you a better trainer and I think that most people need to go through a period of thinking about them in order to be able to fully understand all the ways we can change behavior.  I joke to my students that you have to spend some time learning about the quadrants so that you can move on and forget about them.  That basic understanding will be there and be useful when you are watching other trainers, explaining what you do to other people, and sometimes it can be helpful when you are faced with training challenges where things are not quite going as expected.

Here’s my summary of his talk. Do ask if you have questions or there seem to be errors. I always find it challenging to take the information he presents, process it and be able to share it with others, but it’s a good way to think through it and try to explain it to someone else.

A Little History on the Quadrants

The quadrants were first described by B.F. Skinner and used in Academia long before there was any public awareness of them. In the first descriptions of positive and negative reinforcement, the difference between positive and negative reinforcement was described as “stimulus presentation” and “stimulus withdrawal.”  The word “stimulus” referred to a change in the environment, and the negative and positive reinforcement were used to describe PROCESSES.

B.F. Skinner actually started with only two boxes, positive and negative reinforcement.  He said that punishment didn’t teach anything, but that doesn’t mean he didn’t acknowledge that aversives could be used in training.  Negative reinforcement was defined as the conditions under which “stimulus withdrawal” would increase behavior.  In that case, aversives were used as motivation to change behavior. Even with that information, he didn’t consider “stimulus presentation” an effective way to decrease behavior.

In the 1950’s, punishment shows up in the literature. Murray Sidman was doing work that showed how aversives can be a problem in training and lead to avoidance behaviors.  If you haven’t read his book “Coercion and Its Fallout,” you might want to add that to your reading list.  It’s not beach reading. I read it a few pages at a time.  He did a lot of very interesting work looking at how aversives and punishment affect behavior.   In 1956, there were experiments showing punishment does have an effect on behavior and by 1960 the diagram had been updated to include quadrants for both positive and negative punishment.

Dr. Rosales-Ruiz said that by the 1960s everyone was happy because the diagram of the quadrants was now tidy and symmetrical.   At this point, people were now talking about the quadrants as PROCEDURES.  They were no longer processes, but procedures that could be deliberately carried out in the lab to study behavior.  Now instead of labeling the diagram (along the top) with “stimulus presentation” and “stimulus removal,” they were talking about “response produces a stimulus” and “response removes or prevents a stimulus.”  Using positive reinforcement as an example, the distinction is that the view shifted from the environment determining the response; stimulus presentation -> responding increases, to the response determining whether or not the stimulus was added; response -> stimulus presentation.

More recently there has been another shift, driven partly by the increased use of the terminology by professionals outside of the field of Behavior Analysis. This shift has been to identify the quadrants based on some quality of the STIMULI.  Stimuli are now labeled as appetitive and aversive. In these diagrams, instead of identifying processes or procedures, the diagram identifies types of reinforcers/punishers and what will happen when they are added or removed.

The problem with this is that it assumes that the value of a reinforcer is fixed and the objects themselves are now given properties of being either negative or positive reinforcers.  It might be accurate to say that an object/stimulus functions as a positive reinforcer under specific conditions, but that doesn’t mean the quality of the object/stimulus is fixed and it ALWAYS functions as a positive reinforcer.  It’s important to be empirical and not decide whether or not you think something will be a reinforcer or punisher.

Premack and the Value of Reinforcers

This brings us to the work of David Premack and the relativity of reinforcers.   David Premack studied the behavior of rats (among other things) and found that he could change the relative value of different reinforcers by changing the environmental conditions under which the animal lived.  He did a series of experiments where he looked at the amount of time rats spent drinking water vs. running on an exercise wheel and manipulated the environment (limited access to water or the wheel) in different ways.

What he found was that he could use one behavior to make the other more likely to occur. This is usually referred to as the “Premack Principle” and is often used by animal trainers to increase the likelihood of less preferred behaviors by reinforcing them with the opportunity to do more preferred behaviors.  What he found was that the reinforcement relationship is reversible .

  • If water deprivation makes drinking more probably than wheel running, then the opportunity to drink will reinforce running.
  • If limited access to the wheel makes running more probable than drinking, then the opportunity to run will reinforce drinking.

This ability to reverse reinforcers has been shown with other types of animals and even with humans. Dr. Rosales-Ruiz described a study that was done with two groups of kids. The first group liked chocolate and the second group liked playing ping-pong.   The “chocolate” kids were reinforced with chocolate for playing ping-pong and the “ping-pong” kids were reinforced for eating chocolate by the chance to play ping- pong. They were able to completely change the kids’ preferences so the chocolate group now preferred ping-pong and vice versa.

Here are two related points about the value of reinforcers.

  • Just because something is reinforcing, that doesn’t meant the animal will work for it.
  • Expectation can be a factor. If you expect one type of reinforcement and you get another, it might not function as a reinforcer even thought it might be reinforcing in other situations.

Confusion Inside the Quadrants….

If we focus on processes and procedures instead of the properties of stimuli, and recognize that the value of reinforcers and punishers is not fixed, does that make the quadrants more useful? Yes, but we also have to recognize that there are some other sources of confusion that make it difficult to decide which quadrant is driving behavior.   He shared some more information and examples to help clarify the differences (and possible ambiguities) between positive and negative reinforcement, positive and negative punishment, and how extinction fits into the quadrants as well.

Positive vs. Negative reinforcement:

Using the quadrants, positive reinforcement happens when something is added to the environment to increase behavior. Negative reinforcement happens when something is removed to increase behavior. Sounds simple, but the distinction is ambiguous because the adding and removing can be symmetrical.  Here’s an example from Weiss and Laties (1961).

A rat is placed in cold chamber

  • Lever pressing produces heat
  • Lever pressing increases
  • What maintains lever pressing?
  • Positive reinforcement or negative reinforcement?

It might be …

  • Positive reinforcement – lever presses produce heat
  • Negative reinforcement – lever presses remove cold
  • The problem is that adding and removal are symmetrical, the addition of one event is the removal of the other and vice versa.
  • Adding heat removes cold, adding a smile removes a frown, adding food removes deprivation.

Because of this ambiguity, some professionals have argued that there is no clear distinction between positive and negative reinforcement in many situations and it would be better to get rid of the terms positive and negative and just talk about reinforcement.


There are some significant differences between an animal’s behavior when put under conditions where learning has happened with positive compared to negative reinforcement (phew…say all positive reinforcement trainers….)

First, there is an asymmetry in how the animal responds to the presence/absence of the stimulus:

  • In positive reinforcement, the response occurs in the absence of the stimulus or situation upon which reinforcement is based. This means you do the behavior hoping it will lead to reinforcement, even if you can’t see the reinforcement. My horse offers behavior even if he can’t see the carrots, and he may continue to offer the behavior even if he is not reinforced.
  • In negative reinforcement, the response occurs in the presence of the stimulus or situation upon which reinforcement is based. This means the stimulus must be present in order for you to respond to it. My horse will stand in the location I want if I tie him with a lead rope. But if the lead rope is not there, he won’t stand in the location I want.

Second, under negative reinforcement conditions, the presence of the stimulus to be removed generates responses that can compete with the response to be reinforced. This can make negative reinforcement less effective.  Here’s an example:

  • When cold, a rat huddles in the corner and shivers – this competes with lever pressing.
  • When cold in bed, the human shivers and curls up – this competes with getting up and adjusting the thermostat.

This is an important consideration because if the stimulus is intense enough, it will produce emotional behavior which can interfere with learning. He said that when they used to do shock experiments with rats, up to 60% of them did not respond as expected and could not be shaped to avoid or escape the shock.

Third, you have to take into consideration that there are two types of negative reinforcement and they can have different emotional impacts on the animal.

  • Escape – the behavior results in removal of something already present, there is no warning stimulus. This type of negative reinforcement doesn’t seem to produce anxiety. The animals just learn from it.
  • Avoidance – the behavior postpones or prevents something from occurring. There is a warning stimulus but it is not necessary. The warning actually becomes a source of anxiety and he said this is why poisoned cues can be such a problem. The dog is constantly anxious about what might happen.

Moving on to other possible sources of confusion….

Positive vs. negative punishment.

We have the same problem of symmetry involved in the presentation and removal of the stimuli. I’m actually scratching my head a little over this one as I don’t see quite how you do this, but the general point is that there can be situations where the positive and negative punishment are equal and opposite.

  • Positive punishment – lever pressing produces cold
  • Negative punishment – lever pressing removes heat

Negative punishment had been studied less in the laboratory as it is hard to set up conditions where reinforcement is removed as a way to change behavior. You cannot take food away that is already eaten, or if food is present, the rat would be eating instead of pressing the lever and there is no opportunity for punishment.  What is usually removed are stimuli that signal reinforcement, so negative reinforcement could be called “time out from positive reinforcement.” Here are some examples of negative reinforcement:

  • In the lab, the lights turn off and positive reinforcement is not available for a short period.
  • In the real world, you can signal to the dog that the clicker game is off temporarily. This can be done by taking your treats and leaving.
  • A training example of negative punishment is the method called “penalty yards” which is used to teach dogs to loose leash walking. The dog is walked from point A to point B and if the dog has an “error,” the trainer turns around and the dog has to start again from the beginning of from a previous point on the path from A to B. Reinforcement happens at point B so the dog wants to get there. Any return to point A is taking it away from the destination where reinforcement happens. The exercise is designed to use negative punishment to decrease the incorrect behavior or “error” in the loose leash walking.

What about Extinction?

Extinction is not included in the quadrants, but needs to be considered when we are looking at ways to change behavior.  Extinction is the process of a behavior decreasing over time when it is no longer reinforced.  The reinforcement could have been coming from the trainer or be coming from the environment.   It is often confused with negative punishment because both result in a decrease in behavior.  But the difference is that:

  • In extinction the previously reinforced behavior is ineffective and there is no stimulus change following the behavior. The animal does the behavior and nothing happens, whereas previously when the animal did the behavior, something did happen (your reinforced it in some way).
  • In negative punishment, there is a stimulus change where positive reinforcers are not available or are removed. The animal does the behavior and something changes. You can take your treats and leave or you can remove access to existing reinforcers. Negative punishment is more accurately viewed as “time out from positive reinforcement,” or “reinforcer loss.”

Using the example of loose leash walking again, the method of “being a tree” is an example of using extinction to decrease leash pulling. One reason dogs learn to pull on leashes is because they learn they can get their handler to take them where they want go, or to go there faster.   If the handler stops and stands still, instead of going faster, the behavior will decrease because the previous reinforcement (forward movement) is no longer available.  This is actually a great example of how the quadrants can get confusing because stopping is a change in stimulus conditions.  To make sense of it, you have to view the stopping as removing forward motion, not as adding a stop. If you compare this to “penalty yards,” you’ll get a feeling for how negative punishment is different than extinction.

Extinction and negative reinforcement

We usually think of extinction in terms of positive reinforcement, but you can have extinction with negative reinforcement too.

  • Extinction and Negative reinforcement: discontinuation of negative reinforcement – the response does not prevent or remove the aversive stimulus.   A great example of this is riding a horse that no longer responds to leg pressure. Leg pressure is taught by removing the pressure when the horse goes forward. If the rider keeps her legs on, the horse learns to ignore it because there is no change when the horse moves forward. Over time the horse becomes less and less responsive to leg pressure.

Can we talk about Extinction and punishment?

Well, we could say that:

  • positive punishment – discontinuation of positive punishment
  • Negative punishment – discontinuation of negative punishment

But this invites confusion because extinction is usually discussed in the context of reinforced responding. Since discontinuation of punishment produces an increase in behavior it doesn’t make much sense to talk about extinction and punishment.

Operant quadrants in popular literature

He had a few examples of how the quadrants are now being presented in popular literature. Dog trainers and owners are becoming more educated and some do want to learn more about the science behind clicker training. They are looking for practical information on what the quadrants mean and trying to find new ways to present them to encourage people to change to more positive reinforcement based training.  This is a great step forward, but it’s important that we present them accurately and in keeping with their original use.

In summary, to avoid confusion

  • Use quadrants to identify processes and procedures (not types of stimuli)
  • Don’t assume something is reinforcing or punishing, let the animal’s behavior tell you
  • Remember that reinforcers and punishers are relative and can change over time
  • Don’t get hung up on the terms, look at the emotionality


Notes from the Art and Science of Animal Training conference (ORCA): Steve Aibel on “Learning to Bond”

beluga and calfLearning to Bond:  The successful reintroduction of a beluga whale calf

Steve started by talking a little bit about what animals need to thrive. He had a nice diagram (shaped like a triangle) with physical needs as the foundation, social needs in the middle and mental needs on top.   He said that animals that are not physically or socially comfortable are not ready to train.  A successful breeding program is not possible without meeting the physical and social needs of the animals.  What is not always recognized is that it also depends upon good training.  In this presentation,  Steve talked about all the ways that good training helped to successfully reunite a beluga whale and her calf.

At Sea World they have been successfully breeding belugas for long enough that they have had opportunities to observe both normal and abnormal mothering behavior. There are three main behaviors that should happen in the hours immediately after birth.  These are contact and retrievals, mother following, and nursing. In animals where there is a healthy mother-calf bond,  the trainers would observe all of these happening.  If these behaviors did not happen, then the trainers would have to intervene.

In the year that this story takes place, they had two pregnant beluga whales that were due to give birth around the same time. Crissy was an experienced mother who had successfully raised calves in the past.  Luna had given birth once before, but she had ignored her calf at birth and the trainers had to intervene and raise it instead. The trainers are able to hand raise baby whales and other animals, but they have found that these babies often have a hard time fitting into the social structure of their species and they are never quite “normal.”  Babies are learning from the moment they are born and they really need to be learning how to fit in with their own kind.

Because Luna had ignored her last calf, the team decided they needed to have a post-parturition plan for what to do if Luna ignored her new calf. The plan included closely monitoring Luna and the calf to see if the three behaviors were happening, and if not, intervening to provide the calf with his nutritional and social needs.

They were hoping that Crissy would calve first and provide an example for Luna, but instead Luna had her baby (Samson) first. Samson’s birth was uneventful, but in the first hours after birth Luna did not try initiate any contact and retrievals. Samson just swam around the pool by himself.   The first priority was getting Samson to eat.   While mothers’s milk is ideal, milking a beluga is slow work, so Samson was fed a combination of Luna’s milk and formula.   In anticipation of possibly needed to do this, the trainers had already done some training so that Luna would allow herself to be milked.

Samson was bottle fed to meet his physical needs. But what about his social needs? They introduced him to another beluga whale (Martha) who was an experienced mother. She showed an interest in him, encouraged him to follow her, retrieved him when he wandered off, and even allowed him to “nurse.”  It was never clear if he was getting anything from nursing, but he did continue to try, so perhaps he was getting something.  Martha had not calved recently but animals can sometimes produce milk under the right set of circumstances.

At this point things were going pretty well. Samson was growing and he was bonding with Martha and showing normal calf behavior. The next question was whether or not he could be re-introduced to Luna. The hope was that now that Samson had learned how to interact with his surrogate mother, perhaps he would initiate that kind of relationship with Luna.   The question was what would Luna do?

Luckily, they had a “secret weapon” which was Crissy who was due to calve any day. They put both females in the same tank and let Luna observe Crissy give birth to and bond with her baby.   They kept Luna in there for two days following the birth and then they re-introduced her to Samson.  This time the pair did engage in normal mother-calf behavior.   All three of the major behaviors (contact and retrieval, mother following, and nursing) were observed.  I thought it was interesting that Martha was ok with this and allowed Samson to go off with Luna.

Steve finished his talk by asking the question “Why do we care about this?” Yes, it’s a nice story, but is that all?  No, the reason it’s worth sharing is that it shows that good training can make a big difference in how successful we are with our animals, not just for performance, but in all aspects of their lives. Or to put it simply, “everything matters.”

It mattered that they had trained Luna to be milked. It mattered that they had training and trust in Martha, so they were comfortable introducing her as a surrogate mom. It mattered that they had training and trust in Luna, so that they could allow Luna to observe Crissy deliver and mother her calf. It mattered that they had training and trust in Crissy so they were comfortable allowing her to deliver in the same tank as Luna. If they had not spent time training Crissy, Martha, and Luna, it would have been much harder to make decisions about how to help Samson.   The training time created relationships with these animals so that they had good choices about how to help Luna and Samson.  You never know when something unexpected is going to happen, but if you have spent time training and building a relationship with your animal, you will have built a solid foundation, and that will make a successful outcome much more likely.

Thank you to Steve Aibel for permission to share my notes.  Thank you to everyone at ORCA who worked hard to put on this conference.  You can visit the Art and Science of Animal Training facebook page at https://www.facebook.com/The-Art-and-Science-of-Animal-Training-1460845514215463/?fref=ts.

The dates for next year’s conference have just been released. It will be held on February 25-26, 2017.  See you there!