equine clicker training

using precision and positive reinforcement to teach horses and people

Superstitious Behavior

black  cat leavesLast Friday was Friday the 13th. I don’t consider myself a superstitious person, but I do have to admit that I am unlikely to do anything risky on Friday the 13th. I mean, why take a chance? Interestingly enough, on last Friday I was feeling a bit perplexed over a problem  with one of my training projects and it had to do with a “superstitious behavior” that had crept in and was proving to be a little difficult to remove.

 

What is a superstitious behavior?

In psychology, a superstitious behavior is a behavior that an individual does because he thinks it is associated with reinforcement, but in reality it has no direct effect on whether he will be reinforced or not.

There are lots of examples of superstitious behavior in people including things like wearing your lucky socks, buying lottery tickets at the store where you won before, and rituals associated with games or competitions.

In animals, superstitious behaviors are those behaviors that have become part of the animal’s repertoire (or have increased in frequency) because they are associated with other behaviors that lead to reinforcement. The animal doesn’t actually need to do the superstitious behavior to earn reinforcement, but it doesn’t know that, so it continues to do it, either as a separate behavior or in combination with other behaviors that do lead to reinforcement.

In the laboratory:

The term “superstitious behavior” was coined by B.F. Skinner who noticed that pigeons would end up performing a specific behavior, even under conditions where reinforcement was delivered at random.   The behavior was selected and then further shaped because the pigeon just happened to be doing it when the random reinforcement was delivered. I would assume that the selection of a behavior happens quite soon after reinforcement starts, but I can’t find any documentation on this, it just makes sense to me. Anyway, the reinforcement made the pigeon more likely to repeat the behavior, which increased the probability that it was doing that behavior when reinforcement was delivered, and so on, until that behavior had been reinforced so strongly that the pigeon consistently repeated it.

Iver Iversen talked about superstitious behaviors in his talk, at the Art and Science of Animal Training conference in 2016, when he explained that as soon as reinforcement is available, some behavior will be selected. He had a video example of a superstitious behavior that a rat “learned” when it was put in a Skinner box which he had set up so that food was delivered on a random basis. When he came back later, the rat was doing a full roll between reinforcements. This behavior had been shaped by the random delivery of food and the rat continued to do it, even though it could have done nothing and food would still have been delivered.

This same type of experiment has been done with people and produced similar results.  In “Learning and Behavior,” Paul Chance describes several experiments. In one, thhe experimenters put an individual in a room with a device that would intermittently provide reinforcement.  It was set up to reinforce at random intervals, but the person was not told that information. In most cases, the person would start trying to figure out what he had to do to be reinforced. He would come up with some behavior, start to repeat it, it would get reinforced, and he would repeat it more.  This pattern would continue, with every reinforcement confirming his belief that he had found the behavior that led to reinforcement.  The subjects in the experiment were sure they had figured out what they had to do to get reinforced and could not believe the reinforcement had been totally random.

In real life and training:

What happens in the laboratory is not always a good predictor of what will happen in real life, but it turns out that animals do learn superstitious behaviors in real life and during training sessions, and this can happen for a few reasons.

  • In real life, it can happen for the same reasons it happens in the laboratory where random reinforcement ends up selecting for a specific behavior.
  • In training, it can happen because an animal can be doing several behaviors at once or there can be overlap between behaviors. Behavior rarely happens in discrete little units and while the clicker(or other marker) allows the trainer to mark a specific moment, there can still be several behaviors happening in that exact moment.
  • It can also happen because animals will often connect together several unrelated behaviors (do this and then do that) or fall back on previous associations when they are looking for patterns that lead to reinforcement.  So even though the trainer is trying to select out one behavior, another one can “tag” along.

This why it’s important to set up your training so that you can isolate out individual behaviors and to be watching for unwanted connections that the animal might be making.

While there are similarities between superstitious behavior and unwanted behaviors that have become attached to other behaviors, I don’t think it’s accurate to say that any behavior that contains both wanted and unwanted components contains superstitious behavior.  Sometimes animals offer the “wrong behavior” because of bad timing, cueing issues, or other trainer errors.  But these incorrect responses will disappear if they are not reinforced.   A superstitious behavior is a little different because it is one that was never intentionally reinforced and the animal continues to offer it over a long period of time, even though it may never receive any direct reinforcement.

My personal thought is that, like in people, animals continue to do superstitious behaviors because they think they work and the difference between an unwanted behavior and a superstitious behavior is the animal’s own belief that this behavior is part of what it needs to do to earn reinforcement.  Of course, I have no way to know what exactly what an animal is thinking, but I have found that animals can be very committed to superstitious behaviors, and changing them can take quite a bit of effort.

Here are two examples of superstitious behavior that come from my own training.   In one case the superstitious behavior was reinforced as part of a chain. In the other case, it is not as clear where it came from, but it became associated with the “hind feet on the mat” behavior even though it was never deliberately reinforced.

An example: Willy learns to nicker on cue

My first encounter with a superstitious behavior was many years ago when I wanted to teach my older horse Willy to nicker on cue. I knew that if I went into the barn at certain times of the day, he was more likely to nicker.   I started by reinforcing him for nickering and he did start to nicker more often, but it was not predictable enough that I could add a cue. So I decided it was important to get a few repetitions in a row.

What I had been doing was clicking and treating him for the nicker when I walked in the barn, and then continuing with whatever I was out there to do. My idea was that if I walked out of the barn after reinforcing him for nickering, and then came back in, I could do several repetitions in a row and then I could add a new cue because the current cue (walking into the barn) was not terribly useful.  And, even though he was nickering more often than when I had started reinforcing the nickering, I was not sure he really knew what was being reinforced.  I thought clicking and treating the same behavior a few times in a row would have the additional benefit of helping him understand that it was the nickering that was being reinforced.

Guess what? It worked great. I got him to repeat the nickering a few times in a row and after a few days, I added a cue. But… at that point I realized I had a problem. When I cued the nicker a few times in a row (while standing in front of him), he would walk in a circle in his stall, then come to the door and nicker. It turns out that when I was teaching the behavior and left the barn between each repetition, he would make a little circle and then when I came back in, he would nicker. So the behavior I taught was “walk in circle -> nicker,” not just “nicker.” He thought that walking in a circle was part of the behavior, so even when I stood in front of him and cued him several times, he continued to circle  before each nicker.

I learned a few things from this. One was that it’s really a good idea to keep your horse within view while training (ha!) and the other was how easily behaviors can get linked together so that you get the one you want, but you also get some other behaviors coming along for the ride.

Another example: Red and the resting hind leg

My more recent challenge with superstitious behavior happened for a totally different reason. A few summers ago, I taught Red to back up on to a mat. He had already done some hind foot targeting on the mat, but I had taught this by walking him forward on to the mat. Now I wanted to train him to back up on to it.  Prior to this, I had taught this behavior to other horses by asking them to back and clicking for the hind feet hitting the mat. Usually I start by clicking for backing up and then I add the criteria of backing until they step on the mat and over time I add distance, if I want.

In his case, I had started as usual by clicking for backing and when he got to the mat, I would click and reinforce him in place a few times.  But I had been doing some reading about backchaining and how important it was to practice the final behaviors in a chain, as well as to repeat a behavior to build muscle memory.  Backing to a mat is actually a chain of backing steps followed by a halt. He already knew the halt on the mat so I decided to put the mat directly behind him and click and treat for a single foot stepping back on to the mat. If he stepped on to the mat, I clicked and treated and then took him one step forward off the mat. So I had one hind leg stepping on and off the mat. I practiced both sides and he did great. I did also reinforce for both feet solid on the mat so I thought I was practicing the final two steps in the chain.

The next day when I brought him out and positioned him with the mat directly behind him, he stepped back and rested his foot by balancing on this toe on the mat. Hmm… ok? That was not a behavior I had clicked. The best I could guess was that he was resting his toe so he was prepared to move either way.  From that position, he could lift it up and go forward or continue to step back. Or perhaps I had reinforced the foot in that position by clicking for moving forward and back.

Either way I had not specifically clicked for a resting toe, but that was what he had learned. Now whenever I backed him on to the mat, instead of standing solidly with both feet, he would rest one. I did eventually get him to land on the mat and stand without resting a hind foot but it took quite a few sessions to eliminate the toe resting behavior. At this point I didn’t consider it a superstitious behavior, just an odd by-product of how I shaped foot movement on to the mat.

Fast forward to this spring when I have been  doing some hind foot targeting with Red and on the third or fourth session, the toe resting came back in. He had been doing a nice solid “two hind feet on the mat” behavior and then one day he started to rest the toe again.  So I had to go through the process again of cleaning up the hind foot targeting to get a solid stand.  At this point I did look at it and think “hmmm.. a superstitious behavior” because it was a behavior that he was clearly offering in hopes of reinforcement, even though it had never been reinforced directly and was not a necessary part of the final behavior.

What do you do about superstitious behaviors?

I think the challenge with decreasing the incidence of superstitious behaviors is that they are often being indirectly reinforced and it can be hard to determine why the behavior is happening. The best way to change behavior is by changing the antecedents or the consequences, so when they are both murky, well…..that makes things a little complicated.

And it seems, in many cases, that the animal is SURE that what he is doing is part of the behavior the trainer wants.  Red’s not going to give up resting that hind leg any more than someone is going to give up his lucky socks or rabbit’s foot unless I convince him that something else is a better option.

The good news is that while it might take a little more time and creativity to get rid of superstitious behaviors, I can do it using the same strategies I would use for any unwanted behavior. In some cases, especially if the superstitious behavior is occurring on its own, I  can make it less likely to happen by setting up conditions where it is not associated with any reinforcement and training an alternative behavior instead.   In other cases, it’s better to just start again and be more careful about the set-up and what is reinforced. Now that I know Red might think about resting that toe or Willy might circle, I could plan better next time.

As a final note, I have to say that superstitious behaviors are a good example of how persistent an animal can be about continuing to do a behavior if the animal believes it needs to do it to earn reinforcement. And while that might be annoying if you are dealing with an unwanted behavior, I think it says something great about how committed our horses can be if they think they know the right answer.

Categories: Uncategorized

Tags: , , , ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s