equine clicker training

using precision and positive reinforcement to teach horses and people

ASAT 2019: Hannah Branigan – Close the Loop: Training reinforcement as behavior


Hannah Branigan is a professional dog trainer, teacher, author, and self-proclaimed training nerd. In her presentation, she took a closer look at the reinforcement process and how we can and should include teaching reinforcement procedures as part of our training.

The first step is to think about the reinforcement process as a behavior. You have to teach the dog where and how reinforcement will be given. It’s important to do this before you try to use it.

Key points:

  • Reinforcement is behavior
  • You can train like any other behavior
  • Sometimes it’s more than one behavior
  • You can put it on cue
  • It works a lot better this way

Defining the reinforcement process

Reinforcement is not a single event. It has a beginning, a middle and an end. Before we can start looking at the reinforcement process, we have to sort through a little terminology because there are many different words that we could use to describe the first step in the reinforcement process.

Common terms:

  • an event marker
  • a bridge
  • a marker word
  • a verbal marker
  • a conditioned reinforcer

If people are using different terms, it can can lead to confusion. Therefore, Hannah prefers to use the word “click” as a general term to indicate the moment when the behavior is marked and the start of reinforcement. This doesn’t mean you have to use a clicker, it’s just that it’s easier to understand if she consistently uses the word “click” when talking about the reinforcement process.

The click is:

  • an event marker
  • a conditioned reinforcer
  • a cue

Reinforcement Strategies

There are lots of different options, but in order to take advantage of them, we need to learn how to teach new reinforcement strategies and how different reinforcement strategies affect the dog’s behavior.

Hannah talked about how we can optimize our use of reinforcement by observing and using a dog’s natural behavior, orienting toward reinforcement. Then, she described how to teach your dog to use different reinforcers – ones that might require more training before they can be used successfully.

The big question is: What does the dog do when he hears the click?

The dog can do different things, but we have to decide ahead of time what we want him to do and teach him how to do it. Let’s look at how we can take advantage of a dog’s natural inclination to orient toward reinforcement.

Making Use of Orienting Toward Reinforcement

Some common behaviors are:

  • Dog turns toward hand
  • Dog stares at door
  • Dog runs ahead
  • Other anticipatory behaviors

She shared examples of how the behaviors that occur between the click and reinforcer delivery can affect the behavior you are training. The main point here is that dogs orient toward reinforcement, so anticipatory behaviors will look different depending upon what the dog expects. She had video for each of these examples, but I think you can get the idea from the descriptions.

Example: Dog was forging during heeling

  • The dog was being trained to heel using chasing a frisbee as the reinforcer
  • The handler clicks and then throws the frisbee
  • The dog is anticipating going out to the frisbee, so he starts to forge ahead during heeling. The reinforcer appears in front of him, so that’s the direction he tends to move.
  • The “fix” was to change where the frisbee was presented (closer to handler) and use a new marker word.

Example: Dog doesn’t understand how to get reinforcement

  • This was an example of what happens when you have not considered the dog’s expectation about where or how reinforcement will be delivered.
  • The dog has been trained to wait quietly at the door when someone enters.
  • The dog expects the food to come from the person at the door because that’s who usually delivers food during training.
  • When the trainer (standing near the dog) offers reinforcement, the dog becomes very confused – even does a weird noise, because he doesn’t understand where the reinforcement is coming from.Thi

Example: Dog learning to heel

  • After the click, the food is presented behind the trainer’s back.
  • This works to the trainer’s advantage because it makes it less likely that the dog will position itself too far forward.

Reinforcement is about the process

This means that:

  • click = food, is really click = EAT food
  • catch = ball, is really catch = CHASE ball
  • Say “hi” = person, is really say “hi” = GREET the person

When we picture the ABC cycle, we often focus on what happens between the antecedent and the behavior, but we should pay more attention to what happens between the end of the behavior and the end of reinforcer delivery. We want those behaviors to be fluent, just as we would want fluency with any other behavior.

Fluent behavior happens:

  • Without hesitation
  • In context
  • With low latency
  • Ready to repeat

Teaching Reinforcement as a Behavior

What if you want to use a reinforcer, but your dog isn’t interested or doesn’t have sufficient fluency? In that case, you will need to spend time on the reinforcement process itself, viewing each step or component as a behavior that you can train.

Training food eating

What is food eating? It’s a behavior, and we can break it down into a series of teachable steps.

  • Respond to the cue (presentation of the food)
  • Take the food
  • Swallow the food
  • Look back to the handler

Example: Teaching the reinforcement process for tossed or dropped food

  • Video of drinking bird game.
  • The handler drops food on the ground, the dog eats (his head goes down), and then he looks back up (his head goes up) when he’s ready for the next treat.
  • This teaches the dog to orient back to the handler immediately after eating the food.
  • She had another video of Rugby (her terrier) doing a similar game on grass. In it, she’s teaching him to look back up at her as soon as he has found and eaten a tossed treat. He has even learned to eat only the big chunk, leaving the smaller pieces to clean up later.
  • Sometimes people are concerned that throwing food will lead to more sniffing, but that’s not the case if you take the time to teach the dog to orient back to you immediately after eating.
  • For good food delivery, both dog and human need to be fluent.

Example: Teaching a dog to keep his nose in contact with your hand while you deliver several reinforcers

This is a variation on using food where instead of tossing the food, she holds the food in her hand and feeds the dog in such a way that he learns to stay in position for multiple treats. It’s a reinforcement strategy that is useful if you want the dog to stay in position so you can ask for several behaviors before releasing him.

  • She had a video showing how she holds both treats and the clicker in one hand so that she can click the dog for staying on her hand while she dispenses treats one at a time.
  • The idea is to keep the dog’s nose on her hand so he learns to stay in position throughout the whole cycle.
  • Her other hand is free so she can use it to do something else or ask for a behavior.

Viewing reinforcement as part of the movement cycle

She had several slides that showed how reinforcement has sections – the beginning, the middle, and the end. We need to be able to control all three parts.

You can think of it as a behavior chain. Here’s a comparison between getting a beer (a simple behavior chain) and the three parts of reinforcement.


Treat magnets

A treat magnet is a treat that his held right in front of the dog’s nose. The dog is taught to stay on the treat magnet while the trainer moves her hand. This allows her to guide the dog to a specific location before feeding the treat. It allows the trainer to control all three parts of the reinforcement sequence.

Think of it as a behavior chain:

  • Look for the hand
  • Nose to treat
  • Follow treat
  • Line up

Examples:

  • Video of Figment (border collie) going to a foot target. She uses a “treat magnet” which is a treat held right on the dog’s nose. Figment has been taught to stay on the treat while she moves it. She can use the treat magnet to guide Figment back into position so he can start again.
  • She had another video of using a treat magnet when working on obedience jumps. She can send the dog over the jump and around a cone, click and put him on a treat magnet to bring him back around to the starting position.
  • She also showed using a treat magnet to keep Rugby in position while she was placing out articles.

Toys as reinforcers

We can look at the process in a similar way when we are using toys as reinforcers. For example, if I throw a ball as a reinforcer, the behaviors in the chain would be:

  • the dog runs after the ball
  • the dog catches/picks up the ball
  • the dog brings the ball back to the handler
  • the dog releases the ball

She had a few charts showing possible outcomes. I think this was everyone’s favorite:

But, of course that’s not what she wants. What she wants is for the dog to chase the ball, bring it back, and release it so that she can ask for another repetition.

How do you teach this?

She had some video showing the steps she used to teach Figment to follow the reinforcement sequence she wants. This was done by teaching the basic steps (chase, catch, return, release) in such a way that he was less likely to get overexcited and more likely to release the ball. In the beginning, she kept the ball close and focused on prompt, calm reinforcement.

She also used two balls so that he learned to drop the ball when he returned to her. It was important that he learned to release the ball on his own, because there’s always the risk of introducing an aversive if the trainer has to do something to get the dog to release the ball. Even something as simple as a collar grab can end up poisoning the whole cycle. It may seem like a simple short-cut, but it can create a whole host of problems because the effect of the aversive moves backward.

The collar grab is aversive (red), which affects the “out” (it also becomes red). If the “out” becomes poisoned, that can affect the “bring” and maybe even the “catch.”




Incorporating new reinforcement procedures into a session

  • A new reinforcement procedure is a new variable. If you change that, don’t change anything else.
  • Only change one criterion at a time. If throwing the food farther, don’t add distractions, etc.
  • Using multiple reinforcement procedures increases the challenge on you both. This is especially true if you use more than one reinforcement strategy within a session because remembering which cues to use takes up brain space.
  • She usually starts a session with a few reps where she is just practicing her reinforcement strategy. This tells the dog how reinforcement will be delivered.

She finished with a quick review of the steps she uses to teach reinforcement as a behavior:

  • Fluent behavior
  • Add a cue
  • Stimulus control
  • Put in the chain

Katie’s notes:

While there are some differences in the types of reinforcers we use with horses and the number of different ways we can deliver them, I think that most of what Hannah presented can be applied to horses.

Here are some important points that we can take away from Hannah’s presentation.

  • Every reinforcement process you want to use should be fluent for both the horse and handler.
  • Reinforcement has a beginning, a middle and an end.
  • Where you deliver the reinforcer will have an affect on the behavior.
  • You can teach each one by breaking it down into smaller behaviors.
  • With some reinforcers (grass, movement, tactile, etc.), you will have to provide information (a cue, a set duration) that tells the horse when reinforcement is over.
  • If you use multiple reinforcement strategies, you need to provide some way for the horse to recognize which one you are using. This could be a different marker signal, or we can associate different reinforcement procedures with different behaviors or different situations (relying on context cues).
  • An aversive that occurs during the reinforcement process may have an undesirable effect on the behaviors that preceded it in the movement cycle.


While I try to be accurate in my note taking, there may be some errors either in my understanding or my presentation of the information from this talk. If you have questions, feel free to contact me or leave a comment.

Thanks to Hannah Branigan for allowing me to share her presentation. Thanks to the ORCA students and to the organizers of the Art and Science of Animal Training Conference for all their hard work on putting on this great event.

If you want to learn more about Hannah, you can visit her website www.wonderpupstraining.com.

Categories: Uncategorized

Tags: , ,

4 replies

  1. Love it! Thank you so much

    Like

  2. You’re welcome. Now to figure out if I want to add new reinforcement procedures…

    Like

  3. absolutely applicable to horses, if you haven’t seen Alexandra Kurland’s Loopy Training, written about horses, this is a good summary: http://stalecheerios.com/training-concepts/alexandra-kurland-loopy-training/

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s