equine clicker training

using precision and positive reinforcement to teach horses and people

ASAT Conference 2018: Ken Ramirez on “No Reward Markers (NRMs): Science and Practice”

NMR bucket1This the first in a series of posts based on my notes from the 2018 Art and Science of Animal Training Conference that was held in Irving, Texas on March 24-25, 2018.   To learn more about the conference, you can visit the conference website.

While I try to take accurate notes, it is possible that there are errors or that some detail is lacking.  If you post a comment or email me, I can try to clarify or provide some additional information. Many thanks to the speakers and organizers who allow me to share.

Ken Ramirez:  No Reward Markers (NRMs): Science and Practice

Whether or not No Reward Markers can be used as part of a positive reinforcement training strategy is always a controversial topic.  As part of the Sunday morning presentations on reinforcers and conditioned reinforcers, Ken shared his thoughts on the subject. This was a 20 minute talk.

Ken started by clarifying what he meant by a No Reward Marker.   The term is used by many trainers, but there are significant variations in both definition and practice, so it’s a good idea to start by defining it.

What is an NRM?

  • Most common use is that it marks the moment the animal does the wrong or incorrect answer
  • Opposite of the click
  • Conditioned punisher

If you use NRMs, you might agree with the first two points, but you will probably question the third point.  Most trainers who use NRMs would not describe them as conditioned punishers.  Instead, they prefer to describe them as providing information to the animal so that he doesn’t waste time pursing behaviors that will not earn reinforcement.  But Ken said that in all his years of training,  he has only seen 13 people (out of thousands), who can use an NRM without any visible side effects.

It may be easier to see this if you look at how conditioned punishers are taught and the possible side effects.

What are conditioned punishers?

A punisher is a stimulus that, when applied immediately after a behavior, decreases the likelihood (frequency) of that behavior happening in the future. A conditioned punisher is a stimulus that has been conditioned, through association with another punisher, so that it can be used to decrease behavior.

He shared a video example of a verbal conditioned punisher that was learned through pairing with a finger poke (I’m sure you can guess who).  The dogs clearly responded to the sound with defensive body posture and by recoiling.   The video showed that the conditioned punisher was effective, but also that it had side effects.  There’s no argument that conditioned punishers can be effective, but they are not without risks.

The rest of the talk was looking at various applications of NRMs and evaluating both their effectiveness and side effects.  The conundrum is this… If the NRM is effective at reducing the behavior, then it is, by definition, punishment.  If the NRM is not effective – it does not function as a punisher to decrease the behavior in the future – then why use it?

To unravel this, you have to look at the different applications of NRMs to see whether the NRM is functioning as a punisher, has no effect, or is perhaps functioning as something else like a new cue or a means of redirection.

NRMs: Varied uses and applications

To indicate “no” or “wrong”

  • Marks incorrect response
  • Trainers say they just want it to be information
  • Trainers think it’s ok if it is delivered in a passive manner. How about a passive “oops?”
  • The problem is that if it is effective, then by definition it is a punisher

As a warning signal

  • Last chance before something bad is coming
  • Warning prior to a more aversive stimulus (or a more severe one)
  • Varied effectiveness
  • Can become a new cue for the behavior
  • Do generate an emotional response

Ken had two examples to show some of the things that can happen when an NRM is used as a warning signal.

Example 1:  When he was a kid, his Mom would ask him to take out the garbage.  She might ask him a few times (“Kenny, take out the garbage”) and then if he didn’t do it, she would call him by his full name.  When he heard his full name, he got up and did it.  The use of his full name was effective in that it did cause him to get up and take the garbage out, but it didn’t change his future behavior – he was still likely to ignore her when she said “Kenny, take out the garbage.”  If his full name as an NRM was effective, then he should have learned to take the garbage out when she asked him the first time, but he didn’t. And, over time, the use of his full name just became the new cue (or part of the new cue) to take the garbage out.

Example 2:  The warning “ding, ding, ding” in his car when he leaves the lights on.  The sound is aversive and he feels a moment of frustration when he hears it.  It is effective because when he hears it, he does turn the lights off. But, has it made him less likely to leave the lights on? Maybe a little over time, but it could only be considered a weak punisher because it doesn’t change his behavior very quickly.  He did joke that if it was followed by a strong aversive, it might be more effective, but then he would probably sell the car.   In addition to being a warning, the sound also becomes a cue for a specific behavior – turn off the lights.

To indicate “correct the behavior or you will not be reinforced” 

He had a video showing a blood draw in a hyena where the hyena moved away before the trainer was done holding off the spot. The trainer said “ah ah” and cued the hyena to move back.  He came back into position, she finished, clicked and reinforced him.

Was the NRM effective?

  • Ken can’t see any change in the hyena (good or bad)
  • She uses her cue to bring him back
  • It’s possible the “ah ah” will just become a cue to come back into position
  • the “ah ah” is possibly just superstitious behavior on the trainer’s part

One of the points he made, using this example,  was that since the trainer only uses positive reinforcement, it’s likely that the “ah ah” has no meaning to the hyena, which is why Ken doesn’t see any response. The hyena doesn’t return to position when she says “ah ah,” (he responds to her cue), but it might learn to over time, if she continued to follow it with her cue.

Used as an interrupter

  • Stops behavior in the moment, but doesn’t always change future behavior
  • Still aversive
  • Weak or ineffective punisher (more like redirection)

Used as a “stop” cue?

This is a more common (growing) use among R+ trainers. The idea is to use the NRM and then immediately redirect and reinforce the alternative behavior.  Again, you have to look at the effect on behavior and the animal’s emotional response. What does the animal look like?

Example:  He had a clip of Susan Garrett teaching a dog to do weave poles.   The video shows several NRMs being used.  In each case, the dog is not reinforced and is re-started. He shared this as an example of an NRM that doesn’t seem to have any aversive side-effects.

  • She has a variety of NRMs (I think he said 4)
  • The dog maintains a high level of enthusiasm even after the NRM
  • In the last part of the clip, she placed a toy a short distance from the end of the weave poles. If the dog went through correctly, he retrieved the toy and she would play tug.  If he made an error, she used her NRM and he returned (without retrieving the toy) and was re-started.
  • Note: Steve White pointed out that it’s not the toy that is the reinforcer, but playing with the toy – if the dog learns that he won’t get to play with the toy – then there’s no point in going and getting it.

Final thoughts

  • Traditional use is that an NRM functions as a punisher
  • Can assist in shaping behavior, but can also create frustration
  • Other similar uses may not actually be an NRM (it’s more likely they are a cue or redirection)
  • Often conditioned inadvertently
  • Only skilled and disciplined trainers can use them well, not a bad tool, or at least should be used with thought and care.

Categories: Uncategorized

Tags: , , , , ,

12 replies

  1. Thank you so much for writing this post. I think this sums up Ken Ramirez presentation about the No Reward Marker perfectly. Well done!


  2. I had a greyhound that I was training to “leave it”. I was using the clicker to mark backing away from a treat held out with “Leave it” … he caught on very quickly. But I was also using “oh-oh” and hiding the treat behind my back if he moved to get the treat after being told “leave it”. After he had lost the treat and gotten a couple of “oh oh’s” he started reacting to “oh oh” with a dropped head, turning his head to the side, and looking (the only way I can describe it!) sad and disappointed. He learned “Leave it” very well, but I found that ever after, in any context, a soft “oh oh” would stop him in his tracks and have him look sad and disappointed. He really didn’t like making mistakes…


    • Hi Diana,

      That’s a great example of how difficult it can be to use an NRM without any emotional fallout. I have never intentionally played around with one, but I can see how easy it would be to create a scenario where a word or phrase became associated with a less than positive experience. We had a female border collie who was very sensitive and I had to be careful about my body language around her. I suspect she would have been one who reacted that way to an NRM, no matter how carefully it was applied. Thanks for sharing.



  3. I was hoping you’d post your notes. I know this is a lot of work for you. I so appreciate it!


  4. Thanks Terry. Glad you are enjoying them. Katie


  5. What drives me crazy about this issue when it’s raised is that the conversations about how this is mostly ineffective, but rarely offers NO explanation of useful alternatives. Perhaps that’s why this discussion on NRMs never seems to end. Also, anytime behavior is being decreased in favor of the behavior we want, then by definition some punishment no matter how mild, is occurring.

    Yes – good trainers need to plan ahead, record results, set up the environment and increase performance such that the chance of failure is decreased. But realistically the animal will fail in the process at some point. Then what? Without a proactive plan, what do you do? Ok, we know the NRM is not recommended. So what do we do instead? I almost never see that addressed and that’s always a huge disappointment to me.


    • *rarely offers an explanation of what to do instead.


    • Hi Summer,

      I agree that conversations about NRMs can be difficult. People seem to use them in different ways and how the dog responds is open to interpretation. Ken did mention that he meets a lotsof people who use them and think the dog is fine with it, but he sees a change when the NRM is used. This was a short talk (20 min) and Ken just covered the basic questions, but imbedded within the points were a few suggestions about alternatives. He didn’t list them as such, but he did mention that you can interrupt an “error” by using a cue or redirection. I also know, from past presentations with him, that he uses an LRS.

      My take away from the talk was that if you are going to use an NRM that functions as a conditioned punisher, then there is probably going to be some fall-out – or you should at least acknowledge that it is a punisher. If you’re ok with that, then keep on using your NRM. If you don’t see any side effects, then maybe your NRM is not really a conditioned punisher (it could be functioning as something else), or maybe you are one of the few who can use them well.

      I’m not sure I agree that punishment is the only way to decrease behavior. Behavior also decreases through extinction. If I have two behaviors and I only reinforce one of them, the one I don’t reinforce is going to decrease without me applying punishment.

      Interesting stuff! This conference always makes me think. I’m sure Ken has a better answer to your question than I gave, but this is what I think, based on his presentation.


  6. I listened to Ken address this at Clicker Expo a few years back. I believe he said that the absence of the click is not lost on the dog and that the trainer should immediately ask for another, more predictable behavior or two before then asking for the missed one. He said most animals will then be successful.


    • Hi Pat, Yes, this is a good point and very relevant when we are training with a high rate of reinforcement. I think where it gets tricky is when you are putting together behaviors in a chain or doing several behaviors before clicking. In those cases, the animal has learned to keep going until it hears the click, and we don’t necessarily want it to make a change if the click is absent (delayed). Animals can certainly learn that the “absence” of a click means different things under different conditions, but I think that’s one situation where people seem to think a NRM is helpful. Thanks for your comment.


      • So the withholding of the reward due to the wrong choice is then considered negative punishment ?

        Is this correct ?

        What were Susan’s 4 NRMs?
        I’ve trained with her and she is very strict about not using NRMs so I’m very interested to hear what hers were.


      • Hi Ellie,

        I think in some cases, withholding the reward could be considered negative punishment, but I don’t think it’s accurate to say that is always the case. As usual, it depends upon the conditions. Ken was quite clear that he was not going to make a blanket statement about a NRM being P-, but he did say that he rarely sees it done without some fallout, which leads to some obvious conclusions.

        I am not familiar with Susan’s work so I only know what I shared in the article. Ken showed some videos and he did identify the NRM’s she was using (they were short phrases, if I remember correctly), but I didn’t write them down. I find it interesting that Susan doesn’t herself call them NRMs. Maybe she has a different term? Sorry I can’t be of more help. I’m sure either Susan or Ken would be able to clear this up for you.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s