Notes from the Art and Science of Animal Training Conference (ORCA): Dr. Paul Andronis – Adjunctive Behavior – So What Else Is Going On?

dog drinking

Dr. Paul Andronis is a professor at Northern Michigan University where he is an expert in the experimental and applied analysis of behavior.  In his presentation, he shared some information on “adjunctive” behavior both from an academic and a practical viewpoint. He discussed several varieties of adjunctive behavior, how it differs from other types of behavior, and the necessary conditions under which it occurs. He also talked about how it should be classified and why the concept of adjunctive behavior is relevant for animal trainers.

The history of adjunctive behavior

Adjunctive behavior was first identified in the laboratory as “schedule-induced” behavior (Falk 1961). It was observed in experiments where an animal was subjected to the same schedule or limited contingency over and over.

In this scenario, you would expect to see some regularity of behavior, and that the animal would become very efficient at doing what was necessary to earn reinforcement.  But, in some cases,  what they saw was that the animals were doing a lot of extra and unnecessary behavior.  The types of behaviors they observed, and how much these “extra” behaviors occurred, varied depending upon many factors. But they were often tied to reinforcement schedules or something about the environment, usually the experimental set-up.

Even though adjunctive behavior was not described until 1961, it’s likely that it had been present in other experiments and was ignored or not recognized as being of interest. Or perhaps it had occurred infrequently because the experimental set-up for many experiments had included some level of constraint or restraint that limited the animal’s behavior. Pavlov’s dogs were physically restrained and Skinner’s rats were placed in operant conditioning chambers that offered few options for alternate behaviors.  In any case, it was not until 1961, that adjunctive behavior started to receive attention.

Dr. Andronis did mention that it was around this time that Keller and Marion Breland published their article “The Misbehavior of Organisms.” The article described how some behaviors were difficult to train in certain animals if the behavior they wanted to train conflicted with a strong natural behavior. For example, the Brelands wanted to train a raccoon to put money in a slot, but the raccoon wanted to wash, or would go through washing actions with,  objects that it was given. Even though washing was not part of the behavior the Brelands wanted, or were intending to reinforce, they found it was very difficult, if not impossible, to eliminate it.  The fact that they could not control this “misbehavior” through a reinforcement contingency was used by some scientists to try and discredit operant work.

As part of the  discussion about how experimental set-up can influence results, Dr. Andronis talked about some of his own work training key pecking with pigeons. He said that, in one of his experiments, they had to use some older keys that were stiffer than the usual keys.  They found that the pigeons pecking the stiff keys produced a different pattern of behavior than the pigeons pecking lighter keys.  I think he was comparing results from two different experiments here, not saying they deliberately compared two types of keys.

When the pigeons were put on a fixed rate schedule,  they saw a nice scalloping pattern (surges of pecking followed by pauses) with the birds pecking the stiff keys.  But the birds pecking lighter keys showed a more consistent pecking behavior where they pecked at a more steady rate. The reason he shared this example was that the birds pecking the stiffer keys had time to do adjunctive behaviors because there were gaps in the pecking behavior. The bird with the lighter keys were busy pecking and less likely to do other behaviors. So, something as simple as key stiffness could make a difference in whether adjunctive behavior was likely to occur or not.

Adjunctive behavior in the literature

The history of adjunctive behavior can be difficult to trace because it goes by many different names in the literature, and while these names may refer to the same phenomena, they may also be used for similar but different phenomena as well.

Some of the most common names in the literature are:

  • schedule-induced behavior
  • Collateral responses
  • Adjunctive behavior
  • Ancillary behavior
  • Interim activities (or behavior)
  • Behavioral side-effects
  • Psychogenic behavior

Some of these terms are used more when referring to behaviors that are repeated because they end up being part of the behavior cycle (ABC cycle), even though they are not part of the criteria for reinforcement.  Other terms are more likely to be used to refer to behaviors that the animal does between opportunities for reinforcement, but that seem to be indicative of stress or frustration, such as might happen when the animal is placed on a lean reinforcement schedule.

For example, the term “collateral response” is usually used to describe behavior that is repeated because it gets reinforced along with the behavior the trainer wants the animal to do.  This might happen if the rat presses a lever, gets food,  and goes through a whole cycle of behavior before it is reinforced for lever pressing again.  From the animal’s perspective, it had to do the entire cycle to get reinforcement, because all the behavior in the cycle was reinforced along with the target behavior. The animal doesn’t know that it didn’t actually have to do it, it was just filling time.

On the other hand, behavioral side-effects or psychogenic behavior are terms that are more likely to be used to describe behaviors that occur at a higher frequency when an animal is stressed. These behaviors are usually not done intentionally by the animal.  Examples of these types of behavior are excessive defecation, urinating, and defecating.

I want to mention that professional animal trainers are unlikely to use the terms I listed above, which are used more in academic circles, but they certainly do recognize that adjunctive behaviors exist.  They just have their own names for them and may also have slightly different definitions.  Some common names for “extra” behavior that occurs when trying to train something else are displacement behaviors, stress or frustration behaviors or superstitious behaviors.

Okay, so with all this background information…

How do we define adjunctive behaviors?

Critical attributes of the concept.

An adjunctive behavior is a behavior that:

  1. Reliably accompanies another operant behavior targeted by experimenter-programmed contingencies;
  2. Is not explicitly required for meeting the requirements of those (E-programmed) contingencies;
  3. Is not reinforced (directly or adventitiously) by those contingencies maintaining the operant behavior they accompany; and
  4. Is occurring at rates considered excessive under the particular procedural arrangements.

Various types of adjunctive behaviors have been observed in laboratory settings. Here’s a list of some of them:

  • Polydipsia (excessive drinking)
  • Air-licking
  • Wheel running – common, but varies among species
  • Aggression – also fairly common
  • Bolt-pecking
  • Pica
  • Locomotor activity
  • Paw grooming
  • Defecation
  • Wing-flapping
  • “displacement preening”
  • Escape from schedule-requirements
  • Cigarette smoking (I hope this is among human subjects…)
  • Alcohol consumption (most animals avoid alcohol, but will drink it under certain conditions)
  • Chronic hypertension
  • Nitrogen “drinking”
  • Pellet “pouching”
  • Self-injection with nicotine

When and where does adjunctive behavior happen?

Most adjunctive behavior has been observed in the laboratory (is this suspicious?) and it has been studied in rats, pigeons, mice, hamsters, gerbils, humans, and rhesus macaques.

Dr. Andronis pointed out that adjunctive behavior is more likely to occur when animals are put on certain types of reinforcement schedules. This is where the name “schedule-induced behavior” comes from.  It is more likely to occur on schedules that have longer intervals between reinforcement, or when reinforcement is not delivered as expected.  It also tends to occur at predictable places in the schedule.

The type of schedule-induced behavior can tell you something about the underlying state of the animal.  An animal that is showing aggressive behavior is in a different state than one that thinks it has to do some extra body movement to earn reinforcement.

But, Dr. Andronis did point out, later in the presentation, that some schedule-induced behavior is simply behavior that has been reinforced as part of the reinforcement contingency.  So, even within the category of schedule-induced behavior, you can have adjunctive behavior that is operant, and is being done deliberately, or respondent, and is more a reflection of how the animal is feeling.

I think his point was that the type of behavior can be useful information, or not. You may have to collect additional information before you can interpret it. And of course, a behavior might initially occur because the animal is frustrated and then be maintained or increase because it is being reinforced.  One of the challenges of schedule-induced behavior is that it is probably the result of several variables, not just the schedule, and it’s not always clear how all the variables interact.

I think it’s interesting that most of the behaviors on the list above are mostly ones that you would not want to have happen during training, and some are ones you wouldn’t want to have happen at all, because they indicate that the animal is stressed.

Some examples of adjunctive behavior:

Polydipsia (excessive drinking):

    • Polydipsia was the schedule-induced behavior identified by J. Falk in 1961. He was studying rats and wanted to measure water intake under natural conditions. When he changed his reinforcement schedule to a F1, (1 minute between delivery of reinforcement), he saw more drinking. The rats would drink half their body weight in under one hour.

 

  • Dr. Andronis had an example of this with people living in a hospital.  The residents would walk the halls because they were bored.  At the end of each hall, there was a water fountain and they would stop to get a drink, probably just for something to do. This led to problems because the excessive urination affected their medicine and the hospital had to provide more activities to limit the hall walking and polydipsia behavior.

Aggression

    • In experiments where an animal becomes frustrated due to lack of expected reinforcement, there may be increased aggression toward other animals.

 

  • You may also see aggression due to resource guarding in experiments where food is delivered for lever pressing.  One animal may choose to guard the lever so another animal cannot have access to it.

Increase in Social Behaviors:

  • You can see schedule-induced changes in social behavior if you have multiple animals and multiple levers.  The animals may switch places to “help” another animal earn reinforcement.

Defecation:

  • Schedule -induced defecation has been observed in rats. In one experiment, the researcher noticed that the rats were producing more feces than expected, and also that they were defecating at unusual times.  He discovered that the amount of defecation could be manipulated by changing reinforcement schedules and was also affected by whether or not a water bottle was present.  (Rayfield, 1982)

An experiment that showed how to generate different kinds of adjunctive behavior

Along with the examples of types of adjunctive behavior, Dr. Andronis also described an experiment he did with pigeons.  In the experiment, he had pigeons pecking keys under three different conditions. They were:

  • Hard: the bird had to peck a lot to earn reinforcement
  • In-between: moderate amount of pecking
  • Easy: few pecks required to earn reinforcement

Interestingly, the “in-between” schedule was the one that produced either aggression or social behavior. On the easy schedule, the birds were not stressed. On the hard schedule, the birds were too busy to do anything else. But the in-between one created some frustration and also provided opportunities for other behaviors.

The experiment had several steps:

    1. He taught the birds the schedules. Each bird was in a cage with a lever.

 

    1. He added two new additional keys (in another location in the same cage) that allowed the bird to change the schedule. One key made the schedule harder, the other made it easier.  The pigeons all learned to set their own requirements to “easy.”

 

    1. He switched the function of the keys so the key that used to make it easier now made it harder.  This led to some frustration because the bird would choose the key for “easy” and get frustrated that it no longer worked as it had before. But, the pigeons did learn that when that happened, they should choose the other key. And they also learned that the functions of the keys would change and would anticipate the change and start to peck the other key even before he switched it.

 

    1. He set up a social experiment by placing two birds in side by side cages. I can’t remember the exact arrangement, but it was designed so that one bird could control the “difficulty” level for the other bird, meaning it got to decide if the other bird’s schedule was easy, in-between, or hard.

 

    1. He observed the behavior of the two birds.  He found that the bird who could control the other bird’s level of difficulty would consistently choose to peck the key that made it harder for the other bird to get food.

 

Main Laboratory findings

He provided a brief summary of what they have learned about adjunctive behavior from work in the laboratory.

    • It’s often a “post-reinforcement phenomenon” (it happens right after reinforcement when there is a delay before the next reinforcement is available)

 

    • Rates of occurrence vary as an “inverted-U” function of the inducing-schedule parameters. Rises over some schedule requirements, and then drops off, similar to a dose response curve.

 

    • They tried to show that it was not reinforced by inducing contingency.  Scientists would insert a COD (change over delay) to try and separate the adjunctive behavior from the reinforcement, so it was not accidentally reinforced.  But you don’t know how the animal experiences the COD and some scientists argue that a separation doesn’t mean there’s no effect.

 

    • Probably related to potentiating effect of inducing-schedule on SDs  or reinforcers specific to induced behaviors.

 

    • Idiosyncratic patterns of substitutability among induced behaviors.

 

Where it fits, in theory

Scientist like to know why behavior occurs, so once they identified and starting studying adjunctive behaviors, they came up with some theories about them.

Are they? 

    • Adventitiously-reinforced, superstitious behaviors? (Skinner). This is a reference to Skinner’s experiment on the development of superstitious behavior (1948).

 

    • A third class of behavior? (Wetherington, 1982), i.e., respondent, operant and contingency-induced. She concluded that they were not a third class of behavior and that trying to put behaviors into respondent/operant categories was problematic.

 

    • “Induced states” (Staddon) – purely a function of reinforcement schedules that induce certain behaviors, particularly drinking.

 

    • “Just plain operant behavior,” related to joint-environmental effects of programmed contingencies

 

Of these, Dr. Andronis favors the operant contingency relation.  If you consider that animals are balancing costs and balances all the time, it seems possible that there is an operant aspect to adjunctive behavior.

This doesn’t really tell us where they adjunctive behaviors come from, but the theory that adjunctive behavior is operant fits in with some other things we know about behavior and appears consistent with the innate response hierarchies posited by Lorenz and other ethologists, and with probabilistic model of Epstein in generativity theory.  According to Epstein’s theory, creativity or the generation of “novel behavior” is predictable and you can calculate probabilities for the different options.

Though the languages differ, both acknowledge:

    • Prior histories of occurrence for the behaviors involved;

 

    • Differential probabilities of the response classes in repertoire depend upon the presence of specific antecedent stimuli (“innate releasers” or SDs) and/or other potentiating variables (“motivational” variables);

 

    • When the currently highest probability behavior in the repertoire is momentarily interrupted (eg. Suppressed by punishment, rendered less likely by breaking the contingency, disrupted by changes in background stimuli, physically restrained or other prevented from occurring, etc.) the next most probably behavior in the repertoire occurs, made more probably particularly by its specific antecedent stimuli being present.

 

Ok, there’s a lot of jargon in there. As best I understood, what he was saying was that adjunctive behaviors are behaviors that already exist in the animal’s repertoire and they are “released” by stimuli in the environment under certain conditions.  One of the conditions under which this happens is if the animal’s ability to earn reinforcement is interrupted.

An example of an experiment that supports this theory was described by Dr. Joe Layng in the question and answer session after the presentation. In the first part of the experiment, he and Dr. Andronis taught pigeons to do a new behavior, one that was not a natural behavior for pigeons. The behavior they chose was head banging. Yes the pigeons would bang their heads against the wall for reinforcement. But don’t worry. they made them wear helmets. (Yes, this is true – he showed us a picture).

After they trained this behavior, they put the pigeons on a reinforcement schedule for pecking that was likely to produce schedule-induced (adjunctive) behavior. And what did the pigeons do for the adjunctive behavior? Head banging. Head banging was a behavior with a previous history of high reinforcement, so when the pecking was interrupted, it’s the behavior that re-appeared.

Increasing the likelihood of contingency adduction

In the experiment described above, Dr. Layng and Dr. Andronis intentionally introduced an undesirable behavior, but the same process can be used to produce new behaviors through “contingency adduction.”  This goes back to the idea that adjunctive behaviors are operant and can become part of the behavior.

It is related to Dr. Epstein’s work on creativity in which he showed that what we think of as being creative is often just a new combination of some known behaviors.   What you want to do is teach several behaviors separately and then set up conditions where the learner combines them.

He stated that:

    • Evocative environmental arrangements can introduce controlled variability into behavioral stream (canalization).

 

  • They can make possible (or highly likely) some novel combinations of existing and emerging repertoires that in turn meet new and more complex contingency requirements posed by the trainer.

He showed a picture of Freud’s office, which was filled with lots of weird stuff.  This makes it easy for the “patient” to do the ice breaking and is a way to evoke behaviors without doing a lot of prompting. It’s also useful because history affects how a stimulus is perceived, so an individual’s response to different stimuli can tell you a lot about what has happened to them in the past.

For animal trainers, the equivalent of this would be setting up the environment to get some natural behaviors going.  Behavior is never isolated. You want to be alert to instances when behavior happens systematically and take advantage of it.  Depending upon the type of behavior you want, you may want to have more controlled variables – for motor behavior, or you can leave it more open ended if you are looking for more creativity or teaching cognitive tasks.

Here are a few points from the question and answer session:

If you sometimes reinforce the adjunctive behavior, you will get it at even higher rates than would be expected.  (this is why a mis-timed click that lands on an adjunctive behavior can be such a set-back). It doesn’t take much reinforcement to maintain it, once you have an adjunctive behavior going.

Dr. Killeen said they have done work that showed that a delay is not a guarantee of separation. You can train a rat to press a lever and wait 30 seconds before reinforcement and they still learn to press the lever

2 thoughts on “Notes from the Art and Science of Animal Training Conference (ORCA): Dr. Paul Andronis – Adjunctive Behavior – So What Else Is Going On?

  1. Hi there! Just wanted to offer a slight correction – Dr. Andronis is actually a professor at Northern Michigan University, not University of Michigan. As his daughter, I just like looking out 🙂

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s