Training shelter volunteers to teach dog ... - Wiley Online Library

43 downloads 0 Views 642KB Size Report
behaviorally based dog training implemented by volunteers of an animal shelter. ... Shelter consumers rated the behavior change in dogs and trainers as ...
JOURNAL OF APPLIED BEHAVIOR ANALYSIS

2014, 47, 344–359

NUMBER

2 (SUMMER)

TRAINING SHELTER VOLUNTEERS TO TEACH DOG COMPLIANCE VERONICA J. HOWARD AND FLORENCE D. DIGENNARO REED UNIVERSITY OF KANSAS

This study examined the degree to which training procedures influenced the integrity of behaviorally based dog training implemented by volunteers of an animal shelter. Volunteers were taught to implement discrete-trial obedience training to teach 2 skills (sit and wait) to dogs. Procedural integrity during the baseline and written instructions conditions was low across all participants. Although performance increased with use of a video model, integrity did not reach criterion levels until performance feedback and modeling were provided. Moreover, the integrity of the discrete-trial training procedure was significantly and positively correlated with dog compliance to instructions for all dyads. Correct implementation and compliance were observed when participants were paired with a novel dog and trainer, respectively, although generalization of procedural integrity from the discrete-trial sit procedure to the discrete-trial wait procedure was not observed. Shelter consumers rated the behavior change in dogs and trainers as socially significant. Implications of these findings and future directions for research are discussed. Key words: volunteer training, treatment integrity, procedural integrity, dog training, discretetrial training, video modeling, modeling, feedback

Dogs with problem behavior may face a host of negative and potentially life-threatening outcomes. Research has documented a clear positive relation between problem behavior and relinquishment to shelters (Patronek, Glickman, Beck, McCabe, & Ecker, 1996) as well as longer stays in the shelter (Barnes, 1995; Wells & Hepper, 1992). Dogs that are deemed less adoptable because of problem behavior place significant burdens on nonprofit shelters (Zawistowski, Morris, Salman, & Ruch-Gallie, 1998), which increases the likelihood of behavior-related euthanasia. Although estimates vary, general consensus suggests that over half (52.5%) of all dogs that enter shelters will be euthanized (Patronek & Rowan, 1995). Even if a successful adoption is arranged, 68% of new owners report problem behavior, which accounts for nearly 90% of failed adoptions This study was conducted by the first author in partial fulfillment of the requirements for the MA in applied behavioral science at the University of Kansas. We thank the student participants, research assistants, and staff of the humane society for their assistance. Address correspondence to Veronica J. Howard, University of Alaska Anchorage, Psychology Department, 3211 Providence Drive, SSB352, Anchorage, Alaska 99608 (email: [email protected]). doi: 10.1002/jaba.120

(resulting in the dog being returned to the shelter; Wells & Hepper, 2000). These findings suggest that addressing dog problem behavior before adoption may reduce unsuccessful adoptions and potentially prevent behavior-related euthanasia. Enrichment programs may be one way to improve outcomes for shelter animals. These programs are designed to improve the welfare of confined animals and include, but are not limited to, environmental enrichment (e.g., toys, music), socialization (e.g., interactions with humans and other species), and simple obedience training. Training may produce significant benefit for shelter dogs (Wells, 2004). Enrichment programs with a training component have been shown to have a number of positive effects for dogs, including reductions in stress-related hormones (Coppola, Grandin, & Enns, 2006), improvements in immune function (Coppola et al., 2006), improved scores on a behavioral assessment (Menor-Campos, Molleda-Carbonell, & López-Rodríguez, 2011), and increased likelihood of adoption from the shelter (Luescher & Medlock, 2009). In addition to training behavior that potential adopters deem desirable, simple obedience training provides much needed one-on-one human contact and socialization for shelter pets, which is vital given their limited

344

VOLUNTEER TRAINING contact with and handling by humans (Hubrecht, Serpell, & Poole, 1992). Despite their many benefits, enrichment programs are sometimes cost prohibitive for nonprofit animal shelters (Segurson, 2009). One way to reduce the cost of enrichment programs for shelters is to use the labor of volunteers to implement the procedures. Unfortunately, there are very few experimental articles that examine the work behavior of volunteers, much less those who work with animals. Further, the limited studies that have documented performance improvements in volunteers have done so in a way that is cost prohibitive for most nonprofit organizations. For example, Johnson and Fawcett (1994) adopted a behavioral skills training package that require 4 to 6 hr of one-on-one training for each volunteer. These findings suggest that identifying less resource-intensive training methods to teach volunteers to provide simple obedience training to shelter dogs that lack compliance with basic commands is a worthwhile area of study. A number of empirically supported stafftraining techniques have potential relevance in nonprofit oragnizations that rely on the labor of volunteers to complete important tasks. For example, modeling techniques have been shown to improve staff performance (e.g., Nigro-Bruzzi & Sturmey, 2010; Sarokoff & Sturmey, 2004), but some forms may be more effective and efficient than others. For example, in vivo modeling may be described as more resource intensive because its success relies on the availability of a skilled trainer to model the procedures at the time of training, which may be difficult to accomplish in a setting with limited resources. Video modeling may be a less resource-intensive alternative to in vivo modeling and, when used over time, has been shown to improve staff implementation of various skills such as discrete-trial teaching (Catania, Almeida, Liu-Constant, & DiGennaro Reed, 2009), functional analysis sessions (Moore & Fisher, 2007), and respite care skills (Neef, Trachtenberg, Loeb, & Sterner, 1991). Video modeling has several advantages that might be

345

particularly beneficial when resources are limited: (a) standardization of the models, which may reduce variability in training; (b) ability to incorporate multiple stimulus and response exemplars, which may produce more robust training outcomes; and (c) ease of dissemination. As a training modality, video modeling may be particularly useful when resources are limited, because videos can be used repeatedly across trainees. However, video modeling does not always improve performance to criterion levels without the use of specific performance feedback (DiGennaro Reed, Codding, Catania, & Maguire, 2010). Given the importance of enrichment programs to shelter dogs and the reliance on volunteers to deliver these programs, our study had four goals. The first goal was to evaluate the level of training required for shelter volunteers to reach criterion performance when they implement a discrete-trial obedience training procedure to teach dogs to sit on instruction. Using sequential application, we evaluated the effects of several training methods on trainer integrity. The second goal was to evaluate the effects of obedience skill training on dog behavior. To accomplish this, we measured dog compliance to instruction during the various volunteer training conditions and calculated the correlation between the integrity of discrete-trial training and dog compliance with obedience instruction (DiGennaro, Martens, & Kleinmann, 2007). A third goal of the study was to assess the degree of generalization of volunteer performance and dog compliance to an untrained skill and to novel training partners. Because inappropriate dog behavior is associated with a variety of negative outcomes for shelter dogs, the fourth goal of this study was to measure consumer desirability of the dog before and after training, as well as the acceptability of the procedures used to train the dog. METHOD Participants and Setting Trainers were three undergraduate students recruited from an introductory course in applied

346

VERONICA J. HOWARD and FLORENCE D. DIGENNARO REED

behavior analysis at the University of Kansas. Each student expressed an interest in addressing problems at the humane society in exchange for extra credit. Students earned extra credit based on the total amount of time contributed to the project rather than trainer performance or dog compliance with instructions. Before the start of the study, informed consent was obtained from the trainers for their participation. Trainers were never left alone with the participating dogs, and the first author was present for all sessions to ensure the safety of dog–trainer dyads. Three shelter dogs were selected to participate in the study based on characteristics reported by shelter staff to be correlated with a longer-thanaverage stay in the shelter. These characteristics include (a) black fur or coat; (b) older than 1 year of age; (c) a history of limited socialization; and (d) having nonaggressive problem behavior that, if left untreated, may later lead to euthanasia in the shelter. These characteristics are consistent with the extant literature as being associated with a reduced likelihood of adoption (Posage, Bartlett, & Thomas, 1998). To minimize confounding effects caused by other shelter volunteers who conduct training with the participant dogs, only dogs housed in areas of the shelter that were off-limits to volunteers were considered for inclusion in the study. Additional inclusionary criteria included a lack of instructional control over target skills. During prebaseline probes, participating dogs were brought to the training setting and given the target instructions (i.e., sit and wait) using verbal commands three times without programmed consequences. The dogs did not respond to probe instructions, suggesting that these behaviors were not under instructional control. Dogs and trainers were assigned to dyads (as each became available) that remained the same throughout the course of the study (excluding novel-stimulus assessment probes). That is, each participant trained a single dog, and each dog was trained by a single participant. Dyad A. Trainer A was a 21-year-old female undergraduate majoring in applied behavioral

science. She reported that her family owned dogs, and she had some casual experience training simple skills but no formal history of dog training. She was assigned to train Dog A, an altered 1-year-old male black labrador retriever. Five weeks before the start of the study, Dog A was surrendered to the shelter by owners who moved and, according to an owner survey completed at the time of surrender, had no room or time for him. Dog A had a history of living outdoors, and previous owners reported that he received little training before the start of the study. Dyad B. Trainer B was a 20-year-old female undergraduate majoring in photo media and had no formal dog-training experience. She was assigned to train Dog B, an altered male black labrador retriever who was 1 year old at the beginning of the study. Dog B had been surrendered with his brother Dog A (above) and had experienced similar prestudy environmental conditions. Dogs A and B were housed together during the course of the study and were identified by different colored collars. Visual inspection of each dog before training confirmed each dog’s identity. Dyad C. Trainer C was an 18-year-old female undergraduate with an undecided major who reported no history of dog training prior to the study. She was assigned to train Dog C, an unaltered female black labrador retriever who was 1.5 years of age at the beginning of the study. Dog C had been previously adopted from the present humane society, but was subsequently returned due to destructive in-home behavior when left unattended. The study was conducted at a large Midwestern humane society. On average, the humane society housed up to 500 dogs at a time. The dogs in the current study were housed in kennels (1.1 m by 2.6 m). Each trainer was instructed to collect her dog from the kennel and allow it to exercise in a fenced outdoor exercise area (18.9 m by 49.4 m) for 10 min before each training session. Training sessions were conducted in an

VOLUNTEER TRAINING unused storage area (2.2 m by 4.2 m) to minimize distractions. Only the first author, participant trainer, and participant’s dog were present during sessions. A video camera was used to record all sessions for subsequent data analysis. All necessary training supplies (writing implements, observation forms, edible reinforcers, and leashes) were stored in the training room and were available to the trainer for every session. Dependent Variables and Data Analysis Procedural integrity. The primary dependent variable was the integrity with which the trainers implemented the prescribed nine-step discretetrial obedience training (DTOT) procedure adapted from Catania et al. (2009) and informed by obedience training literature (Braem & Mills, 2010; Burch & Bailey, 1999). A task analysis of the DTOT procedure designed to teach the sit skill (DTOT-S) consisted of the following steps: (a) Wait for dog-ready behavior (i.e., dog is disengaged from other activities and standing), (b) arrange antecedents for instruction (i.e., trainer is standing up straight, dog and trainer are face to face), (c) deliver scripted verbal instruction, (d) deliver instruction once with clear articulation, (e) pair verbal instruction with discriminative hand cue or prompt, (f ) deliver an edible reward contingent on target behavior or conduct increasingly intrusive correction trials (using gestural then physical prompting) for an incorrect or no response, (g) correctly deliver differential reinforcement, (h) record data following each trial, and (i) place a 5-s intertrial interval between trials. Procedural integrity was calculated as the number of training steps performed correctly divided by the number of steps possible during the session and converting the result to a percentage. Generalization of training effects. Performance of the dogs and trainers on a second skill (wait in position for 3 s when presented with a treat) was intermittently probed to assess for possible generalization from the target skill. Instructions, feedback, and modeling were not provided for the

347

generalization skill until just prior to the end of the study. The steps of the discrete-trial obedience training for wait (DTOT-W) procedure were identical to the DTOT-S procedure, although the target instruction and hand cue were individualized to the wait skill. Procedural integrity was calculated as the number of training steps performed correctly divided by the number of steps and converting the result to a percentage. Treatment effectiveness. To measure the effectiveness of DTOT, data were collected on the frequency of dog compliance with verbal instructions for the target (sit) and generalization (wait) skills. Compliance for sit behavior was defined as the dog’s hindquarters lowered to the floor within 3 s after the verbal instruction while the upper body remained in an upright position. Compliance for wait behavior was defined as the dog maintaining proximity to the trainer but making no forward movement toward the treat for 3 s after the wait instruction. Target skills emitted in the absence of explicit instruction or that occurred following a gestural or physical prompt did not meet criterion and were not scored as instances of compliance. Compliance was calculated as the number of trials in which the dog exhibited compliance to verbal instruction divided by the number of trials and converting the result to a percentage. Correlational analysis. The correlation between the integrity of DTOT-S and dog compliance was calculated to determine the degree to which trainer integrity was associated with dog-training effectiveness. Because the distribution of results did not meet the assumptions of parametric analysis, a nonparametric statistic was employed (i.e., Spearman’s rank order coefficient, rho). Treatment acceptability and social validity. At the end of the study, trainers were asked to complete a 16-item questionnaire informed by the Intervention Rating Profile-15 (Martens, Witt, Elliott, & Darveaux, 1985) to assess the acceptability of the training methods used to teach DTOT. Thirteen of the questions asked

348

VERONICA J. HOWARD and FLORENCE D. DIGENNARO REED

respondents to rate statements on a 6-point Likert-type scale (1 ¼ strongly disagree, 6 ¼ strongly agree), and three items were free response openended questions. The trainers were also provided an opportunity to give qualitative feedback and suggest improvements for the training they received. To assess further the social validity of the goals, procedure, and results (Wolf, 1978), we recruited potential adopters, shelter staff, and animal trainers employed by or volunteering at the shelter. In addition, we contacted professional trainers in Kansas, Missouri, and Nebraska who held at least one of the following certifications: Canine Good Citizenship Test Evaluator, Animal Behavior College Certified Dog Trainer, and trainers certified through the Certification Council for Professional Dog Trainers holding either Knowledge Assessed or Skills Assessed certifications. Because no Certified Applied Animal Behaviorists lived in this area, we recruited them across the United States via e-mail. Due to the low number of National Association of Dog Obedience Instructors living in this area, a nationwide sample of these instructors was also included. Consumers were shown brief samples of a video of Dyad B in baseline (before training, when DTOT-S procedural integrity was low) and during the feedback and modeling condition (after training, when DTOT-S procedural integrity was high) for the sit skill before they answered a 9-item survey. For four items, consumers were asked to rate the effectiveness and acceptability of the training procedure used by the trainer in each video on a 6-point Likert-type scale (1 ¼ strongly disagree, 6 ¼ strongly agree). The remaining questions asked consumers to select which video (before or after training) they considered “better” along five dimensions: (a) effectiveness of trainer, (b) desirability of trainer, (c) adoptability of dog, (d) which dog would be better with children, and (e) which dog would be better for a first-time pet owner. Potential adopters, shelter staff, and shelter trainers watched the videos and completed

the questionnaire at the shelter. Consumers who had been recruited via e-mail watched the videos online and completed the questionnaire via SurveyMonkey. Design and Procedure A concurrent multiple baseline design across three trainer–dog dyads was used to evaluate the effectiveness of training on trainer procedural integrity and dog compliance. To identify the most effective training with the least amount of resources, sequential application of increasingly intensive methods of volunteer training was evaluated. The analysis consisted of the following phases: (a) baseline, (b) written instructions, (c) video modeling, (d) feedback and modeling for sit, (e) feedback and modeling for wait, and (f ) novel stimulus assessment. Each training session consisted of the 10 training trials presented to each dog. (Due to a technical error, only nine wait instructions were recorded for Trainer A during the first observation of the feedback and modeling for wait condition.) If a trainer delivered more than 10 instructions during a session, she was told to stop training to allow the dog a break; only the first 10 trials were scored. One to four training sessions were scheduled per day, 2 to 3 days per week, based on the availability of the training room, participants, and researcher. Participating dogs were provided a brief rest between sessions (approximately 5 min), with access to water. Baseline. Given the trainers’ recent completion of an introductory behavior analysis course, we included a naive baseline to assess their level of skill before procedures to train DTOT-S were introduced. Trainers were provided with basic instructions regarding how to safely handle shelter pets and how to prepare for training sessions. Trainers were not given instructions with respect to the procedures used to teach the dogs the target skill. Trainers were instructed to use their discretion to teach their assigned dog to sit and wait on instruction using only nonaversive training methods. Trainers were instructed “do

VOLUNTEER TRAINING your best” if they asked for clarification or feedback regarding performance. At the end of each day, trainers were given general praise (e.g., “thank you” or “thank you for your work today!”) for their participation to help guard against extinction-induced attrition. Written instructions for sit. The purpose of this phase was to evaluate the effect of written instructions on trainer integrity of DTOT-S and, subsequently, dog compliance. Written instructions are a common training technique used in shelters with limited resources due to their low cost of training per capita. Detailed written instructions were developed for this study and included a rationale for DTOT-S and step-bystep instructions for correct implementation of all elements of the procedure. Trainers received a copy of the written instructions 30 min before the first observation of the condition and were asked to read the instructions before the session began. All trainers were observed to read the instructions. Trainers could ask questions to clarify the written instructions but were given no further instructions or feedback. The trainers were allowed to keep the written instructions and were encouraged to refer to them any time outside the training session, but they were not required to review these instructions before each session. Feedback was not provided, but general praise was given at the end of each training day, as in baseline. Video model for sit. The purpose of this phase was to assess the effect of a video model that contained descriptions and recorded models of correct and incorrect training methods on the integrity with which the trainer implemented DTOT-S. Before the first session, trainers viewed a 14-min 14-s video model of DTOT-S. The video model included detailed, step-by-step written and verbal instructions for each step of the sit training procedure, as well as video footage of correct and incorrect examples of DTOT-S performed by a student assistant with a nonparticipating dog. Trainers viewed the video only once at the start of this condition but were

349

permitted to watch any or all of the video at any time during the initial observation and ask questions to clarify the information presented. Trainer A asked the researcher to clarify the hand shape shown in the video, and Trainer B reviewed a portion of the training video that was difficult to hear due to shelter commotion (Steps 7 through 9). There were no additional questions or requests to review the video. Trainers received general praise at the end of each training day as in earlier conditions. Feedback and modeling for sit. The purpose of this phase was to increase trainer performance of the sit skill to criterion levels (90% procedural integrity or above). Before the first observation, trainers received a brief one-on-one training session that consisted of descriptive praise, feedback, and modeling until criterion performance of 100% integrity was met during a roleplay. Descriptive praise consisted of identification of steps performed correctly combined with a praise statement (e.g., “You’re approaching the dog and getting her attention before giving the instruction; that’s great!”). Feedback consisted of identifying which steps of the procedure had been consistently performed incorrectly in the previous condition, an explanation as to why these steps were incorrect, and modeling how to implement the steps correctly. We asked the trainer to perform these steps in a role-play situation assuming both compliant and noncompliant dog behavior. The dog was present in the room but was not used as a modeling aid to minimize the confounding effect that this might have on the dog’s compliance with instructions. If a step was performed incorrectly during the roleplay, additional feedback and modeling were provided until the trainer implemented all steps of DTOT-S correctly during a role-play. At this point, one-to-one training ended, and data collection resumed. After each session in this condition, praise was given for steps performed correctly, and feedback and modeling were provided for any steps performed incorrectly only until criterion

350

VERONICA J. HOWARD and FLORENCE D. DIGENNARO REED

performance was achieved. After trainers achieved criterion performance during a session, we no longer provided specific feedback. However, generalized praise was given at the end of data collection each day. Feedback and modeling for wait. After criterion performance of DTOT-S had been achieved, feedback and modeling were provided for trainer performance of DTOT-W. Due to time constraints at the end of the study, criterion performance of the wait skill was reached by only two of the three trainers before they moved on to the final condition of the study. Novel stimulus assessment. A final probe was conducted to examine whether trainers could implement DTOT-S and DTOT-W with integrity when given a novel dog and whether dogs would continue to comply with instructions given by a novel trainer. To assess trainer skills, two novel dogs were selected from healthy, adoptable shelter dogs. The first author’s foster dog was used to assess skill generalization of one trainer because of a medical quarantine of adoptable dogs. These novel dogs were capable of complying with instructions, but did so inconsistently during pretraining probes. To assess dog compliance with a novel trainer, one student research assistant and one shelter staff member were recruited to act as trainers. Novel trainers had experience delivering obedience training and were also given brief training before the novel stimulus assessment was conducted. Probes were assessed in the room where previous training sessions occurred 1 to 3 days after the last observation during feedback and modeling. Interobserver Agreement Two independent observers collected data on integrity and dog compliance for 37% of sessions across all conditions for the sit instruction and for 43% of probes across all conditions for the wait instruction. The observers were in agreement when they independently scored the trainer’s implementation of a step of the DTOT procedure identically (i.e., as correct, incorrect, or not

applicable). Interobserver agreement was calculated as the number of instances of agreement divided by agreements plus disagreements, and the result was converted to a percentage. Mean agreement for DTOT-S for Trainer A was 91% (range, 80% to 96%), for Trainer B was 88% (range, 81% to 94%), and for Trainer C was 89% (range, 81% to 100%). Mean agreement for DTOT-W for Trainer A was 95% (range, 91% to 98%), for Trainer B was 87% (range, 82% to 95%), and for Trainer C was 85% (range, 83% to 89%). For interobserver agreement for dog compliance, the observers were in agreement when they independently scored dog behavior identically (i.e., compliance did or did not occur). Agreement was calculated as the number of instances of agreement divided by agreements plus disagreements, and the result was converted to a percentage. Mean agreement on compliance with the sit instruction for Dog A was 100% across all conditions, for Dog B was 99% (range, 90% to 100%), and for Dog C was 96% (range, 80% to 100%). Mean agreement on the wait instruction for Dog A was 87% (range, 80% to 100%), for Dog B was 92% (range, 80% to 100%), and for Dog C was 90% (range, 80% to 100%). RESULTS Procedural Integrity and Effectiveness Sit training. Figure 1 depicts trainer (percentage of DTOT-S integrity) and dog (percentage of compliance with target instruction) behavior, organized by training dyad. Overall, the procedures used by trainers in baseline did not share many similarities with the DTOT-S procedure (Trainer A: M ¼ 25%; Trainer B: M ¼ 33%; Trainer C: M ¼ 20%). Dog compliance was also low during baseline (Dog A: M ¼ 0%; Dog B: M ¼ 1%; Dog C: M ¼ 10%). Written instructions produced little to no improvements in trainers’ performance of DTOT-S (Trainer A: M ¼ 17%; Trainer B: M ¼ 42%; Trainer C: M ¼ 28%). Dog compliance remained low when

VOLUNTEER TRAINING

351

Figure 1. Trainer integrity of discrete-trial obedience training for sit (DTOT-S) and dog compliance with sit instructions.

trainers were given written instructions (Dog A: M ¼ 32%; Dog B: M ¼ 32%; Dog C: M ¼ 0%); however, Dogs A and B demonstrated greater compliance in later observations within this condition. A single viewing of the video model produced marked improvement in the correct implementation of the training procedure for all trainers; however, performance remained below the criterion level of 90% integrity (Trainer A: M ¼ 62%; Trainer B: M ¼ 71%; Trainer C: M ¼ 59%). Dog compliance with instruction

increased in this condition, but was highly variable (Dog A: M ¼ 86%; Dog B: M ¼ 51%; Dog C: M ¼ 47.5%). The highest levels of integrity were observed during feedback and modeling (Trainer A: M ¼ 88%; Trainer B: M ¼ 97%; Trainer C: M ¼ 93%). Dog compliance with the sit instruction also increased to its highest levels during feedback and modeling for sit (Dog A: M ¼ 90%; Dog B: M ¼ 83%; Dog C: M ¼ 100%). During a probe with a novel dog, the trainers’ integrity of DTOT-S was generally

352

VERONICA J. HOWARD and FLORENCE D. DIGENNARO REED

Figure 2. Trainer integrity of discrete-trial obedience training for wait (DTOT-W) and dog compliance with wait instructions.

high (Trainer A: 100%; Trainer B: 81%; Trainer C: 100%). During a probe with a novel trainer, dog compliance with the sit instruction also remained high (Dog A: 80%; Dog B: 90%; Dog C: 100%). Generalization (wait training). Figure 2 depicts trainer (percentage of DTOT-W integrity) and dog (percentage compliance with target instruction) behavior, organized by training dyad. Dashed horizontal lines show the average performance of DTOT-S for each condition. During baseline probes, the procedures used by

trainers did not share many similarities with DTOT-W (Trainer A: M ¼ 22%; Trainer B: M ¼ 19%; Trainer C: M ¼ 29%). Dogs were noncompliant with all wait instructions in this condition. No change in the trainers’ behavior was evident when written instructions were provided for DTOT-S, and the integrity of implementation for all trainers remained low and stable during generalization probes when trainers received written instructions for DTOT-S, despite the similarities between DTOT-S and DTOT-W (Trainer A: M ¼ 27%; Trainer B:

VOLUNTEER TRAINING M ¼ 24%; Trainer C: M ¼ 36%). Dogs were noncompliant with all but one wait instruction in this condition. Effects of video modeling and individualized feedback on DTOT-S also did not generalize to the implementation of DTOT-W (Trainer A: Ms ¼ 35% and 37%; Trainer B: Ms ¼ 43% and 55%; Trainer C: Ms ¼ 44% and 40%, respectively). All dogs were noncompliant with all wait instructions during video modeling for sit, but performance improved slightly for Dog B during individualized feedback for sit (Dog A: M ¼ 0%; Dog B: M ¼ 20%; Dog C: M ¼ 0%). When the trainers’ integrity of DTOTW was directly targeted during feedback and modeling for wait, the performance of all trainers improved. However, criterion performance of 90% was met during a single probe for only one trainer (Trainer C: M ¼86%). Due to time limitations, observations ended before criterion performance was achieved for Trainers A and B (Trainer A: M ¼ 80%; Trainer B: M ¼ 75%). Dog compliance with the wait instruction during this condition improved for two dogs (Dog B: M ¼ 70%; Dog C: M ¼ 77%) and produced some improvement for Dog A (M ¼ 20%). Two of three trainers displayed similar performance when they implemented DTOT-W with a novel dog during a single session probe as they had with their training dog (Trainer B: 67%; Trainer C: 88%), although the procedural integrity for Trainer A declined substantially (Trainer A: 49%). During a probe with a novel trainer, dog compliance with the wait instruction remained consistent with performance observed during feedback and modeling for wait. Dogs A, B, and C were compliant with 80%, 60%, and 90%, respectively, of wait instructions delivered by a novel trainer. Correlational Analyses The correlation between the integrity of DTOT-S and dog compliance was calculated to determine the degree to which trainer integrity was associated with dog-training effectiveness. A nonparametric test, Spearman’s rank order

353

coefficient, was used. The integrity of DTOT-S and dog compliance was significantly correlated for Dyad A, rs ¼ .84, p < .01; Dyad B, rs ¼ .88, p < .01; and Dyad C, rs ¼ .86, p < .01. Training Acceptability and Social Validity Trainers (participants). Trainers rated the effectiveness and acceptability of the procedures used to train them on a 6-point Likert-type scale (1 ¼ strongly disagree, 6 ¼ strongly agree). Ratings of the effectiveness of written instructions were conservative (M ¼ 4.7, SD ¼ 1.2). All trainers indicated that both video modeling and the combination of feedback and modeling (M ¼ 6, SD ¼ 0) effectively taught them to implement the training procedure with the participating dogs. The trainers also indicated that they would recommend the use of the video model with other volunteers (M ¼ 6, SD ¼ 0). However, all trainers identified feedback and modeling as the element of the training package that was most effective. All trainers reported that they would be willing to receive feedback again in the future (M ¼ 6, SD ¼ 0). Trainers indicated that they disagreed that the dog would have learned to sit without instruction (M ¼ 1.67, SD ¼ 0.58), and disagreed that they would have quickly and easily taught the dogs to sit without DTOT-S (M ¼ 2.3, SD ¼ 0.58). All trainers agreed or strongly agreed that they would use DTOT-S again in the future (M ¼ 5.7, SD ¼ 0.58) and that they would prefer to use DTOT-S rather than their own baseline training procedure if they continued to train shelter dogs. Trainers disagreed that they would have given up on teaching the dogs if they were not receiving class credit (M ¼ 2, SD ¼ 1). Ratings of whether the dog’s slow rate of learning was frustrating were mixed (M ¼ 3.7, SD ¼ 1.25), with some trainers agreeing and some disagreeing with the statement. Shelter consumers and professional trainers. From a pool of 604 recruited certified trainers, 189 (34%) completed the online social validity survey. Of those respondents who completed the

354

VERONICA J. HOWARD and FLORENCE D. DIGENNARO REED Table 1 Consumer Ratings

Rank items I thought the [baseline] training procedure was safe for both dog and trainer. I thought the [baseline] training procedure was effective. I thought the [DTOT-S] training procedure was safe for both dog and trainer. I thought the [DTOT-S] training procedure was effective.

Potential adopters M SD

Shelter staff M SD

Professional trainers M SD

Total M

SD

3.34

1.41

2.67

0.47

2.64

1.35

2.83

1.39

1.89

0.96

1.67

0.47

1.19

0.62

1.38

0.78

5.77

0.45

5.67

0.47

4.74

1.18

5.03

1.13

5.73

0.58

5.67

0.47

4.56

1.27

4.89

1.23

Forced-choice questions

n

%

n

%

n

%

n

%

Which trainer was more effective at teaching the skill “sit”? Which trainer would you rather have training your dog, clients’ dog, or shelter dog? If both dogs’ behavior remained the same, which dog is more adoptable? If both dogs’ behavior remained the same, which dog do you think would be the best pet in a home with children? If both dogs’ behavior remained the same, which dog do you think would be the best for first time dog owners?

71

100

3

100

186

100

260

100

70

98.6

3

100

186

100

259

99.6

69

97

3

100

183

98.4

255

98.1

69

97

3

100

178

95.7

250

96.2

71

100

3

100

183

98.4

257

98.8

survey, seven (4%) failed to rate the safety and effectiveness of the procedures before and after training and were omitted from analysis, which resulted in a sample of 182 certified dog trainers. In addition, we recruited 71 potential adopters, three shelter staff, and four volunteer or paid dog trainers who worked at the shelter to complete a paper version of the survey. All respondents rated the safety and effectiveness of the training procedures used during baseline and feedback and modeling on a 6-point Likert-type scale (1 ¼ strongly disagree, 6 ¼ strongly agree). The items for this section were presented as affirming statements (e.g., “I thought the training in this video was effective.”). Lower ratings are indicative of ratings of disagreements with these statements, or may be conceptualized as low ratings of the variable being rated (i.e., effectiveness). Respondents were also asked to select

which dyad was better along a number of dimensions (see forced-choice questions in Table 1). Table 1 displays the mean ratings of the effectiveness and acceptability of the training procedures as well as the number and percentage of respondents who selected the dyad in the highintegrity DTOT-S condition as the better dyad. Respondents rated the effectiveness of the training procedures consistently. The procedure used by trainers during baseline (when trainers used their own methods to train the dog) was rated as ineffective and more than slightly unsafe. DTOT-S used with high integrity (i.e., during the modeling and feedback condition) was rated as both effective and safe for dog and trainer. Raters selected the trainer who implemented DTOT-S with high integrity as the more effective and the more desirable trainer. Raters also selected the dog who

VOLUNTEER TRAINING demonstrated high levels of instructional compliance as more adoptable, better in a home with children, and a better dog for first-time pet owners. DISCUSSION The primary goal of this study was to evaluate the effects of sequential training methods on the procedural integrity of volunteer-implemented obedience training for shelter dogs. Procedural integrity was low for all trainers during baseline and did not improve with the addition of written instructions. Although integrity improved with use of the video model, it did not reach criterion levels until feedback and modeling were provided for steps that had been omitted or performed incorrectly. The results of this study support three key findings from previous staff-training literature: (a) Written instructions alone are insufficient to produce desired changes in staff behavior (Ducharme & Feldman, 1992; Gardner, 1972; Krumhus & Malott, 1980; Roscoe & Fisher, 2008); (b) video modeling alone may be insufficient to produce desired outcomes (DiGennaro Reed et al., 2010); and (c) in vivo modeling (Nigro-Bruzzi & Sturmey, 2010; Sarokoff & Sturmey, 2008) and performance feedback (Alvero, Bucklin, & Austin, 2001) produce robust improvements in integrity. Our findings make several contributions to the volunteer-training literature. First, and most important, the current study directly examined the effects of training on the work behavior of volunteer trainers. Experimental analysis of volunteer work behavior has been underexplored in the literature, and our findings suggest that training methods currently used exclusively in staff-training literature may also be effective to train volunteers to implement procedures with integrity. If the training techniques that are effective for paid staff can also be applied to unpaid staff, the number of empirically supported training and supervision methods available for volunteer managers increases dramatically. This study also challenges current

355

misconceptions about volunteer performance (“you get what you pay for”; Pearce, 1993; Wharton, 1991) by providing evidence that unpaid staff may be able to implement procedures with integrity when given training and support. Whether the performances of participants in the current study are analogous to paid or unpaid employees is unclear. Participants were university students who received extra credit for participation, which may be viewed as a form of payment. However, the students were clearly informed that their participation credit was based on time contributed rather than on the dog’s performance, and the amount of time required to participate in the project was disproportionate to the amount of extra credit earned (e.g., 0.5% per half hour at the shelter, not including travel time). Future research should evaluate the effects of training on the performance of volunteers who do not receive extra credit for a university course. Results of the current study may also inform volunteer-training methods used in animal shelters, where identifying training methods that can produce skillful volunteer work behavior while still being cost effective for the organization is an important goal. Careful use of personnel and financial resources through a resource-efficient and effective training package may foster sustainability of training programs, possibly lowering volunteer turnover and better promoting the mission of the shelter. These variables were not measured in the present study and should be a focus of future research. For example, researchers could measure the long-term use of a video-based training package as an index of sustainability and social validity. Moreover, shelter administrators could also measure volunteer turnover before and after changes to volunteer training, improvements in dog performance as a function of volunteer training (e.g., sit on instruction, appropriate leash walking), rates of successful adoptions, and rates of and latency to adoption. These measures reflect important shelter and dog outcomes. We encourage future

356

VERONICA J. HOWARD and FLORENCE D. DIGENNARO REED

researchers to conduct systems-level analyses so that we may draw firm conclusions about the relations among training components, skill areas addressed, and outcomes for dogs, volunteers, and shelters. Organizations could adopt a preservice training package that uses video modeling to promote the efficiency of training resources while the effectiveness of volunteers is maximized. Although the materials used in this study required an investment in training resources (e.g., 3 hr to develop written instructions, 15 hr to develop the video model) and required access to specialized equipment (mass-market handi-cam [$100], free or low-cost video software, and a personal computer [$500]), they may represent a wise investment when used over time, because the same materials may be used repeatedly. These costs could be further ameliorated by using volunteer labor to develop and create the training materials. With little upfront cost, an organization could distribute a preservice training package consisting of written instructions and a video model on DVD or hosted online (e.g., YouTube or College of Direct Support) to interested volunteers. When volunteers arrive on-site, organizations could proceed with direct observation, feedback, and modeling until criterion performance is met. This sequence of training may be more cost effective than delivering the same training using sessions of didactic instruction or workshops individually or in groups followed by individualized feedback and modeling. Furthermore, this type of indirect training may also be more convenient and accessible to volunteers than scheduled workshops. Future research should evaluate the effects of indirect preservice training combined with direct observation, feedback, and modeling on volunteer performance, longevity, and volunteer satisfaction. With regard to efficiency, it appears that the trainers required half as many feedback sessions for DTOT-S, for which they received written instructions and video models, than were necessary to improve performance for DTOT-

W. We speculate that the sequential application of increasingly intensive training methods decreased the amount of face-to-face training time required to reach criterion performance during feedback and modeling; however, we did not measure the duration of feedback and did not study this particular issue. Thus, we are unable to draw conclusions about the cost effectiveness of these training procedures. Future researchers may wish to conduct cost analyses of training methods to determine the most cost-effective method of training volunteers to criterion performance. Although we used a sequential training protocol to determine whether less resourceintensive training methods could produce satisfactory improvements in integrity, procedural variations in our methodology likely influenced our findings and may not be consistent with actual use in applied settings. That is, we intentionally arranged our training conditions to emulate environments with limited resources similar to the setting in which we conducted the study (e.g., distributed written instructions alone, volunteer trainers accessed the video model only once). It is possible that different variations of the training elements (e.g., allowing access to written instructions during sessions, providing additional exemplars or repeated viewings of the video model, combining the training elements into a package delivered simultaneously at the outset of training) would have produced higher trainer integrity sooner and is an area worthy of investigation. The second goal of this study was to evaluate the effects of obedience skill training on dog behavior. Although some studies have evaluated the effects of training on dog compliance (Braem & Mills, 2010; Thorn, Templeton, Van Winkle, & Castillo, 2006), the current study is novel in several ways. First, our results demonstrated that poor volunteer-training integrity produces little to no instructional compliance (i.e., a lack of dog acquisition of sit or wait). Our findings also showed that dog acquisition was accomplished only when training integrity improved. Second,

VOLUNTEER TRAINING the current study calculated the correlation between training integrity and instructional compliance. Integrity of DTOT-S was significantly and positively correlated with dog compliance to sit with instructions. That is, high trainer integrity of DTOT-S was associated with high dog compliance for all dyads. Third, this study evaluated the effects of training with shelter dogs, an understudied population in animal-training research. The third goal of this study was to evaluate generalization across training partners (i.e., novel trainers and dogs) and skills (i.e., wait on instruction). Although generalization across trainers and dogs was observed during the novel stimulus assessment, trainers did not generalize skillful performance from DTOT-S to DTOT-W. Our findings suggest that direct training in DTOT-W was necessary for trainers to implement DTOT-W with integrity. Despite similarities in several of the training steps of DTOT-S and DTOT-W, it appears that the conditions that promoted integrity of DTOT-S (i.e., the volunteer training sequence) may be necessary to ensure correct implementation of DTOT-W. In addition, our analyses revealed a large proportion of trainer commission errors during DTOT-W implementation in the form of delivering reinforcement early (i.e., the trainers did not require the dog to wait for the full 3 s). It is possible that the dogs were capable of waiting on instruction but were not granted the opportunity to demonstrate this skill due to this error. Consequently, increases in dog compliance during feedback and modeling for wait may be due to a reduction of trainer commission errors rather than actual improvement in dog compliance. Future research should evaluate the conditions necessary to teach trainers to implement the DTOT-W effectively and identify a resourceefficient video-based training and feedback package to teach volunteers to implement both DTOT-S and DTOT-W with high integrity. Our final goal was to produce socially significant changes in dog behavior using

357

procedures that are acceptable to consumers. Acceptability, as well as the desirability of the dog before and after training, were measured via social validity surveys. Consumer ratings of DTOT-S and dog performance after training were favorable. The dog that displayed instructional compliance was identified as “better” and “more adoptable” than the same dog that displayed jumping and instructional noncompliance. These results extend previous literature by demonstrating that the behavior of shelter dogs influences the verbal reports of adoptability by consumers (adopters, shelter staff, and trainers), with low rates of desirability associated with noncompliance and high rates of desirability associated with instructional compliance. These results are consistent with previously reported correlations between problem behavior and decreased likelihood of adoption (Lepper, Kass, & Hart, 2002). The current study suggests that simple obedience training, as part of an enrichment program, can increase instructional compliance. Simple skill training can improve the dog’s quality of life by teaching appropriate behavior to contact reinforcement (Leuscher & Medlock, 2009) but may also increase the likelihood of adoption. Thus, an important line of future research involves (a) identification of the behavior of dogs that improves adoption outcomes, (b) evaluation of dog-training procedures and environmental enrichment to teach those behaviors, and (c) evaluation of resource-efficient training technology that trains volunteers to teach dogs to perform those skills. REFERENCES Alvero, A. M., Bucklin, B. R., & Austin, J. (2001). An objective review of the effectiveness and essential characteristics of performance feedback in organizational settings (1985–1998). Journal of Organizational Behavior Management, 21, 3–29. doi: 10.1300/ J075v21n01_02 Barnes, D. D. (1995). Retrospective cohort study of factors affecting time to adoption of dogs in a humane society (Unpublished master’s thesis). University of California, Davis, CA.

358

VERONICA J. HOWARD and FLORENCE D. DIGENNARO REED

Braem, M. D., & Mills, D. S. (2010). Factors affecting response of dogs to obedience instruction: A field and experimental study. Applied Animal Behaviour Science, 125, 47–55. Burch, M. R., & Bailey, J. S. (1999). How dogs learn. New York, NY: Howell. Catania, C. N., Almeida, D., Liu-Constant, B., & DiGennaro Reed, F. D. (2009). Video modeling to train staff to implement discrete-trial instruction. Journal of Applied Behavior Analysis, 42, 387–392. doi: 10.1901/jaba.2009.42-387 Coppola, C. L., Grandin, T., & Enns, R. M. (2006). Human interaction and cortisol: Can human contact reduce stress for shelter dogs? Physiology & Behavior, 87, 537– 541. doi: 10.1016/j.physbeh.2005.12.001 DiGennaro, F. D., Martens, B. K., & Kleinmann, A. E. (2007). A comparison of performance feedback procedures on teachers’ treatment implementation integrity and students’ inappropriate behavior in special education classrooms. Journal of Applied Behavior Analysis, 40, 447–461. doi: 10.1901/jaba.2007.40-447 DiGennaro Reed, F. D., Codding, R., Catania, C. N., & Maguire, H. (2010). Effects of video modeling on treatment integrity of behavioral interventions. Journal of Applied Behavior Analysis, 43, 291–295. doi: 10.1901/jaba.2010.43-291 Ducharme, J. M., & Feldman, M. A. (1992). Comparison of staff training strategies to promote generalized teaching skills. Journal of Applied Behavior Analysis, 25, 165–179. doi: 10.1901/jaba.1992.25-165 Gardner, J. M. (1972). Teaching behavior modification to nonprofessionals. Journal of Applied Behavior Analysis, 5, 517–521. doi: 10.1901/jaba.1972.5-517 Hubrecht, R. C., Serpell, J. A., & Poole, T. B. (1992). Correlates of pen size and housing conditions on the behaviour of kennelled dogs. Applied Animal Behaviour Science, 34, 365–383. doi: 10.1016/S0168-1591(05) 80096-6 Johnson, M. D., & Fawcett, S. B. (1994). Courteous service: Its assessment and modification in a human service organization. Journal of Applied Behavior Analysis, 27, 145–152. doi: 10.1901/jaba.1994.27-145 Krumhus, K. M., & Malott, R. W. (1980). The effects of modeling and immediate and delayed feedback in staff training. Journal of Organizational Behavior Management, 2, 279–293. doi: 10.1300/J075v02n04_05 Lepper, M., Kass, P. H., & Hart, L. A. (2002). Prediction of adoption versus euthanasia among dogs and cats in a California animal shelter. Journal of Applied Animal Welfare Science, 5, 29–42. Luescher, A. U., & Medlock, R. T. (2009). The effects of training and environmental alterations on adoption success of shelter dogs. Applied Animal Behaviour Science, 117, 63–68. doi: 10.1016/j.applanim.2008. 11.001 Martens, B. K., Witt, J. C., Elliott, S. N., & Darveaux, D. X. (1985). Teacher judgments concerning the acceptability of school-based interventions. Professional Psycholo-

gy: Research & Practice, 16, 191–198. doi: 10.1037/ 0735-7028.16.2.191 Menor-Campos, D. J., Molleda-Carbonell, J. M., & LópezRodríguez, R. (2011). Effects of exercise and human contact on animal welfare in a dog shelter. Veterinary Record, 169, 388. doi: 10.1136/vr.d4757 Moore, J. W., & Fisher, W. W. (2007). The effects of videotape modeling on staff acquisition of functional analysis methodology. Journal of Applied Behavior Analysis, 40, 197–202. doi: 10.1901/jaba.2007.24-06 Neef, N. A., Trachtenberg, S., Loeb, J., & Sterner, K. (1991). Video-based training of respite care providers: An interactional analysis of presentation format. Journal of Applied Behavior Analysis, 24, 473–486. doi: 10.1901/jaba.1991.24-473 Nigro-Bruzzi, D., & Sturmey, P. (2010). The effects of behavioral skills training on mand training by staff and unprompted vocal mands by children. Journal of Applied Behavior Analysis, 43, 757–761. doi: 10.1901/ jaba.2010.43-757 Patronek, G. J., Glickman, L. T., Beck, A. M., McCabe, G. P., & Ecker, C. (1996). Risk factors for relinquishment of dogs to an animal shelter. Journal of the American Veterinary Medical Association, 209, 572– 581. Patronek, G. J., & Rowan, A. N. (1995). Determining dog and cat numbers and population dynamics. Anthrozoös, 8, 199–205. Pearce, J. (1993). Volunteers: The organizational behavior of unpaid workers. New York, NY: Routledge. Posage, J. M., Bartlett, P. C., & Thomas, D. K. (1998). Determining factors for successful adoption of dogs from an animal shelter. Journal of the American Veterinary Medical Association, 213, 478–482. Roscoe, E. M., & Fisher, W. W. (2008). Evaluation of an efficient method for training staff to implement stimulus preference assessments. Journal of Applied Behavior Analysis, 41, 249–254. doi: 10.1901/ jaba.2008.41-249 Sarokoff, R. A., & Sturmey, P. (2004). The effects of behavioral skills training on staff implementation of discrete-trial teaching. Journal of Applied Behavior Analysis, 37, 535–538. Sarokoff, R. A., & Sturmey, P. (2008). The effects of instructions, rehearsal, modeling, and feedback on acquisition and generalization of staff use of discrete trial teaching and student correct responses. Research in Autism Spectrum Disorders, 2, 125–136. doi: 10.1016/j. rasd.2007.04.002 Segurson, S. (2009). Behavioral enrichment for dogs. Paper presented at the ninth North American Veterinary Conference, Orlando, FL. Abstract retrieved from http://www.scribd.com/doc/59599411/BehavioralEnrichment-Dogs Thorn, J. M., Templeton, J. J., Van Winkle, K. M. M., & Castillo, R. R. (2006). Conditioning shelter dogs to sit. Journal of Applied Animal Welfare Science, 9, 25–39. doi: 10.1207/s15327604jaws0901_3

VOLUNTEER TRAINING Wells, D. L. (2004). A review of environmental enrichment for kennelled dogs, Canis familiaris. Applied Animal Behaviour Science, 85, 307–317. doi: 10.1016/j. applanim.2003.11.005 Wells, D., & Hepper, P. G. (1992). The behavior of dogs in a rescue shelter. Animal Welfare, 1, 171–186. Wells, D. L., & Hepper, P. G. (2000). Prevalence of behaviour problems reported by owners of dogs purchased from an animal rescue shelter. Applied Animal Behaviour Science, 69, 55–65. doi: 10.1016/ S0168-1591(00)00118-0 Wharton, C. S. (1991). Why can’t we be friends? Expectations versus experiences in the volunteer role. Journal of Contemporary Ethnography, 20, 79–106. doi: 10.1177/ 089124191020001004

359

Wolf, M. M. (1978). Social validity: The case for subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11, 203–214. doi: 10.1901/ jaba.1978.11-203 Zawistowski, S., Morris, J., Salman, M. D., & Ruch-Gallie, R. (1998). Population dynamics, overpopulation and the welfare of companion animals: New insights on old and new data. Journal of Applied Animal Welfare Science, 1, 193–206. Received February 16, 2012 Final acceptance January 8, 2014 Action Editor, Jennifer Austin