Alumni Research and Criminal Justice Brian Payne

+(,121/,1( Citation: 21 J. Crim. Just. Educ. 245 2010 Content downloaded/printed from HeinOnline (http://heinonline.org) Mon Mar 18 18:45:52 2013 -- Your use of this HeinOnline PDF indicates your acceptance of HeinOnline's Terms and Conditions of the license agreement available at http://heinonline.org/HOL/License -- The search text of this PDF is generated from uncorrected OCR text. -- To obtain permission to use this article beyond the scope of your HeinOnline license, please use: https://www.copyright.com/ccc/basicSearch.do? &operation=go&searchType=0 &lastSearch=simple&all=on&titleOrStdNo=1051-1253

JOURNAL OF CRIMINAL JUSTICE EDUCATION

VOLUME 21

NUMBER 3 (SEPTEMBER 2010)

Alumni Research and Criminal Justice Brian Payne, Wendy Guastaferro and Dean Dabney

Alumni studies use past graduates of educational programs or institutions to address various types of research aims. While alumni research is common in disciplines such as business, psychology, nursing, and social work, few criminal justice alumni studies have been published in the academic literature. This article reviews the types of alumni studies that are typically conducted, discusses reasons for conducting criminal justice alumni surveys, considers barriers to these surveys, identifies criminal justice-specific themes that address why alumni studies are rare in the discipline, and suggests strategies for enhancing

and expanding this sort of research in the discipline.

Introduction Alumni research is relatively common in fields such as business, psychology, nursing, and social work (Haga and Heitkamp 2000; Hartman and Schmidt 1995; McGovern and Carr 1989; Palomba and Banta 2001). These inquiries often include issues that are best addressed through surveys of those who have graduated from a specific program, college, or university, as opposed to issues that

impact or apply to the larger discipline. In fact, unpublished alumni research is a staple of institutional research protocols at universities across the country, as this line of inquiry provides valuable insight on dimensions of organizational effectiveness/capacities and/or the experiences and perceptions of students graduating from the academic units. A number of different types of alumni-based studies exist. Given that a sizable portion of the alumni research goes unpublished one should not be surprised that there exists a lack of published material on alumni research efforts that apply to criminal justice or any other academic discipline. Because of the inherent value that these surveys have for criminal justice, more attention needs to given to the way that alumni surveys can be used in criminal justice. To generate more awareness about these approaches, in the following sections, the authors address (1) types of alumni studies, (2) why criminal

0 Routledge 1byor&mncCrowp

ISSN 1051-1253 print/1745-9117 online/10/030245-21 © 2010 Academy of Criminal Justice Sciences DOI: 10.1080/10511253.2010.487836

HeinOnline -- 21 J. Crim. Just. Educ. 245 2010

246

PAYNE ET AL.

justice alumni surveys are important, (3) barriers to conducting alumni surveys in criminal justice, (4) criminal justice themes related to alumni surveys, and (5) strategies to overcome barriers that have limited criminal justice alumni studies in the past. Addressing these areas wilt provide a foundation from which criminologists can better incorporate alumni research into the discipline.

Types of Alumni Surveys Generally speaking, four types of studies can be identified. These include: (1) empirically driven studies, (2) recruitment/institutional development surveys, (3) recruitment/institutional advancement surveys, and (4) assessment-focused studies. Empirically driven alumni surveys are conducted with the aim of conducting a broader study wherein college graduates are deemed to be the most appropriate sample. These include theory tests, opinion surveys, and job satisfaction studies. Theory tests of alumni research are often conducted with the aim of determining whether some educational theory has merit (Burley, Butner, Causey-Bush, and Bush 2007; Glassman, Werch, Jobli, and Bian 2007). Opinion surveys of alumni tend to ask respondents what they think about some issue related to their prior educational experiences (Landrum and Lisenbe 2008). In one of the few published criminal justice alumni surveys of this kind, criminologists Bohm and Vogel (2004) surveyed alumni of a criminal justice program to see whether having a criminal justice course on the death penalty influenced attitudes about the death penalty 10 years after they had completed the course. Job satisfaction studies, in turn, focus on whether graduates are satisfied with their careers (Parks and Parra 1994; Pike 1994), and many of these studies focus on whether the educational experience is tied to job satisfaction (Cabrera, Weerts, and Zulick 2005). Recruitment/institutional development surveys focus on whether alumni would be supportive of participating in additional educational programs at a particular college or university. In some cases, these surveys are conducted to determine if alumni interest exists for the development of new graduate programs. At Old Dominion University, for example, an alumni survey was conducted to determine if the graduates of the M.A. in applied sociology program would be interested in applying for a proposed PhD program in Criminology and Criminal Justice (Old Dominion University 2006). At Kennesaw State University, faculty in the Department of Political Science and International Affairs recently surveyed their International Affairs alumni to see how many students would be interested in a graduate program in International Affairs (Kennesaw State University 2008). They found that 42% of alumni were interested in the proposed programs. Surveys such as these help determine whether programs are justified and identify possible alumni who could be recruited into the program. Recruitment/institutional advancement surveys focus on the degree to which alumni are willing to financially support the department, college, or


ALUMNI RESEARCH

247

university (Cabrera et at. 2005). These surveys are typically conducted (or supported) by the college/university's institutional advancement office (Melchiori 1988). The aim is to identify the kinds of alumni the college or university should target in fundraising activities. Alumni provide 25-30% of the total support for higher education and this proportion has remained steady over the past 30 years (Council for Aid to Education 2008; Leslie and Ramey 1988). Research has found that the greater an alumni's perception of their educational experience the more likely they are to donate (Hartman and Schmidt 1995). Knowing how successful your alumni are financially, while sensitive, is important for fundraising as well. Those with a greater capacity to donate are usually responsible for the bulk of donations. This is known as the "90/10" rule, which holds that a small percentage of people make the larger donations. Fundraisers should identify the top 10% of income earners and target them for capital campaigns while communicating the value of donations at all levels to those who are able to contribute on a smaller scale or, as will be addressed below, in different ways (Huleat 2007; Weinstein and Hartsook 2002). Assessment surveys are probably the most common form of alumni studies. These studies are designed to assess different aspects of the pedagogical experience from the perspective of college graduates. Assessment has been defined as a "reality check" that allows faculty to identify gaps and needs in a particular program (Graveel and Vorst 2007, p. 87).

Why Conduct Alumni Surveys? Undergraduate and graduate programs in criminal justice have grown over the past 30 years (Flanagan 2000; Frost and Clear 2007; Southerland 2002). In 2007, there were over 18,000 criminal justice undergraduates, 1376 master's students, and over 800 PhD students enrolled at three-quarters of the criminal justice/criminology doctoral-granting programs alone (ADPCCJ 2009). The discipline's growth is evident. Practice is out ahead of data in terms of documenting program quality and relevance, program assessment, and learning outcomes. To this end, there are several reasons for extending and enhancing our use of alumni-based surveys. These reasons include the following: * * * * * * *

Empirical reasons Assessing the product of criminal justice pedagogy Consumer reasons Alumni connections Because we have to Community (Department) Integration Demonstrating the importance of the discipline

Each of these reasons is addressed below.


248

PAYNE ET AL.

Empirical Reasons Assessment-focused empirical research using criminal justice alumni can improve both teaching and research. These studies are not different than other empirical criminal justice studies, other than the fact that the sample would focus on a specific group of criminal justice alumni. As an illustration, researchers can look to the way other disciplines have used alumni samples to develop questions specific to criminal justice. Research questions relevant to program content, career choices/opportunities, real-world practices, and system responses could be developed. Here are a few examples: (1) Is success in policing tied to academic success in a university's criminal justice program? (2) Does exposure to domestic violence coursework influence the way that police officers respond to domestic violence cases? (3) Is career satisfaction in a criminal justice career related to satisfaction with coursework in a criminal justice program? (4) Do criminal justice alumni with advanced degrees perform their jobs with greater success than those without advanced degrees? (5) Does exposure to criminal and constitutional law courses influence the way criminal justice graduates process criminal cases? (6) Do criminal justice alumni perform differently in justice careers than criminology alumni or alumni from other disciplines? (7) Do criminal justice alumni who transferred into the college/university fare differently in their careers than those who did not transfer? (8)What factors contribute to criminal justice graduates' decisions to seek advanced degrees? (9)What other topic areas or courses would have been helpful given your current job? These questions only begin to highlight the possible avenues for research using criminal justice alumni as samples. Given that other disciplines have asked similar questions of their alumni (see Danis 2003; Danis and Lockhardt 2003), it seems natural to question the way that criminal justice coursework from specific institutions influences various outcomes.

Assessing the Product of Criminal Justice Pedagogy Criminal justice alumni surveys provide an indirect measure of student learning outcomes. Criminal justice curricula are constantly in flux. It is imperative that criminal justice faculty make sure that their curricula are producing graduates who are able to contribute in a meaningful way in their careers. It is not enough to assume that curricula are making a difference; instead, faculty should survey criminal justice alumni to determine how their courses changed the lives of


ALUMNI RESEARCH

249

graduates and prepare them for future careers. Alumni can provide some indication of trends, new practices, and changes that could be relevant to curriculum development. Some fields (nursing, accounting, social work) have a list of specific outcomes that are used as indicators to assess a particular program's strengths. These outcomes are often specified by accrediting bodies and get used to develop licensure exams that graduates must pass in order to seek professional employment in careers requiring Licensing. While accreditation efforts are underway for criminal justice, no similar list of outcomes exists in the field. This makes it difficult to develop a list of generic, broad-based outcomes for criminal justice alumni surveys. Despite the absence of precise indicators to assess criminal justice curricula in alumni surveys, where broad-based comparisons are the goal, general indicators can be used as a foundation for criminal justice alumni surveys. For instance, career satisfaction has been recognized as an important indicator of success (Mayzer and DeJong 2003). Other potential indicators include gainful employment, job promotions, and satisfaction with one's education (Pettit 1991). As past Academy of Criminal Justice Sciences (ACJS) President Laura Moriarty (2006) notes, it is common for criminal justice programs to use grades as assessment indicators, despite the fact that grades are not useful in measuring writing or critical thinking. Moriarty calls for strategies to track progress in criminal justice curricula. Developing and carrying out alumni surveys (with specified outcomes) on a regular basis would be one such strategy. The task at hand is to identify a set of outcomes that are general enough to relate to all criminal justice curricula at each specific degree level (e.g., Associates, Bachelor's, Master's, and PhD). It is important to note that assessment can be done on at least five different levels (see Figure 1). These levels include the following: * * * * *

Teaching tool level Course level Curriculum (major) level Curriculum (institutional) level Community level

First, some faculty may conduct alumni surveys to assess student learning in an effort to determine if different teaching tools effectively contribute to knowledge about course content. Second, faculty may conduct surveys to determine whether specific courses are helping to meet the overall aims of the curriculum (Owen, Fradella, and Burke 2006; Takata 2006). Third, departments may survey alumni as part of a strategic planning process in which faculty seek to determine whether and how the major curriculum should be revised (Davidson-Shivers, Inpornjivit, and Sellers 2004; Hoey and Gardner 1999; Landrum and Lisenbe 2008; Ma 2002). Fourth, most colleges and universities routinely conduct surveys of all alumni to determine the degree to which the entire curriculum meets the goals


250

PAYNE ET AL.

Levels are interrelated

Assessing each level has implications for subsequent levels

Assessment of institutional curriculum

Community needs assessment

Figure 1 Assessment levels. of the institution. Finally, some researchers have surveyed alumni to determine the community needs or insight with regard to a specific substantive topic. Here, the goal is to link intended learning outcomes (skills or substantive content) to real-world situations encountered by program graduates. For instance, researchers have surveyed social worker alumni to see how many felt their education prepared them to recognize and respond to domestic violence (Danis 2003; Danis and Lockhart 2003). Surveys have also been used to determine what gaps existed


ALUMNI RESEARCH

251

in terms of social workers' awareness about domestic violence issues within the clients that they serve (Kanuha 1998).

Consumer Reasons Alumni surveys should also be conducted for consumer reasons (or for the purposes of marketing). When consumers buy cars, for example, they are inundated with a series of surveys asking about consumer satisfaction related to the purchasing experience and the newly acquired product. The surveys are used as marketing tools to demonstrate the success of the car dealership and the company's product in general. A similar model could be used with criminal justice graduates. Considering that an education typically costs more than a new car-at least more than the kind of car most professors purchase-it makes sense that faculty make sure that alumni (as consumers) were satisfied with their interactions with the faculty and administration as well as the coursework. Demonstrating satisfaction will strengthen the program's identity for internal audiences at the university (e.g., administrators making funding decisions) and external audiences (e.g., students who are deciding on majors or data on employment or graduate school placements by alumni). Where problem areas are identified, the responsible parties can make an effort to change.

Alumni Connections Criminal justice alumni studies should also be conducted as a strategy to foster connections between a criminal justice program and its graduates. It is far too easy to forget about graduates once they leave an institution. Alumni surveys are one strategy that can be used to maintain contact with graduates. Such surveys are mutually beneficial. They show alumni we are interested in their past and current experiences, needs, and goals, and that their insight is appreciated. In addition, they provide a strategy for promoting social engagements with alumni as well as a tool for fundraising. It is well established that making connections early with alumni will have lasting benefits: as alumni get older their desire to give back increases as does their capacity to do so (Jaschik 2008; Taylor and Martin 1995). Programs and departments should consider mechanisms for donating that are non-monetary such as speaking to current/potential students, volunteering at a program event, serving in a program advisory capacity, or helping graduating seniors find employment.

Because We Have To Another reason criminal justice programs should promote alumni surveys can be coined "because we have to." What this means is that some departments are


252

PAYNE ET AL.

required to conduct alumni surveys every few years as part of a department's self-study process. Typically, these surveys are limited to items that demonstrate satisfaction or focus on specific issues of relevance solely to the criminal justice program conducting the survey. As will be shown below, with some modifications, these sorts of surveys can serve as the basis for published works (empirical or conceptual in nature) that inform the discipline.

Community (Department) Integration Criminal justice programs should also conduct alumni surveys because of the way that such efforts promote department integration. If a large number of faculty is involved in an alumni survey, the process accomplishes at least three things. First, it brings together faculty members who may have never actually had the opportunity to work together before and allows them to work as a group toward a specific aim. Second, as part of this integration, an alumni survey encourages faculty members to think about how their courses and teaching practices fit in with the broader aims of the department and university. Third, the process of an alumni survey fosters integration between department faculty and other campus entities such as the alumni office and institutional research office (with whom the faculty may have had very little contact in the past).

Demonstrating the Importance of the Discipline Alumni surveys of criminal justice graduates also help to demonstrate the importance of the criminal justice discipline to the academic and professional communities. In recent years, the ACJS has developed and promoted a certification process that is designed to promote the importance of the discipline by providing an avenue through which criminal justice programs can demonstrate that they are adhering to a specific set of principles approved by ACJS. This process fosters excellence at the department level. Part of the certification process asks departments for evidence about successful pedagogy. Alumni surveys are one strategy that can be used as part of the certification process. The surveys would demonstrate that departments (and the discipline) contribute in a meaningful way to the betterment of the community. As an example, consider that in May 2005, the ACJS approved the ACJS Certification Standards for Academic Programs. A certification review process was implemented at the same time to provide a mechanism through which criminal justice programs could be assessed. The main goal of the standards and process is to "measurably improve the quality of criminal justice education" (ACJS Certification Standards n.d.). In the introduction to these standards, the following statement appears: "The emphasis is on evidence, not intention." What this means is that programs going through the certification process must


ALUMNI RESEARCH

253

be able to show evidence that they are meeting the standards set forth by ACJS. Not coincidentally, alumni surveys are identified (along with other types of surveys) as a source of data that can be used to demonstrate quality of criminal justice education. In particular, alumni surveys, according to the ACJS Standards, are an indicator for the following aspects of quality: * The program provides an environment that fosters the intellectual and personal development of its students, consistent with its mission and purposes. * All students in the criminal justice program have access to appropriate and effective orientation, academic advisement, career development, and placement counseling. * The institution systematically identifies the characteristics and learning needs of its criminal justice student population and makes provision for responding to them. As a part of the certification process-designed to show quality-alumni surveys also help to demonstrate the importance of criminal justice programs to the community.

Barriers to Criminal Justice Alumni Surveys Barriers exist for any form of survey; thus, it should come as no surprise that specific barriers likely have contributed to the lack of alumni research in criminal justice. These specific barriers include the following: * * * * * *

Out of sight, out of mind Sampling issues Financial issues Academic culture Faculty resistance Faculty development

Each of these barriers is addressed below. It is far too easy to simply forget about our graduates once they become alumni. This may be particularly problematic for criminal justice programs with Large numbers of majors and can be coined "out of sight, out of mind." On their face, alumni surveys appear to be focused backwards. Efforts are directed toward the "here and now" and with hundreds of current majors, itmay be difficult for faculty to even think about past students. For departments with a handful of majors and graduates, it is likely far easier to maintain some sort of contact between faculty and alumni. Given that criminal justice departments are often among the largest academic programs on many colleges and campuses, it is reasonable to suggest that the sheer number of criminal justice


254

PAYNE ET AL.

alumni is a barrier for many departments seeking to establish contact with alumni through surveys or other measures for that matter. A related barrier has to do with sampling issues. Those who have tried to find the mailing addresses of alumni wilt likely describe several difficulties they had in locating active street addresses for past graduates. Working with the college or university alumni office may provide some leads, but the fact of the matter is that alumni move and they often do not want their college or university to have their contact information because they want to avoid the avalanche of phone calls for donations to the institution. As a simple example, it would be interesting to ask members of the academy if they alert their graduation institution of their new addresses when they relocate. To be sure, alumni offices are still able to track down addresses of most graduates, but it would be virtually impossible for a criminal justice department to maintain up-to-date information on graduates' whereabouts without some sort of assistance. An accessible point of contact remains theoretically open when institutions allow students to maintain active email accounts upon graduation. Although, the viability of this email route relies upon the active use of the account by a large number of graduates; this is not always a tenable proposition. Financial issues also surface when considering alumni surveys. Consider a survey of 1000 alumni. Assuming the use of a mailed delivery protocol, postage costs alone could quickly escalate to $2000-especially if follow up mailings are conducted. Funds for survey instruments and envelopes will add another $1000 to the costs. Paying a graduate assistant to help with managing the survey would add another $2000. Faculty time, if it were charged against a grant, would likely be around 10% of the faculty member's salary. Assuming a junior professor were involved, this would be about $5000. The bottom line is that the survey would cost a minimum of $10,000. Some of these costs can be defrayed though alternative delivery protocols (email or online surveys), but these options pose new and often equally challenging hurdles to overcome. One issue that departments must consider is whether it is useful to spend such resources on surveys that are known for producing very low response rates (from 10% to 30%, see Aviles 2000; Eliza, Rodriguez, and Rosario 1999; Porter and Umbach 2001). Also, it would be difficult to find funding for these surveys because they typically do not address the kinds of issues that are judged to be relevant or worthy of funding by grantors. In the end, departments are often forced to pay the costs themselves. This is not always an attractive option, particularly during tough budget times. Another barrier has to do with the role of criminal justice within the academic culture. The academic culture promotes scientific research and scholarly development that is based on the empirical advancement of knowledge. It is well accepted that criminal justice is sometimes confused as an applied discipline rather than a scientific discipline (Clear 2001; Finckenauer 2005; Flanagan 2000). Thus, research in criminal justice faces an inherent obstacle with the domain of the broader academic culture: research must build on a knowledge base specific to the discipline, but tied to the broader academy,


ALUMNI RESEARCH

255

in a way that advances discipline and academic interests-research on pedagogy tends to be placed within the domain of the discipline of education. What this means is that criminal justice research tends to focus on issues specifically relevant to criminal justice. On the surface, most of these topics have very little to do with topics that would be relevant to an alumni survey (e.g., tests of theory, program evaluations, differences in the way offenders are processed, characteristics of criminal justice professionals). Given that criminal justice researchers aim to fit in with this broader academic culture, efforts are directed toward research that better fits in with the ideals of criminal justice as a scientific discipline. As an example, consider theses and dissertation topics by graduate students. They typically focus on an issue specific to the discipline of criminal justice. Imagine if a student said they wanted to do their thesis or dissertation on the implications of group work in criminal justice courses. She or he proposes to survey alumni to see how they defined their experiences with group work in their criminal justice courses and how those experiences related to their careers. It is easy to imagine that the student would be told that their proposed topic belongs in an education graduate program, but not a criminal justice program. The bottom line is that alumni surveys are avoided in criminal justice partly because the kinds of questions that are most easily addressed through the studies are often judged as outside of the interests of the discipline. Faculty resistance potentially makes it difficult to conduct alumni studies in criminal justice. While there are several possible types of alumni studies, when individuals think of alumni surveys, they likely automatically think of assessment studies. After all, why else would we contact our alumni but to see what we did right and what we did wrong? Faculty members are similar to others facing evaluation-there is a resistance to assessment strategies (Muffo 1996); they take time and are often mandated by the central administration. Yet this is ironic for criminal justice researchers, most of who are trained in program evaluation targeting top-down bureaucracies. We hold program evaluation in these environments central to the values of our discipline. We study all sorts of criminal justice programs; we just do not want to be expected to evaluate our own programs. A related barrier has to do with faculty development. Throughout an academic's career, faculty development occurs in isolation. Beginning as graduate students, faculty members are told to develop their own research agendas. These agendas, faculty members are told, should focus on some broader research topic that advances the scientific aims of the discipline. In fields such as education, assessment represents a viable and respected area of study. The same cannot be said in criminal justice. Once they are hired into an assistant professor position, criminal justice faculty are told they must continue this research agenda in order to be tenured and promoted. The very nature of faculty development is such that research agendas focus primarily on criminal justice topics-surveying alumni, particularly with alumni assessment surveys, 1s not a type of research that fits well in most criminal justice assistant professors'


256

PAYNE ET AL.

research agendas. Devoting a great deal of effort to such research potentially jeopardizes research portfolios, which could conceivably make it more difficult to achieve tenure and promotion. Even if this difficulty is imagined, the fact that junior professors perceive the limitations of such research results in consequences that make them avoid promoting or engaging in alumni assessment surveys. Indirectly describing this barrier, Erroll Davis, chancellor of the University System of Georgia, made the following comments: In academe, you spend your early years focusing on your discipline. To earn a terminal degree, you have to have tremendous focus-which is, by necessity, usually quite self-centered. It takes a median of seven years after graduating from college to earn a PhD in the humanities. You end up with a lot of knowledge, but the process is designed to be all about you-your needs, your learning, your dissertation. Then, around age 29, your next goat-six years out-is earning tenure ... tenure acquisition requires a focus on research, publishing, and supposedly teaching and service. But again, it's all about you-your research, your persona, your scholarly stardom ... In the academic environment, processes are seen as something bureaucracies do to you not for you. The culture rein-

forces astrong sense of self as opposed to the team-a strong sense of individual goals versus a sense of the greater good or common goat (Davis 2008, p.A64). The general thrust of Davis' comments suggests that disciplines need to hold themselves accountable to the community, but that there is no incentive at the individual level to participate in these efforts. At the department level, though, departments are now routinely called upon to demonstrate their worth or value to the institution and the community. Academic programs do this through assessment programs that demonstrate success or avenues for change. Alumni surveys are one possible tool that can be used in this regard. In the following section, attention is given to reasons criminal justice researchers should conduct and publish more criminal justice alumni studies. Criminal Justice Themes Related to Alumni Surveys Four themes characterize the specific challenges of criminal justice alumni research and demonstrate the need to increase scholarly efforts using alumni samples. These themes include: distinctiveness of criminal justice majors; distance between criminal justice, institutional assessment and alumni office; criminal justice as a liberal arts discipline; and measuring outcomes in criminal justice. Each of these themes is addressed below.

Distinctiveness of Criminal Justice Majors A number of studies have shown that criminal justice majors are different from non-criminal justice majors. One study, for example, found that non-criminal


ALUMNI RESEARCH

257

justice majors display more proficient statistical conceptual abilities and computational skills than criminal justice majors do (Proctor 2006). Another study found that criminal justice students are less supportive of harsh punishments and more supportive of offenders' rights than other students, and criminal justice majors were found to be more supportive of treating juveniles leniently (Tsoudis 2000). Differences have been found between alumni from different social sciences majors (Johnson, Sigler, and Crowley 1994; Velasco, Stockdale, and Scrams 1992). Recognizing the differences between criminal justice majors and non-criminal justice majors is relevant to alumni surveys inasmuch as one can assume that differences in majors results in differences in the professions. Generally speaking, it is reasonable to suggest that criminal justice majors have a wider range of academic goals than other majors (e.g., some want to go to law school or graduate school, others want to go into law enforcement-whereas other majors have more clearly defined career choices). This diversity makes creating career-oriented questions that are broad enough to have applicability to the group yet specific enough to provide useable individual-level information challenging. Institutional alumni surveys often lump all graduates together to assess various factors (Nagle and Bohovich 2000). To fully assess how this "criminal justice influence" manifests itself, it seems necessary to focus specifically on criminal justice alumni to gather information relevant to the curriculum, profession, and community.

Distance between Criminal Justice and Institutional Assessment Criminal justice professors are typically far removed from the institutional assessment process and the alumni offices across colleges and universities. This is problematic because criminal justice faculty should have an active role in working with institutional assessment researchers to survey criminal justice graduates and develop strategies to maintain contact with alumni. Research suggests that alumni identify more with their faculty and departments than with their colleges or institutions (Eliza et at. 1999). Most often, alumni surveys are distributed by the institutional research office, with whom the graduate has had no prior interactions. Criminal justice faculty must work with institutional research to identify strategies that will encourage criminal justice alumni to participate in the alumni surveys.

Criminal Justice as a Liberal Arts Discipline Tracing the history of criminal justice as an academic discipline, one can see how the discipline changed in the 70s from an applied discipline in which students learned the tools of the trade so to speak, to a more broadly defined liberal arts discipline (Clear 2001; CulLen 1995; Flanagan 2000; Southerland 2002). Curricula were designed to promote the scientific nature of the discipline


258

PAYNE ET AL.

from a liberal arts orientation. Rather than teaching students how to perform in criminal justice careers, most Bachelor's degree programs are now oriented toward broadening students' horizons by focusing on writing, critical thinking, empirical skills, and other areas related to the scientific nature of the discipline. The result is that students seek majors, not simply to find careers in criminal justice, but to learn about criminal justice-with a large segment of majors at some institutions potentially having little interest in working in the field. Insofar as alumni surveys are concerned, this broader nature of the discipline means that the dynamics of criminal justice alumni surveys are different from the dynamics of alumni surveys conducted in other disciplines. Indeed, the nature of the field is such that graduates go into all sorts of careers-more so than they would in education, business, economics, social work, or nursing programs. Most education majors go on to become teachers; most business majors seek careers working in a commerce related capacity; most social work graduates become social workers; most nursing graduates become nurses. The same equation does not exist for criminal justice majors. While a sizeable proportion of them select criminal justice careers, the diverse nature of these careers makes it difficult to specify specific career paths for criminal justice majors (Hemmens 2002). Four consequences related to alumni surveys arise from this liberal arts approach to studying criminal justice. First, because graduates will go into so many different careers, it will be more difficult to locate them to participate in such survey efforts. Second, the liberal arts orientation of the field means that many graduates will be in careers with lower salaries than careers that employ other graduates-this means that alumni surveys designed to assist with fundraising efforts in criminal justice might recoup fewer dollars than other disciplines. Third, given the diverse nature of the careers that criminal justice majors go into, it is difficult to develop surveys relevant to all of the occupations. Fourth, in terms of assessment surveys, it is more difficult to develop a set of outcomes or indicators that would relate to all of the criminal justice careers. This will have implications for how graduates perceive the relevance of their education (which might artificially make it seem like the curriculum is not meeting students' needs). Difficulties identifying outcomes related to criminal justice curricula are addressed in the next section. These four themes help to understand the broader reasons why there have been so few alumni studies published in criminal justice. Several specific factors also have inhibited alumni studies in the discipline as addressed above. Despite these challenges, useful alumni surveys and research are possible. Next we offer strategies for overcoming these barriers.

Strategies for Overcoming Barriers to Conducting Alumni Research Given the number of reasons that exist for doing criminal justice alumni surveys, and the fact that so few alumni surveys are published in the criminal


ALUMNI RESEARCH

259

justice literature, some recommendations for future criminal justice alumni surveys are warranted. These recommendations stem from the authors' experiences with a recent alumni survey conducted in their department. These recommendations are not meant to be exhaustive, but are simply provided to guide other criminal justice faculty conducting alumni surveys. We offer strategies for overcoming the barriers outlined above. First, faculty must decide on the frequency of the time-frame of the survey (e.g., every three years) and the target audience (only those who have graduated in the past three years). Such decisions will be influenced by the purpose of the specific alumni survey. If the survey is designed, for example, to promote institutional advancement and fundraising, the sampling frame should include all graduates from the program and faculty may want to over-sample those who graduated a longer time ago (assuming they have more resources to donate to the program). In a similar way, if the aim is to determine what types of materials should be covered in the curriculum in order to meet the needs of criminal justice agencies, faculty may want to focus more on alumni who have been in the professions longer in order to gain more insight about the needs of the criminal justice professions. Alternatively, if the aim is to assess the specific curriculum, the sample should include those who were actually exposed to the curriculum. If a curriculum has been in place for six years, it does not make sense to ask alumni who graduated 15 years ago about their reactions to the curriculum. Second, in conducting alumni surveys, criminal justice faculty must agree on the goals of the survey before the survey is distributed. If the survey is being conducted solely for assessment purposes, faculty should decide how the information will be used. Benchmarks, objectives, and the target sample population (e.g., graduates from past three years) must be set before the survey is conducted. Doing so will make the results more valuable for assessment purposes. If the data are intended for publication purposes, IRB approval will be necessary and the involved scholars will need to tie their efforts to accepted practices and principles readily available in the scholarly literature. Third, alumni surveys should be defined as a process rather than an event. What this means is that the surveys should be done routinely over time. They are not one time events; instead, alumni surveys should be seen as part of a department's evolution. Alumni surveys could help lead criminal justice departments into appropriate courses of action. Such continuity will make alumni surveys easier to conduct over time (Muffo 1996). Yet criminal justice researchers must not overstate the importance of alumni surveys that are used for assessment purposes. Alumni surveys used for assessment must not be seen as the assessment tool; instead, alumni surveys should be seen as one tool that is part of a broader assessment toolkit. Alumni can help identify specific needs related to professional employment. For example, graduates who are police officers, victim advocates, or who work in a drug court may identify the need to know more about ancillary social services that can help offenders and their families. Recognizing the role of alumni surveys, and their


260

PAYNE ET AL.

context within the curriculum, will help criminal justice faculty appropriately use and disseminate the survey findings. Fourth, criminal justice faculty, administrators in particular, should develop relationships with the alumni office and the institutional research office. It is far too easy to operate independently of these other units. Recognizing that employees in these other units have specific expertise in the area of alumni research studies, it is clear that alumni studies involving these units should produce more valuable information than those conducted without input from institutional research or alumni officials (Eliza et at. 1999). As well, in developing relationships with these units, it is plausible that the units would be able to provide funds or some other form of support to help criminal justice faculty carry out their alumni studies. Fifth, criminal justice faculty should resist the temptation of using surveys that are constructed solely by the institutional research office. It is imperative that items specific to criminal justice majors, and the criminal justice curriculum, be included on the alumni survey. The items on the survey must be tied to the purpose of the survey (Borden 2005). Also, note that research shows that alumni surveys with items specific to the department tend to have higher response rates (Aviles 2000). Figure 2 shows one example of an instrument designed to complement a more general alumni survey. The items included are specific to criminal justice coursework and provide direction for understanding how the criminal justice curriculum influenced graduates. Sixth, programs should take advantage of technology to maintain contact with alumni. An alumni/fan page on Facebook or other networking website is another avenue for keeping people connected that is efficient and effective. Email should become the mode of communication as people are less likely to change their email than mailing address. A few universities allow students to keep their email addresses after graduation. Paper-based surveys continue to be important however. While neither method (paper- or webbased) leads to a higher response rate, differences by race in terms of who responds to which format have been noted (Carini, Hayek, Kuh, Kennedy, and Ouimet 2003). Seventh, in designing alumni surveys, and to address some of the barriers that were discussed above, criminal justice faculty should seek to tie empirical aims into their alumni surveys that have historically been conducted solely for assessment purposes. Given the demand placed on faculty to conduct and publish research, criminal justice researchers are encouraged to find ways to broaden alumni surveys that can lead to publishable products for the faculty devoting a great deal of time to the assessment. Eighth and somewhat related, professional associations could develop alumni surveys, or a core set of questions that could have individualized program issues added, for criminal justice programs nationwide. Alternatively, regional associations could develop alumni surveys and/or conduct a benchmarking study of the current alumni-based data collection efforts currentLy underway. National and regional professional associations couLd serve as a clearing house for alumni


ALUMNI RESEARCH

What is the highest degree you hase earned? 0

BS

Are you currently studying for an advanced degree'?

MA/MS Yes

261

Other:

PhD No

1f yes, please list the area of study, degree program and the university you are attending.

If no, do you have plans to carn an advaticed degree in the future?

Yes

Which of the following best describes your current employment status? Student Employed & student Employed Other: Military -Unemployed

j

No _

Caring for family

If employed, who is your employer? \hat is your position? How satisfied are you with your job/career? Dissatisfied Satisfied _ Very satisfied

Very dissatisfied

Please rate your education in the GSU Department of Criminal Justice on the following items, using the familiar grading scale (A,BC,D,F, or not applicable):

j

z E

Preparation to communicate through writing Preparation to effectively communicate orally Preparation to think critically and analytically Preparation to study and understand crime using a social science approach Preparation for your career/profession/graduate study Develop an understanding of the justice system Develop an understanding of law in that system Develop an understanding of crime Quality of instruction in major Availability of courses in major Appropriateness of courses in major Availability of research/independent study opportunities Quality of advisement Satisfaction with your experience in the major Satisfaction with your overall GSUexperience

A A A

B B B

C C C

D D D

F F F

N/A N/A N/A

A A A A A A A A A A A A

B B B B B B B B B B B B

C C C C C C C C C C C C

D D D D D D D D D D D D

F F F F F F F F F F F F

N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A

Considering your career path and professional experiences, what were the most helpful courses or experiences you had with the Department of Criminal Justice?

What significant change(s) would you like to see the Department make to the criminal justice program?

Figure 2 Criminal justice alumni surveys.

surveys, as the National Association of Schools of Public Affairs and Administration has done. These efforts could be linked with the effort to improve criminal justice education generally and with elements of program accreditation such as those sponsored by the ACJS. Given the national changes in assessment processes, it is expected that an increased focus will be given to assessment in the future. If criminal justice programs do not define specific outcomes of a criminal justice curriculum, programs run the risk of having standards forced upon them (see Moriarty 2006). As Hemmens notes, "If we allow disciplinary outsiders into the process, we cannot control their demands" (2002, p. 10). Finally, criminal justice programs are encouraged to use alumni surveys for promotional purposes. As noted above, other disciplines use alumni surveys either to recruit graduate students or solicit donors. Alumni surveys could also be used as a recruitment tool. Once a strong survey has been developed and


262

PAYNE ET AL.

feedback received, potential students could be encouraged to review the results. Criminal justice researchers could build these aims into their own alumni surveys.

Conclusion Departments of criminal justice often serve large numbers of majors and produce sizable pools of alumni who go on to live and work in the surrounding geographic area. These individuals hold valuable insight on the curriculum and service delivery of the university and home department that can aid in backward looking assessment efforts (e.g., impact of selected teaching tools, course effectiveness, curriculum utility) and forward looking planning efforts (e.g., fundraising potential, recruitment strategies, curriculum redesign, degree expansion). Specifically, alumni research affords us an opportunity to (1) assess various dimensions of instructor, course, or departmental productivity, (2) tap into customer satisfaction, (3) enhance alumni connections and giving, (4) augment efforts to respond to the skill and knowledge base needs of the agencies that employ our graduates, (5) publish discipline specific scholarly works in the area of assessment and strategic planning, and (6) gradually bolster the standing and identity of criminal justice as a field of study. To be sure, for alumni research to increase in criminal justice, criminologists, criminal justice departments, and the academic discipline itself must promote and be responsive to alumni studies. At the individual level, it is important that criminal justice faculty define alumni assessment surveys specifically, and assessment generally, as a value in the department (Cress 1996). On the one hand, this could be achieved by educating current criminal justice master's and doctoral students about the value of alumni research. On the other hand, current faculty should be encouraged, where appropriate and feasible, to conduct alumni studies. At the department level, the surveys must be defined as ways to strengthen criminal justice programs rather than ways to identify weaknesses (Myers 1994). To encourage faculty to engage in these studies, department chairs and administrators should seek ways to reward, or at least encourage, faculty to conduct these studies. Rather than defining alumni surveys and assessment as inferior research, administrators should make an effort to communicate the value of these research projects. At the discipline level, academics must be more open to alumni surveys as empirical studies. ACJS has taken a step in this direction by recommending in the ACJS Certification Standards for Academic Programs that alumni studies be used to demonstrate a criminal justice program's effectiveness. As shown above, however, alumni studies have the potential to benefit the discipline in many ways that extend beyond assessment. Indeed, broad based and far reaching goats underlie alumni research. Our aim in this article was to provide guidance on how to best approach alumni research studies while overcoming the


ALUMNI RESEARCH

263

numerous potential pitfalls that exist. Given the current climate encouraging or mandating assessment of individual and departmental activities, it seems to be the perfect time to expand the use of alumni studies in our discipline.

References ACJS Certification Standards. n.d. Retrieved from http://acjs.org/pubs/167_667 3517.cfm ADPCCJ. 2009. Annual report. Association of Doctoral Programs in Criminology and Criminal Justice. Retrieved from http://www.adpccj.com/documents/2009 ADPCCJ_SurveyReport. pdf Aviles, C. B. 2000. Successful collaboration between student affairs and academic affairs with a graduate follow-up survey (No. 446707). Buffalo, NY: Eric Document Reproduction Service. Bohm, R., and B. Vogel. 2004. More than ten years after: The long-term stability of informed death penalty opinions. Journal of Criminal Justice 32: 307-327. Borden, V. 2005. Alumni research to align program improvement with institutional accountability. New Directions for Institutional Research 126: 61-72. Burley, H., B. Butner, T. Causey-Bush, and L. Bush. 2007. African American alumni feelings of attachment to a predominately white research institution. College Student Journal 41: 203-216. Cabrera, A., D. Weerts, and B. Zulick. 2005. Making an impact with alumni surveys. New Directions for Institutional Research 126: 5-17. Carini, R. M., J. C. Hayek, G. D. Kuh, J. M. Kennedy, and J. A. Ouimet. 2003. College student responses to web and paper surveys: Does mode matter? Research in Higher Education 44 (1): 1-19. Clear, T. R. 2001. Has academic criminal justice come of age? Justice Quarterly 18 (4): 709-726. Council for Aid to Education. 2008. Voluntary support of education survey. Retrieved February 25, 2009, from http://www.cae.org/content/projdatatrends.htm Cress, C. 1996. Assessment and testing: Measuring up to expectations. Los Angeles, CA: ERIC Clearinghouse for Community Colleges. Cullen, F. 1995. Fighting back: Criminal justice as an academic disapline. ACJS Today 13 (4): 1,3. Danis, F. S. 2003. The emerging field of crime victim assistance: Are social workers ready? Professional Development: The International Journal of Continuing Social Work Education 6: 13-19. Danis, F. S., and L. Lockhardt. 2003. Guest editorial: Domestic violence and social work education: What do we know, what do we need to know? Journal of Social Work Education 39: 215-224. Davidson-Shivers, G., K. Inpornjivit, and K. Sellers. 2004. Using alumni and student databases for evaluation and planning. College Student Journal 38: 510-518. Davis, E. 2008. Colleges need to offer clear paths to leadership. Chronicle of Higher Education 18: A64. Eliza, S.M., L. Rodriguez, and E. Rosario. 1999. Overcoming the challenge of low response rates in alumni surveys (No. 33771). Buffalo, NY: Eric Document Reproduction Service. Finckenauer, J.0. 2005. The quest for quality in criminal justice education. Justice Quarterly 22 (4): 413 426. Flanagan, T. J. 2000. Liberal education and the criminal justice major. Journal of Criminal Justice Education 11 (1): 1-13.


264

PAYNE ET AL.

Frost, N. A., and T. R. Clear. 2007. Doctoral education in criminology and criminal justice. Journal of Criminal Justice Education 18 (1): 36-52. Glassman, T., C. Werch, E. Jobli, and H. Bian. 2007. Alcohot-reLated fan behavior on college football game day. Journal of American College Health 56: 255 260. GraveeL, J. G., and J. J. Vorst. 2007. Using alumni input as a reality check of agronomy teaching and advising. Journal of Natural Resources and Life Sciences Education 36: 87-94. Haga, M., and T. Heitkamp. 2000. Bringing social work education to the prairie. Journal of Social Work Education 36: 309-324. Hartman, D. E., and S. L. Schmidt. 1995. Understanding student/aLumni satisfaction from a consumer's perspective: The effects of institutional performance and program outcomes. Research in Higher Education 36 (2): 197-217. Hemmens, C. 2002. Teaching criminal justice: Reply to a sociologist. ACJS Today November/December: 8-11. Hoey, J., and D. Gardner. 1999. Using surveys of alumni and their employers to improve an institution. New Directions for Institutional Research 101: 43-55. Jaschik, S. 2008, February 20. Donations are up, but not from alumni. Inside Higher Ed. Retrieved February 18, 2009, from http://www.insidehighered.com/news/2008/02/ 20/gifts Johnson, I., R. Sigler, and J. Crowley. 1994. Domestic violence: A comparative study of perceptions toward abuse cases among social service and criminal justice professionals. Journal of Criminal Justice 22: 237-248. Kanuha, V. 1998. Professional social work and the battered women's movement: Contextualizing the challenges of domestic violence work. Professional Development 1 (2): 4-18. Kennesaw State University. 2008. New program proposal: Master of science with a major in International Policy Management. Kennesaw, GA: Kennesaw State University. Landrum, R., and N. Lisenbe. 2008. Use of alumni perceptions to evaluate instructional and departmental quality. College Student Journal 42: 120-127. Leslie, L. L., and G. Ramey. 1988. Donor behavior and voluntary support for higher education institutions. Journal of Higher Education 59 (2): 115-132. Ma, A. 2007. Evaluating the curricula for entry-level care professionals in aging-related fields. Educational Gerontology 33: 881-896. Mayzer, R., and C. DeJong. 2003. Student satisfaction with distance education in a criminal justice graduate course. Journal of Criminal Justice Education 14: 37-52. McGovern, T. V., and K. F. Carr. 1989. A review of alumni surveys on undergraduate psychology majors. Teaching of Psychology 16: 52-57. Melchiori, G. S. 1988. Applying alumni research to fundraising. New Directions for Institutional Research 60: 51-65. Moriarty, L. 2006. Investing in quality: The current state of assessment in criminal justice. Justice Quarterly 23: 409-427. Muffo, J. 1996. Lessons learned from a decade of assessment (No. 397716). Buffalo, NY: Eric Document Reproduction Service. Myers, L. 1994. The evaluation of criminal justice programs. Journal of Criminal Justice Education 5: 31-48. Nagle. R., and J. Bohovich. 2000. What goes into graduates' career decisions? Journal of Career Planning and Employment 60: 24-30. OLd Dominion University. 2006. PhD proposal in criminology and criminal justice. Norfolk, VA: OLd Dominion University. Owen, S., H. Fradella, and T. Burke. 2006. Conceptualizing justice: Revising the introductory criminaL justice course. Journal of Criminal Justice Education 17: 3-22.


ALUMNI RESEARCH

265

Palomba, C. A., and T. W. Banta. 2001. Assessing student competence in accredited disciplines: Pioneering approaches to assessment in higher education. Sterling, VA: Stylus Publishing, LLC. Parks, J. B., and L. Parra. 1994. Job satisfaction of sport management alumnae. Journal of Sport Management 8: 49-56. Pettit, J. 1991. Listening to your alumni. Air Professional Files Online. Retrieved September 23, 2008, from http://www.airweb.org/page.asp?page=73Eapppage= 85&id=41 Pike, G. R. 1994. The relationship between alumni satisfaction and work experience. Research in Higher Education 35: 105-123. Porter, S., and P. D. Umbach. 2001. What works best? Collecting alumni data with multiple technologies (No. 456786). Buffalo, NY: Eric Document Reproduction Service. Proctor, J. 2006. Academic achievement and statistical knowledge. Journal of Criminal Justice Education 17: 143-161. Southerland, M. D. 2002. Criminal justice curricula in the United States: A decade of change. Justice Quarterly 19 (4): 589-601. Takata, S. 2006. The importance of integrating research experiences in today's undergraduate criminal justice curriculum. Journal of Crime and Justice 14: 77-98. Taylor, A. L., and J. C. Martin, Jr. 1995. Characteristics of alumni donors and nondonors at a research 1, public university. Research in Higher Education 36 (3): 283-302. Tsoudis, 0. 2000. Does majoring in criminal justice affect perceptions of criminal justice? Journal of Criminal Justice Education 11: 225-236. Velasco, S., S. E. StockdaLe, and D. Scrams. 1992. Sociology and other social sciences: California State University alumni ratings of the BA degree for development of employment skills. Teaching Sociology 20: 60-70. Weinstein, S., and R. F. Hartsook. 2002. The complete guide to fundraising management (2nd ed.). Wiley Nonprofit Law, Finance and Management Series 2: 350.
