Clinical psychology trainees' research ... - Wiley Online Library

Clinical Psychology and Psychotherapy Clin. Psychol. Psychother. 14, 54–62 (2007) Published online in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/cpp.513

Clinical Psychology Trainees’ Research Productivity and Publications: An Initial Survey and Contributing Factors Myra Cooper1* and Graham Turpin2 1 2

Oxford Training Course, University of Oxford North Trent, University of Sheffield Course

Clinical psychology research productivity is an issue of great professional concern. The current paper explores the rate and nature of trainee publication of research projects in peer-reviewed journals, together with reasons likely to be relevant to its success and failure. Twenty-one out of 28 courses responded to a survey composed of closed and open-ended questions. Data were analysed using inferential statistics and content analysis. Twenty-four percent of trainees in any one cohort during the period 1999–2004 successfully wrote up their research for publication. Most publications were in psychology journals with modest but respectable impact factors. The most important theme thought to be related to success was supervisor factors, followed by trainee factors, general course factors, study characteristics and the demands of a new job following qualification. Recommendations to enhance publication success are made within the context of a broader discussion of the importance of facilitating the development of the new generation of clinical psychology researchers. Copyright © 2007 John Wiley & Sons, Ltd.

INTRODUCTION Psychological research has been a major influence on the production of clinical guidelines within health care, particularly within mental health. These guidelines have advocated the efficacy, safety and acceptability of psychological interventions (e.g., National Institute for Clinical Excellence, 2005) and have also identified psychological research in a broad range of areas (e.g., relevant to service user engagement and involvement, service delivery and organization) as an important priority. Although as a subject area, clinical psychology performed well within the recent Research Assessment Exercise (RAE; Davey, 2002), the productivity of clinical psychologists as practitioners has * Correspondence to: Myra Cooper, Isis Education Centre, University of Oxford, Warneford Hospital, Oxford OX3 7JX. E-mail: [email protected]

Copyright © 2007 John Wiley & Sons, Ltd.

been a subject of keen debate and is one that has caused concern for some time (Salkovskis, 1984). In particular, it is apparent that very few clinicians in the National Health Service (NHS) manage to be research productive once qualified despite their extensive training in clinical research at a doctoral level (Milne, Britton, & Wilkinson, 1990; Milne, Keegan, Paxton, & Seth, 2000). The reason for the lack of research activity once qualified is not entirely clear and does not appear to have been systematically investigated (Milne & Paxton, 1998). More generally, those clinical psychologists who wish to pursue an academic or research career, either in parallel with clinical work or full time, often face significant difficulties in obtaining support for clinical psychology research within the academic Departments of Psychology and Psychiatry. These issues have been the subject of recent discussions (Thomas, Turpin, & Meyer, 2002; Turpin, 2002). Worryingly, it has been noted that

Psychology Trainee Research there is a growing problem with the recruitment of clinical psychologists to academic positions. This includes both research positions, and perhaps of even greater concern, academic positions on training courses, which provide the professional training of future generations of clinical psychologists (Morris & Turpin, 2004; Sperlinger & Burns, 2005). The research productivity of trainees on doctoral clinical psychology courses and the factors that might influence this, particularly once qualified are, therefore, an important topic. They might relate to the discipline’s contribution to future RAEs, to the decisions and behaviour of newly qualified staff and clinicians within the NHS about whether or not to take part in research activity. They may relate to the ability to recruit research productive clinicians to academic posts in years to come, including those who will take responsibility for the training of clinical psychologists to work in the NHS of the future. At the Group of Trainers in Clinical Psychology (GTiCP) conference in Buxton, 2004, the Research Tutor Forum spent some time discussing the importance of trainee research, its dissemination and, particularly, its publication in peer-reviewed journals. There is a broad perception in the training community and amongst clinical psychologists generally that few trainee research theses ever get written for publication (and, subsequently published in respectable journals). This is despite the fact that they are examined at a doctoral level where the potential for publication is an implicit and even an explicit quality standard. As already suggested, the reasons for this are not well understood since reports from courses, external examiners and external bodies frequently comment on the high quality of trainee research work, and trainees are very often encouraged to publish their thesis research work. Although there have been some surveys of publication rates amongst qualified clinical psychologists (Milne et al., 1990), to date, no survey of publications arising out of their research work has been conducted. We felt that an important first step in examining some of the issues surrounding clinical psychology research productivity would be to establish the current rate of trainee publication of theses projects in peer-reviewed journals and the nature of these publications. A secondary aim was to investigate some of the reasons likely to be relevant to lack of productivity by asking courses to comment on the reasons why trainees might or might not succeed in publishing their thesis project and by exploring themes associated with success and failure. Copyright © 2007 John Wiley & Sons, Ltd.

55

METHODS Participants Twenty-eight courses were contacted by the course e-mail system run through the British Psychological Society (BPS). A response to the survey was requested from a member of course staff involved specifically in the research training. Twenty-one replies were received. Four courses felt unable to compile a list of trainee publications (too busy/not enough staff = 2; new course = 1; no good system for getting the information = 1). Three courses did not reply, and no information was available on why they were unable to participate.

The Survey The survey asked for details (i.e., the full reference) of all papers published in peer-reviewed journals from trainee thesis projects, including ‘in press’ but not ‘submitted’ or ‘in preparation’ papers. This was an open-ended question. It requested information on the dissertation format (traditional Ph.D. or two paper), on the year of change in format (if any) and on the five most important factors determining trainee success and failure (asked separately) in obtaining a publication from their research training. These factors were generated by respondents and then rated for importance, where 0 = not at all important and 10 = extremely important. The name of the course and of the person returning the survey were collected. Information on the RAE ranking of departments was obtained from http://www.hero.ac.uk/rae, and data on the impact factor of journals was obtained from either the publisher’s web site or the Web of Knowledge (http://www.isiwebofknowledge.com). Data on course intake for 2000 were obtained from the Clearing House Handbook for that year (University of Leeds, 2000).

Data Analysis Quantitative data were collated, and means and Standard Deviations (SDs) were calculated, as appropriate, to summarize the data. Where between group comparisons were made, chisquare tests were conducted. Content analysis was performed on the themes generated in response to the success and failure questions, following the procedure outlined by Krippendorff (1980). Missing data were recorded for one course for RAE rating and for two courses for the number of Clin. Psychol. Psychother. 14, 54–62 (2007) DOI: 10.1002/cpp

56

M. Cooper and G. Turpin

trainees taken in 2000. All other data reported below include all 17 courses.

RESULTS Publication Rates and Nature Courses were asked to include publications since 1999 that were published or in press with peerreviewed journals, and that had resulted from trainee dissertations only (and not small-scale projects). The survey was conducted in early 2005. The mean number of publications for the 17 courses was 16.9 (SD = 11.8, range 3–47) for approximately a 5-year period, and the grand total was 373. The mean number of publications per course per year was 3.4 (SD = 2.4). Given that courses differ greatly in number of trainees, a score was computed that reflected the number of publications per trainee. Using the actual intake figures for each course in 2000, and multiplying this by 5, gave an estimate of the total number of trainees during that period. The number of publications per trainee was then calculated. This calculation yielded a mean ratio of trainee numbers to publications of 1:0.24 (SD = Table 1.

0.26) (over the period 1999–2004). In other words, approximately one out of four trainees during this period successfully wrote their thesis for publication. This is probably an underestimate since most courses had smaller cohorts before 2000, and some of the publications here will have been from trainees completing their course prior to 2000.

Which Journals and Impact Factors The three most popular journals for publication were the British Journal of Clinical Psychology (18 publications), Clinical Psychology and Psychotherapy (11 publications) and Behavioural and Cognitive Psychotherapy (10 publications). Two journals received seven publications each: the British Journal of Anaesthesia and the Journal of Reproductive and Infant Psychology. In total, 11 journals received more than four publications each—all were primarily psychology journals with the exception of the British Journal of Anaesthesia, the International Journal of Eating Disorders and the Journal of Applied Research in Learning Disabilities. A list of the most popular journals and the number of papers published in each journal are presented in Table 1.

Most popular journals published in by trainee dissertation projects

Journal British Journal of Clinical Psychology Clinical Psychology and Psychotherapy Behavioural and Cognitive Psychotherapy Journal of Reproductive & Infant Psychology British Journal of Anaesthesia Behaviour Research and Therapy International Journal of Eating Disorders Clinical Child Psychology & Psychiatry Journal of Health Psychology Psychology, Health & Medicine Journal of Applied Research in Intellectual Disabilities Ageing and Mental Health Brain Injury British Journal of Health Psychology Clinical Psychology Review Dementia Clinical Psychology (Forum) Cognitive Therapy and Research Journal of Community & Applied Social Psychology Pain European Journal of Pain Personality & Individual Differences Journal of Child Psychology, Psychiatry and Allied Disciplines Journal of Intellectual Disability Research

No. of trainee publications

Citation impact rating (2004/2005)a

18 11 10 7 7 6 5 4 4 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3

1.12 0.52 N/A 0.46 2.47 2.36 1.87 2.78 0.68 1.68 1.07 1.06 1.14 1.02 2.86 N/A N/A 1.19 0.66 4.06 1.81 1.32 2.78 1.03

a

Obtained from Web of Knowledge or publisher’s web site. N/A = not applicable.


Clin. Psychol. Psychother. 14, 54–62 (2007) DOI: 10.1002/cpp

Psychology Trainee Research

57

Table 1 also shows the impact factor (measure of the frequency with which the ‘average article’ in a journal has been cited) for 2005. It should be noted that the impact ratings of these journals tend to be modest but respectable for psychology journals. There are few papers published in highly esteemed psychology journals (American Psychology Association publications or impact factors above 2.0), although some medical journals had higher impact ratings, as to be expected.

Factors Associated With Successful Publication The following factors were examined to see if they were associated with successful publication: (1) thesis format (traditional Ph.D. vs. journal paper); (2) size of course; (3) status of the university (old vs. new); (4) RAE rating of the department hosting the course; and (5) whether or not the university was a member of the Russell Group. The traditional Ph.D. format courses (n = 10) produced a total of 216 papers (M = 21.5, SD = 14.3, range = 6–47) publications; the journal paper courses (n = 4) produced a total of 92 publications (M = 11, SD = 4.8, range = 4–15). This is a trainee : publication ratio of 1 : 1 for the Ph.D. format and a ratio of 1 : 0.48 for the journal paper format, i.e., adjusting for size of course. The difference was not statistically significant. Size of course was also examined in more detail. Median splits were used to form two groups, one of seven (N < 23 trainees) and one of eight courses (N > 23 trainees), and a comparison of the number and ratio of trainee : publications, including a ratio adjusted for course size, were calculated. Smaller courses (N = 7) had a mean of 20.3 publications, SD = 15.9, while larger courses (N = 8) had a mean of 19.8, SD = 9.2. Respective trainee : publication ratios were: smaller courses 1 : 1.1 and larger courses 1:0.50. This difference showed a trend towards statistical significance (χ2 = 1.15, df = 14, p = 0.07). The statuses of the university—old vs. new; Russell Group or not; RAE score—were not asso-

Table 2.

ciated with publication rates on different courses (adjusting for size of course in all calculations). However, only three universities fell into the ‘new’ category, i.e., established since 1992. The ratio score for those in the Russell Group was 0.939, and for those not in the Russell Group, it was 0.586. Those with the highest RAE ratings (5*) had a ratio of 0.950, while those with lower ratings had a ratio of 0.574. However, numbers were too small, and power too limited, to permit any serious examination of the relative influence of each of these variables. Many confounds between these variables are also likely to exist in the data set.

Perceived Indicators of Success and Failure to Publish Courses were also asked to complete open-ended questions in which they listed up to five factors that they felt contributed most to success and failure to publish dissertation research. Each factor was then rated on a 10-point scale from 0 = ‘not at all important’ to 10 = ‘extremely important’ in order to establish how important each factor was considered to be by each course. Ratings were made by a member of course staff involved in the research training. Eighteen courses completed this part of the survey. Content analysis was conducted on the items. For success, five themes emerged, and mean ratings of importance were calculated for each theme. The themes for success were: (1) supervisor factors; (2) trainee factors; (3) general course factors; (4) study characteristics; and (5) demands of a new job following qualification. Mean importance ratings for each factor can be seen in Table 2. The number of items in each category is in brackets under the category title. The most important theme was thought to be supervisor factors. Examples of items from this theme (and others) are presented in Table 3. The themes for failure were: (1) supervisor factors; (2) study characteristics; (3) demands of a new job following qualification; and (4) ‘trauma’ or trainee-related factors. The latter often referred to

Success factor ratings Supervisor (N = 22)

Trainee (N = 19)

Course (N = 6)

Study (N = 15)

New job (N = 8)

M SD 7.9 2.3

M SD 7.5 1.7

M SD 6.5 1.5

M SD 7.2 1.5

M SD 6.1 1.7

Importance Rating (0–10) SD = Standard Deviation.



58

M. Cooper and G. Turpin

difficulties experienced by trainees getting their research approved by NHS Local Research Ethics Committees, and which had decreased their motivation to follow up on their research and to pursue publication. Mean importance ratings for each factor can be seen in Table 4. The number of items in each category is in brackets under the category title. The most important factor was thought to be study characteristics. Examples of items from each theme can be seen in Table 5.

DISCUSSION In summary, a relatively large number of papers in absolute terms, (around 75 per year) are being published by clinical psychology trainees based on their thesis research. We suspect that no other branch of health professions training, including medicine or membership training for the Royal College of Psychiatrists, would match this research productivity within training. However, only Table 3.

Examples of items from success themes

Supervisor Supervisor support, motivation, ‘chasing trainee’, writing it for trainee, part of tutor’s research programme, staff expectations, good supervisory relationship etc. Trainee Trainee enjoyment, belief research is worthwhile, high level of research skills, high level of confidence in research skills etc. Course Remaining in touch with course, positive viva feedback/experience, publication suggested by course/examiners, thesis format. Study characteristics Adequate power, positive findings, strong methodology, quantitative project, strong theoretically, high-quality study. New job Continuing Professional Development (CPD) time and support available, job in related area, workplace support, location near course, write within 2 years.

Table 4.

around a quarter of all 3rd-year trainees successfully publish their thesis. Unfortunately, the survey did not address submission and rejection data, so we are unable to comment on the 75% of trainees who appear not to have published. Anecdotally, many supervisors have copies of submissions that were rejected the first time around from a journal and were not resubmitted. Trainees are also publishing other pieces of work such as small-scale research, single cases and audit. Some also get involved in writing book chapters in collaboration with course staff during training. These were not surveyed here. The small numbers make it difficult to identify confidently the factors that contribute to publication success, and we view this survey as an initial attempt to identify some of the factors that may be involved. Future surveys need to be conducted when courses have collected more data on publications, and which may have been more reliably recorded in the future. From comments made by those who completed the survey, it was clear that most courses did not routinely collect such data; thus, its reliability is unclear. While it is conceivable that the validity of the data was problematic,

Table 5.

Examples of items from failure themes

Supervisor Lack of supervisor motivation, supervisor too busy to follow up on trainee, research not part of tutor’s programme, lack of trainee awareness of supervisor need to publish etc. Study Poor results, design, inadequate data set, findings not interesting, poor design, obscure topic etc. New job Research not valued or emphasized in new job/NHS, lack of time and support, low expectation of finding time for research in new job, little knowledge of how to publish etc. ‘Trauma’ and trainee factors Trainee disenchanted with research process, ‘traumatized’ by it, exhausted after training, little interest in research (means to an end only), no interest/motivation, lack of cooperation with coauthors etc.

Failure factor ratings Supervisor (N = 7)

Study (N = 12)

New Job (N = 17)

Trauma/trainee (N = 16)

M SD 7.3 1.7

M SD 8.1 2.1

M SD 6.9 1.8

M SD 7.2 2.4

Importance Rating (0–10) SD = Standard Deviation.



Psychology Trainee Research this seems unlikely given that all courses provided full references for all publications, and only those published or accepted for publication were included. Nevertheless, some preliminary comments on possible factors may be helpful. The commonly held belief that the journal paper format (review and/or research reports written in journal paper format) leads to more publications was not supported. However, the timing of the survey was such that many courses were in the process of moving to a journal paper format. Only four reported using this format during the period spanned by the study, whereas a recent, unpublished e-mail survey of GTiCP courses, undertaken by the authors, found that 17/24 (71%) of courses had adopted the new format. It is unlikely that many trainees in the current survey would have had the opportunity to benefit from this new format. It is also possible that courses that may have experienced problems encouraging trainees to publish their thesis project may have deliberately changed formats. There are also trends towards the RAE rating and membership of the Russell Group being associated with more publications, and these courses might be expected to retain the more traditional thesis format, especially if their doctorate award is under the auspices of the University Higher Degrees Committee. The qualitative feedback from courses suggests that a range of factors are associated with trainee success, including both the enthusiasm of the supervisor and the trainee’s enjoyment and past experiences of research, as well as the quality of the final product and the time made available in the first job after qualifying. The absence of these factors, and including some who may feel traumatized by the process, is associated with failure to publish. However, it should be emphasized that these factors were identified by course staff, and there is an urgent need to survey trainees individually following graduation about the nature of the research and publication process.

Evaluation What, therefore, is the status of trainee research productivity on courses? In absolute terms, it is apparent that doctoral training in clinical psychology is making a contribution to the knowledge base in clinical psychology. The annual publication of some 75 journal publications in mediumranking peer-reviewed journals must be seen as an indicator of success for training courses. To a Copyright © 2007 John Wiley & Sons, Ltd.

59 degree, this answers the critics who question the value of the doctoral degree and the importance of the thesis component within professional training (Smith, 2005). The valuable contribution provided by trainee research also provides some justifications for the significant NHS investment in clinical training, which is perceived to be more expensive than that of many other health care professions in training. However, it should also be remembered that the investment in research support for each trainee is usually quite modest; a value of around £250–£500 per thesis is usual on most courses. From an economic perspective, the publications produced via clinical training thus have very low resource costs compared to either Ph.D. studentships or research programme grants. Taking a more critical stance, it could be argued that much of the research is published in modest impact-rated journals and lacks the thematic coherence of a research programme. Moreover, it is also a fact that the majority of trainee research is not being published either due to initial poor quality without the potential for publication, failure to submit, or rejection during the submission process.

RECOMMENDATIONS The results are heartening, but we should not be complacent about them. We need to consider ways of enhancing both the quantity and quality of trainee publications as well as ways to encourage greater research productivity amongst qualified clinical psychologists. Although based on an initial analysis of the problems of trainee research publication and the factors influencing this, we would like to suggest the following measures that might further enhance publication success:

Enhancing Output and Impact We suggest that the following might help raise the quality of clinical psychology doctoral research either to achieve higher publication rates or to encourage publication in higher impact-rated journals: 1. Doctoral research should be supervised by clinical or other psychologists experienced in research. 2. Supervision loads (including all forms of supervision such as undergraduate projects, Ph.D.s, clinical supervision etc.) should be consistent with the Higher Education Funding Clin. Psychol. Psychother. 14, 54–62 (2007) DOI: 10.1002/cpp

60

3.

4.

5.

6.

7.

M. Cooper and G. Turpin Council for England good practice guidance for higher degrees (Higher Education Funding Council for England, 2004). Any full-time staff member should be discouraged from having more than six significant supervisory responsibilities. Staffing levels on Training courses should allow staff to be research active, in their own right, and available to provide good quality supervision to trainees (see above on staffing levels). Adequate resources should be provided. This will mean that research support costs have to be clearly identified in any future agreed National Benchmark Price within the NHS. Some projects could be raised in quality by provision of additional resources through either charitable or NHS R&D funds allowing, for example, the collection of larger sample sizes, dealing with the transcription or data processing costs of more participants etc. Consider organizing research projects so that they follow the research interests and expertise of course staff (c.f. Thomas et al., 2002) rather than the trainees’ own interests. The interest and enthusiasm of the supervisor might then also enthuse the trainee, plus the supervisor may have a greater interest in achieving a publication. Organize projects thematically either within cohorts or across cohorts so as to facilitate the production of a series of studies developing a research theme or question. Such studies might be published collectively in an attempt to access the highest-ranking journals whose study requirements are usually beyond the single doctoral thesis.

Quantity Based on the qualitative feedback obtained in the survey, we suggest that the following might encourage more trainees to write and submit their work for publication: 1. Enhancing university support on graduation by creating Honorary Research Associate/ Fellow posts for ex-trainees actively engaged in writing. This enables recognition for the activity and access to library and computing facilities. 2. Creating academic training or clinical postdoctoral fellowships (see later). 3. Time for course staff to continue with the supervision and to help in the writing for pubCopyright © 2007 John Wiley & Sons, Ltd.

lication. Anecdotally, writing for publication is often much harder for newly qualified trainees to achieve successfully than they anticipate, particularly as few have any previous experience of successful publication. First publications often need considerable ‘mentoring’, especially following referee feedback and resubmission. 4. Support in the first NHS post when qualifying. Anecdotally, the availability of study days/ leave for publication purposes within the first NHS posts facilitates publication. 5. Greater recognition of the importance of ongoing research for career development within the NHS through the use of the Knowledge and Skills Framework, and recognition of research and publication through monitoring Continuing Professional Development. 6. Creating a supportive and positive course culture around the importance of clinical research and minimizing negative aspects of research such as poor supervision or overbureaucratic research ethics and governance procedures (Cooper, Turpin, Bucks, & Kent, 2005; Department of Health, 2006; Peck & Jones, 2004).

The New Generation of Clinical Psychology Researchers There are a series of initiatives that the British Psychological Society, the NHS and Higher Education Institutions should consider in order to enhance clinical academic careers. As we stressed within the introduction, the recruitment of clinical staff to professional training courses is problematic and is becoming increasingly difficult to achieve, which may mean that it is likely to be one of the most important limiting factors to any future expansion of clinical psychology training (British Psychological Society, 2005; Layard, 2005). This situation is not unique to clinical psychology but is a major human resource issue across medicine (Academy of Medical Sciences, 2005; Higher Education Policy Institute, 2005), nursing and the allied health professions (Ferguson, Owen, Beswick, & Baguley, 2004; StLaR, 2005). The development of future generations of NHS researchers has also been recently recognized by the Department of Health (2006). General issues such as differential levels of remuneration and terms and conditions of service between Higher Education Institution and NHS sectors, inflexibility of employers in allowing cross Clin. Psychol. Psychother. 14, 54–62 (2007) DOI: 10.1002/cpp

Psychology Trainee Research sector working, lack of attractiveness of clinical academic careers for students and trainees, and the absence of training posts in academe on qualification, as well as a poorly identified career structure, are just some of the problems identified to date. A major investment is taking or about to take place in improving medical and nursing careers (UKCRC, 2005; Walport Report, 2005), which includes the creation of a national programme of training fellowships to retain and attract the best quality staff within clinical education. The status of NHS clinicians who are research active has also been the subject of a recent NHS R&D consultation and paper (Department of Health, 2005, 2006; Paxton, 2006). We suggest that similar steps urgently need to happen for clinical and the other applied psychologies. If qualifying clinical psychologists could be offered training posts or post-doctoral positions (such as those being suggested for medical and nursing colleagues) whereby they could continue their doctoral research interests, we would anticipate a major improvement in the quality and quantity of research productivity, as well as a much stronger base from which to recruit the clinical academic staff of the future. It is our belief that unless such initiatives are put into place, there will be a significant decline in the contribution that clinical psychology currently makes to the development of clinical guidelines and NHS research priorities; a decline that could well have a negative impact on both research quality and ultimately upon the effectiveness of psychological health care.

ACKNOWLEDGEMENT The help and support of the Research Tutors Forum of the Group of Trainers in Clinical Psychology in completing this survey is gratefully acknowledged.

REFERENCES Academy of Medical Sciences. (2005). Strengthening clinical research. Retrieved October 20, 2006, from http://www. acmedsci.ac.uk/images/publication/pscr.pdf British Psychological Society. (2005). Recommendations for expanding numbers of applied psychologists 2005/2010. Workforce Planning Advisory Group, January 2005. Cooper, M.J., Turpin, G., Bucks, R., & Kent, G. (2005). Good practice guidelines for the conduct of psychological research within the NHS. Leicester, UK: British Psychological Society. Davey, G. (2002). President’s column: Clinical research— worth our support. The Psychologist, 15, 331.


61 Department of Health. (2005). Best research for best health: A new national health research strategy—consultation. Retrieved October 20, 2006, from http://www. dh.gov.uk/Consultations/LiveConsultations Department of Health. (2006). Best research for best health: A new national health research strategy. Retrieved October 20, 2006, from http://www.dh.gov.uk/ researchstrategy Ferguson, K., Owen, S., Beswick, S., & Baguley, I. (2004). Time to act—choosing to work in mental health: The recruitment of health and social care professionals. The Centre for Clinical Academic Workforce Innovation, University of Lincoln. Higher Education Funding Council for England. (2004). Improving standards in postgraduate research degree programmes: Statement on progress. Retrieved October 20, 2006, from http://www.hefce.ac.uk/pubs/circlets/ 2004/cl04_04/ Higher Education Policy Institute. (2005). The education and training of medical and health professionals in higher education institutions. Retrieved October 20, 2006, from http://www.hepi.ac.uk/pubs.asp?DOC=Reports Higher Education and Research Opportunities in the United Kingdom (2001). 2001 Research Assessment Exercise: The Outcome. Retrieved October 20, 2006, from http://hero.ac.uk/rae Krippendorff, K. (1980). Content analysis: An introduction to its methodology. Newbury Park, CA and London: Sage. Layard, R. (2005). Therapy for all on the NHS. Sainsbury Centre Lecture. Retrieved October 20, 2006, from http://www.scmh.org.uk/website/SCMH_News.nsf/ 59ae2dc0a615100a802567e4004f2d1e/747f1ee04a13adca 80257077004e8b11/$FILE/Layard%20Lecture%20SCM H%20120905.doc Milne, D., & Paxton, R. (1998). A psychological reanalysis of the Scientist-Practitioner Model. Clinical Psychology and Psychotherapy, 5, 216–230. Milne, D., Keegan, D., Paxton, R., & Seth, K. (2000). Is the practice of psychological therapists evidence-based? International Journal of Health Care Quality Assurance, 13, 8–14. Milne, D.L., Britton, P.G., & Wilkinson, I. (1990). The scientist-practitioner in practice. Clinical Psychology Forum, 30, 27–30. Morris, R., & Turpin, G. (2004). Employment of clinical psychology training staff: A national survey. Clinical Psychology, 38, 55–59. National Institute for Clinical Excellence. (2005). Published guidelines and cancer service guidance. Retrieved October 20, 2006, from http://www.nice.org.uk/ page.aspx?o=guidelines.completed Paxton, R. (2006). Research in the NHS: Survival of the biggest. The Psychologist, 19, 284–286. Peck, D., & Jones, A. (2004). Bureaucratic barriers to research and research training in the NHS. Clinical Psychology, 36, 7–10. Salkovskis, P.M. (1984). Psychological research by NHS clinical psychologists: An analysis and some suggestions. Bulletin of the British Psychological Society, 37, 375–377. Smith, M. (2005). DClin Sigh? Clinical Psychology, 46, 34– 37.


62 Sperlinger, D., & Burns, J. (2005). Clinical psychology staffing survey: Vacancies, remuneration and pay scales. London, UK: GTiCP, British Psychological Society. StLaR. (2005). Support for learning and research in health and social care. (StLaR) Human Resources Plan. Retrieved October 20, 2006, from http://www.stlarhr.org.uk/ index.asp?pgid=3029 Thomas, G., Turpin, G., & Meyer, C. (2002). Clinical research under threat. The Psychologist, 15, 286– 289. Turpin, G. (2002). Fostering clinical research. The Psychologist, 15, 505–506. UKCRC. (2005). Building up the research workforce. UKCRC Update, 2, 7. Retrieved October 20, 2006, from http://www.ukcrc.org


M. Cooper and G. Turpin University of Leeds. (2000). Clearing house handbook for clinical psychology training courses. Leeds, UK: University of Leeds. Walport Report. (2005). Medically- and dentallyqualified academic staff: Recommendations for training the researchers and educators of the future. Report of the Academic Careers Sub-Committee of Modernising Medical Careers and the UK clinical Research Collaboration. Retrieved October 20, 2006, from http://www.academicmedicine.ac.uk/uploads/ Medically-and-Dentally-Qualified-academic-staffrecommendations-Report.pdf Web of Knowledge (2006). Journal Citation Reports. Retrieved October 20, 2006 from http://www. isiwebofknowledge.com
