Planning for the South African National Income. Dynamics Study ..... The Panel Study of Income Dynamics (PSID) ............................................................47 ..... different sources of income such as wage income, remittances and social grants? .... questions that are consistent with the October Household Survey (OHS) which ran.
Planning for the South African National Income Dynamics Study (NIDS): lessons from the international experience Ingrid Woolard & Murray Leibbrandt, Southern Africa Labour & Development Research Unit (SALDRU), University of Cape Town Assisted by Deborah Lee, Dept of Economics, Nelson Mandela Metropolitan University
Final draft, 6 February 2006
Office of the Pr esid ency
Department of Social Development
This project was funded by the Foreign Assistance Agenci es of Australia (AusAID), the UK (DFID), and the United States of America (USAID), and b y the United Nations Development Programme (UNDP) of South Africa. The management and technical assistance was provided by the Joint Economics AIDS and Poverty Programme (JEAPP), which is affiliated to the African Asian Society (AAS). We thank Vusi Gumede, Mastoera Sadan and Hilary Southall fo r help ful comments on earlier drafts.
1
TABLE OF CONTENTS Executive Summary .......................................................................................................6 Introduction....................................................................................................................8 Why South Africa needs a household panel survey.....................................................10 The importance of longitudinal studies........................................................................14 Repeated cross-sectional studies..........................................................................14 Panel surveys........................................................................................................15 Retrospective studies............................................................................................15 Concepts and Definitions.........................................................................................17 Advantages and Usefulness of Panel Surveys .........................................................17 Topics covered .........................................................................................................19 Income and Expenditure Dynamics .....................................................................19 Determinants of Poverty and Wellbeing..............................................................19 Controlling for Unobserved Heterogeneity..........................................................19 Organisational issues in Household-based Panel Surveys.......................................21 Problems and Difficulties with Panel Surveys.........................................................21 Issues in the Planning of Panel Surveys...................................................................22 Budget Planning...................................................................................................22 Data Production and Processing ..........................................................................22 Data Analysis.......................................................................................................23 Documentation and Dissemination ......................................................................23 Challenging Issues in the Design of Panel Surveys.................................................23 Attrition and Non-response..................................................................................24 Panel Conditioning...............................................................................................24 M anagement Strategies and Techniques ..............................................................25 Following Rules ...................................................................................................26 Sample size ..........................................................................................................27 Refreshing the sample..........................................................................................28 Adding new units to maintain sample size...........................................................28 Adding new rotation groups.................................................................................29
2
M aintaining high response rates across waves ....................................................29 M aintaining contact with households between waves .........................................29 Providing incentives .............................................................................................30 Retaining wave non-respondents .........................................................................30 M odifying the questionnaires across waves.........................................................30 Recommendations ....................................................................................................32 Conclusion ...............................................................Error! Bookmark not defined. References ............................................................................................................34 Appendix 1A: International Case Studies ....................................................................37 The British Household Panel Survey ...........................................................................37 Background Information ..........................................................................................37 Sample specifications...............................................................................................37 Who is followed? .................................................................................................37 M ain themes/modules analysed ...............................................................................38 Operational considerations.......................................................................................39 Access to the Data................................................................................................39 Conclusions ..............................................................................................................39 The German Socio-Economic Panel ............................................................................40 Background information ..........................................................................................40 Sample specifications...............................................................................................40 M ain themes/modules analysed in survey ...............................................................40 Operational considerations.......................................................................................41 The Indonesian Family Life Survey.............................................................................42 Background Information ..........................................................................................42 Sample specifications...............................................................................................42 Who is followed? .................................................................................................42 M ain themes/modules analysed ...............................................................................42 Operational considerations.......................................................................................43 Access to the Data................................................................................................43 Conclusions ..............................................................................................................44
3
The M exican Family Life Survey ................................................................................45 Background Information ..........................................................................................45 Sample specifications...............................................................................................45 Who is followed? .................................................................................................45 M ain themes/modules analysed ...............................................................................45 Operational considerations.......................................................................................46 Access to the Data................................................................................................46 The Panel Study of Income Dynamics (PSID) ............................................................47 Background Information ..........................................................................................47 Sample specifications...............................................................................................47 Who is followed? .................................................................................................47 M ain themes/modules analysed ...............................................................................47 Operational considerations.......................................................................................48 Access to the Data................................................................................................48 Conclusions ..............................................................................................................49 Appendix 1B: South African panel surveys.................................................................50 Kwa-Zulu-Natal Income Dynamics Study...................................................................50 Background information ..........................................................................................50 Sample specifications...............................................................................................50 M ain themes/modules analysed in survey ...............................................................50 Operational considerations.......................................................................................51 Funding ................................................................................................................51 Africa Centre Demographic Surveillance System.......................................................52 Background information ..........................................................................................52 Sample specifications...............................................................................................52 M ain themes/modules analysed in survey ...............................................................52 Operational considerations.......................................................................................52 Fieldwork .............................................................................................................52 Data Capture ........................................................................................................53 Funding ................................................................................................................53
4
Agincourt demographic surveillance ...........................................................................54 Background information ..........................................................................................54 Sample specifications...............................................................................................54 M ain themes/modules analysed in survey ...............................................................54 Operational considerations.......................................................................................54 Field Procedures ...................................................................................................54 Data M anagement and Analysis ..........................................................................55 Cape Area Panel Study ................................................................................................57 Background information ..........................................................................................57 Sample specifications...............................................................................................57 M ain themes/modules analysed in survey ...............................................................57 Operational considerations.......................................................................................57 Funding ................................................................................................................57 Appendix 2...................................................................................................................59 Appendix 3...................................................................................................................82
5
Executive Summary 1. Household-based panel studies are now conducted in most industrial nations and several developing countries. 2. These studies, however, vary quite markedly in terms of design, collection method and strategies adopted for both processing and disseminating data. 3. There is no single approach that is universally accepted as “best practice”. 4. The data from many of these household panel studies have stimulated a large amount of research analysing change and dynamic behaviour. This, in turn, has fed into improved policy-making. 5. We propose that the survey design for NIDS be loosely based on the design utilised in Australia, Britain, Indonesia and M exico. 6. Despite the advantages of panel studies, it must be stressed that unlike crosssectional surveys that are always representative samples for their particular point in time, longitudinal surveys cease to be representative of the overall population after their first survey round. The representativeness of subsequent rounds is diminished due to the sample attrition that inevitably occurs over time. 7. Representativeness is further diminished in rapidly changing countries. For NIDS, consideration needs to be given to eventually adding a refresher sample for recent immigrants. 8. Panel surveys are complex in both administration and operation. The longterm analytical goals of the survey must be considered from the very beginning together with the financial budget. 9. M ost national household panels are conducted annually but we argue that this would probably not be possible given the lack of human resource capacity in South Africa. 10. International practice is that during the first few waves at least, panel surveys are conducted on a face-to-face basis. Face-to-face interviewing is generally thought to be more successful in eliciting cooperation, which is vital during the earliest years of the panel when sample member identification with the study is still developing. 11. Because of the importance of comparability over time, the phrasing of questions should only be changed in subsequent waves if absolutely necessary. For this reason, it is essential that the instrument used in Wave 1 is taken through a lengthy process of review and pre-testing. 12. The time imposition on respondents is an issue that needs to be considered in instrument design. A central feature of pre-testing and piloting is the insurance that each question efficiently is actually gathering the intended information and that the instrument is easily administered and is not too long.
6
13. Because of the importance of comparability over time, the phrasing of questions should only be changed in subsequent waves if absolutely necessary. For this reason, it is essential that the instrument used in Wave 1 is taken through a lengthy process of review and pre-testing. 14. Based on international experience, the content of NIDS can be expected to change slightly over time, as it evolves in line with what are the major economic and social issues of the day. 15. We recommend an “indefinite life” (as opposed to rotating panel) design which means that the original sample members remain in the sample for the duration of their lives. Following international precedent, the original panel can be refreshed periodically to retain its representivity. 16. Our proposed “following rules” are: (a) All members of the original selected households (including the children) should be tracked indefinitely (these are known as continuing sample members or CSM s); (b) any children born to or adopted by members of the selected households become CSM s; (c) new household members resulting from changes in the composition of the original households become temporary sample members or TSM s; and (d) all female TSM s who have a child by a CSM are converted to CSM status. 17. The keys to achieving high response rates and low rates of attrition appear to be at least fourfold: (i) use of respondent incentives; (ii) a long field period; (iii) a committed and motivated interviewer workforce; and (iv) a fieldwork agency that works to academic standards. 18. Other strategies identified as influencing response and attrition rates include: distribution of marketing material in advance of each survey round; allocating considerable effort to converting refusers; distributing feedback to sample members; assigning the best interviewers to the most difficult cases; and providing financial incentives to the fieldwork agency for achieving high response rates. 19. International experience strongly suggests that a realistic timetable is important in delivering a high quality product. Most studies, for example, provide for a two-year planning period. Second, the fieldwork period for each wave typically extends up to anywhere from 6 to 9 months. Third, most studies allow at least 9 months for the processing of data from each wave.
7
Introduction Large scale, nationally representative household surveys are relatively new in South Africa, having only really taken off in the last decade. Nevertheless, there is now a wide range of both regular and ad hoc household survey data which have contributed to our knowledge of the living conditions of South African households. For example, cross-sectional surveys such as Statistics South Africa’s (Stats SA) General 1 Household Surveys, Labour Force Surveys and Income and Expenditure Surveys provide information about the levels of well-being of sub-groups at any point in time. They do not, however, permit an analysis of whether the same households are consistently poor on non-poor or whether there is a high degree of upward and downward mobility. It is for this reason that most developed countries and several developing countries now engage in household panel surveys in order to better understand social change, income mobility and poverty dynamics. Since 1993 there has been an explosion of data, almost all from cross-sectional surveys, i.e. surveys for which a cross-section of the population is interviewed, once, providing a snapshot of conditions or attitudes at one point in time. Such surveys have generated a wealth of findings on life in South Africa. But they are much better in answering 'what?-type questions than 'why?'- or 'how?-type questions. For example, a cross-sectional survey can tell us what proportion of women are employed at any given point in time, but it cannot tell us whether the same women are moving into and out of employment and what life events preface, accompany or follow on from these changes in labour market status. Cross-sectional surveys cannot answer a number of important dynamic questions. For example, repeated Income & Expenditure Surveys can tell us whether poverty rates are decreasing, increasing or holding level. But these surveys cannot tell us about the fate of individual households over time; who is getting ahead and who is falling behind over time and why. Suppose two Income & Expenditure Surveys reveal that the poverty rate is the same in each period. This could be the result of the same households having been in poverty in both time periods. Alternatively, it may be that some households exited poverty over the period, while an equal number entered. Such distinctions, missed by cross-sectional surveys, might be very important in determining an effective policy response which may differ for chronic versus transitory poverty (Chaudhuri and Ravallion, 1994). Panel surveys study the same group of households or individuals over time. This has pushed poverty analysis to a broader and advanced frontier. In particular, panel surveys allow researchers to: • • •
analyse income and consumption dynamics investigate the causes of poverty; and control for unobserved heterogeneity.
1
The Labour Force Survey is techni cally a rotating panel o f dwellings, but owing to matching problems is invariably treated as a cross-sectional survey.
8
South Africa is a society undergoing rapid economic, political and social transformation. Therefore, with increasing urgency policy makers are keen to learn about the emerging patterns of socio-economic mobility, how South Africans have coped with these various changes and how they have been helped by new policies, something which the cross-sectional surveys alone cannot do. Only with panel data is it possible to control for unobserved heterogeneity. This means that in the study of correlation and/or causality when there are more explanatory variables than the ones observed (which is usually the case) and when the influence of these unobserved factors is constant over time, it is possible to control for their effects using panel data. This implies that panel studies are particularly important for monitoring and evaluation purposes. A panel survey observes households and individuals both before and after a sudden change in their circumstances or their participation in a government programme. This allows for a richer and more precise assessment of the impact of the programme or the shock. Panel surveys are, however, more costly and complex than cross-sectional studies and this imposes several challenges in terms of planning and design. Issues related to planning and management include reconciling long-term needs and short-term resources, ensuring the comparability of measures from wave to wave, combining longitudinal analysis with cross-sectional analysis, documenting survey activities, and disseminating the outputs in a timely fashion. These issues are all discussed in detail below and we make some specific recommendations based on a review of the 2 international experience. In total, we analysed the practices in 18 countries .
2
Appendix B summarises our findings for Australia, Belgium, Canada, France, Germany, Hungary, Indonesia, Luxembourg, Mexico, the Netherlands, Poland, Korea, Russia, Switzerland, Sweden, Taiwan, the UK and the USA.
9
Why South Africa needs a household panel survey An analysis of economic mobility throws a unique light on the unfolding of South African socio-economic dynamics. Such analysis shows who is getting ahead, who is falling behind and who is staying put where they are. The analysis of mobility in South Africa requires a national panel study. The mobility perspective has proven very useful in giving attention to poverty dynamics but it is certainly not limited to such dynamics. It as much about which households and which individuals move from the middle to the upper end of the distribution of well-being and why such moves occur as it is about who moves into poverty, who escapes poverty and who stays in poverty. To describe and explain poverty one has to compare the poor and the non-poor. It is crucial to know how the income sources and expenditure patterns of the better off contrast to those of the poor. In a static, descriptive sense such comparisons can be done with the available census data and national cross-sectional survey instruments that are produced by Statistics South Africa. However, if one wants to understand how these static conditions feed into a perpetuation of poverty or what sort of events really put the worse off on the road to something better or the better off on a downward path then one has to have panel data to track these experiences in actual households and individuals through time. Such tracking enables the isolation of cause and effect. Internationally, the analysis of the impact of positive or negative income or asset shocks using panel data have proved to be very useful in understanding household coping and adjustment mechanisms. Cross sectional data may suggest that households have the assets or savings to smooth the shock of a death in the family or the loss of a job. However, such questions can only be answered decisively by observing how people and households cope when these shocks arrive. The same is true of the analysis of a positive change in a household. Panel data is needed to analyse the impact of the employment of a family member or access to an old age pension or another social grant or improved electricity and road provision. Consider the following example. Large national cross-sectional surveys can tell us whether access to education is improving, who is paying for this and whether this is leading to more people accumulating more education. However, if one wants to understand whether these improvements in human capital endowments are leading to improvements in labour market outcomes, the cross-sectional data is limited. Cross sectional data can tell us that average years of schooling of school leavers are increasing or that the returns to education have changed in a cruel way. However, these data do not allow one to trace the impact of the changing returns to education on the decision to stay in school for longer or on a labour market participation decision. One needs panel data for this. Then, it is only panel data that enables the exploration of connections between these changes in individual behaviour, attendant changes in the behaviour of other household members and the resultant well-being of their households. In like fashion, census data and national cross-sectional sample surveys can spell out changes in access to grants or to social services such as water or electricity. Census data can even detail improvements in access per local area. However, it is only panel data that will reveal the processes through which these changes in income sources 10
and/or services are translated into behavioural changes in people and consequent changes in well-being. There is a famous South African example. National sample surveys show that households that contain a recipient of a state old age pension have a larger number of resident working age unemployed and non-participant individuals than similar households that do not contain an old age pension recipient. This correlation has been interpreted in two completely different ways. First, it has been said to evidence the fact that unemployed people stop looking for work if they can rely on a share of this pension income to survive. In contrast to this, it has been said to evidence the fact that unemployed individuals are driven to move in with their aging grandparents (and their pension income) as a survival strategy. These two suggestions have completely different policy implications. Looking at a cross-section one cannot decisively mediate between these two explanations. With panel data, one observe the dynamics. Panel data is also essential in answering many of the important demographic issues that are unfolding in the country. Do large households fall apart under the stresses of poverty? What is the impact of increased income on household composition, on school and labour market participation and on fertility? Are these impacts different for different sources of income such as wage income, remittances and social grants? Does it matter which household member is earning the income or is the income effectively pooled? Do you have to leave rural areas to escape poverty? In the South African context there is this symbiotic relationship between attrition issues and a serious study of household formation and reformation and migration dynamics. These demographic issues are also key aspects for policy purposes as although the changing migration and household size and composition trends are largely know from the census and other national sample surveys, very little is know about the micro-foundations and behavioural underpinnings of these demographic outcomes. So, these demographic issues are certainly worthy of investigation. The investigation and isolation of lifecycle effects is another demographic benefit that one derives from panel data. These poverty dynamics provide crucial information for the scope and targeting of anti-poverty policy. Some of those that are measured as poor in any cross section are temporarily or transitorily poor, having suffered a short-term shock with the capacity and the assets to recover after an adjustment period. This group requires policies that facilitate access to short term finance and insurance options in order to cope with their poverty. Others that are measured as poor in the same cross section have been poor for a long time. They are best seen as being trapped in poverty in the sense that they clearly do not have the assets to fashion a positive livelihood trajectory for themselves. This group requires welfare support, with the choice of support being selected based at least in part on what is shown to promote their accumulation of the needed human and social capital to break the poverty trap. The balance of antipoverty policy depends very much on the balance of the poor between the transitory and structurally poor. Income is determined by allocation of endowments and returns to activities to which allocations were made. Is are short-run and a long-run dimensions to these allocations and returns. Even in the short-run there is not a one-to-one mapping between income and income changes and consumption and consumption changes. The major reason for this is that households can change behaviour in the light of an income shock.
11
They can change their labour supply behaviour OR they can access other sources of income in form of gifts, remittances, or grants OR they can dissave or save assets. The ability of households to make these sorts of adjustments determines their ability to smooth their consumption. Generally, it is assumed that poor households faced with limited savings or insurance alternatives because of liquidity constraints have limited ability to smooth consumption in a volatile income environment. However, this is an empirical question depending on the family and social capital that can be accessed by households in any particular context. One has to try and get to the bottom of this context in South Africa as it relates directly to the issue of designing the most appropriate social safety net and anti-poverty policy. Policy makers need to consider increasing endowments, the returns to endowments and also vulnerability to short-term shocks that can lead to sub-optimal longer run decision making. Evidence around the world seems to suggest that assets or endowments change rather slowly in most settings and that these changes have a rather muted impact on money-metric welfare. However, changes in returns to given endowments (for example, changes in returns to education because of the demand for labour or changes in the returns to small-holder agriculture because of new seeds) can be a potent source positive income change. In terms of vulnerability, there is a need to understand whether most shocks are idiosyncratic or covariate. Idiosyncratic shocks (a member of your household is sick or a particular skill is not longer in demand in the regional labour market) impact on an individual household of a segment of the supply of labour but not necessarily y on all households in the community or the entire regional labour market. Common or covariate shocks (a drought or the closure of a mine that is the dominant employer in an area or the opening of a new road brings your village onto the tourist trail) are pervasive enough to change the available social and community capital and resources. If shocks are idiosyncratic then there is a good chance that social and community level support and insurance structures will be able assist the affected households with consumption smoothing. However, covariate shocks are likely to undermine these social support systems and require government intervention. Then, it is necessary to understand the role that savings and investment play in short-run and longer run mobility paths. All of this in turn suggests that the core information for the NIDS has to include income, consumption and assets. There is a need to triangulate the evidence on asset changes and returns to assets, income changes, and consumption changes in order to ensure that change is not being driven by measurement error. In any event, these relationships are at the heart of the empirics of whether households can move out of poverty or what it is exactly that enables houseshold or individuals to move up the distribution. So, they go to the heart of the existence of and explanations for transitory or chronic poverty. You generally need 3 waves of a panel to start to get a decent measure of permanent income that, in empirical work, lies at the heart of the distinction between transitory versus chronic poverty). Because a number of waves are necessary to tease out mobility pathways and to try and identify the presence or absence of hysteresis and the factors causing such hysteresis, there is an important role for carefully designed retrospective work, especially in the early waves. As one is essentially gathering life histories, and this is not the comparative advantage of quantitative survey people, this would seem to be an area in which there is definite potential for a partnership between quantitative and qualitative people.
12
We are ten years into the post-apartheid era and we do not know very much about these issues that are at the heart of our national project. They are central to understanding the extent of and routes to success in the new South Africa as well as the appropriate framework for dealing with poverty in South Africa.
13
The importance of longitudinal studies A longitudinal study is a research design in which the researcher collects data from the same population at more than one point in time. This does not always mean that the same subjects are used to collect data at more than one time, but that the subjects are selected from the same group or population for data more than once. This method is used when the researcher is concerned not just with the existing phenomena but also with changes that result over time. Thus longitudinal research is essential if the research purpose is to measure social change. Longitudinal research can potentially provide fuller information about individual behaviour. The analysis involves some comparison of data between periods. There are a number of different designs for the construction of longitudinal evidence: • • •
repeated cross-sectional studies; panel surveys or cohort panels; retrospective studies, such as oral histories and life and work histories.
Repeated cross-sectional studies In the social sciences, cross-sectional observations are the form of data most commonly used for assessing the determinants of behaviour (Coleman 1981; Davies 1994; Blossfeld and Rohwer 1995). However, a cross-sectional survey, because it is conducted at just one point in time, is not well suited to the study of social change. It is therefore common for cross-sectional data to be recorded in a succession of surveys over time, with a new sample on each occasion. These samples either contain entirely different sets of cases for each period, or the overlap is so small as to be considered negligible. Where cross-sectional data are repeated over time with a high level of consistency between questions, it is possible to incorporate a time trend into the analysis.
An example of a repeated cross-sectional social survey is the General Household Survey (GHS) collected by Stats SA. This survey collects a wide range of living standards information at a household and individual level. Almost 30 000 households are interviewed annually. The GHS has been running since 2002 and contains many questions that are consistent with the October Household Survey (OHS) which ran from 1994 to 1999.
The principal limitations of the repeated cross-sectional design are its inappropriateness for studying developmental patterns within cohorts and its inability to resolve issues of causal order. Both of these limitations result directly from the fact that in a repeated cross-sectional design, the same cases are neither measured repeatedly nor for multiple periods (M enard 1991). Thus, more data are required to characterise empirically the dynamic process that lies behind the cross-sectional snapshot (Davies 1994).
14
Panel surveys In panel surveys the same individuals or households are interviewed repeatedly across time. The fundamental feature they offer is that they make it possible to detect and establish the nature of individual or household change. An example of a panel survey is the British Household Panel Study (BHPS ) which is discussed in more detail in Appendix 1. The BHPS began in 1991 and has run annually since. The main objective of the BHPS is '...to further…understanding of social and economic change at the individual and household level in Britain into the next century'. The BHPS panel were selected in 1991 as a nationally representative sample across Great Britain, consisting of around 5,500 households. The first wave of interviews were conducted in 1991 (13,840 individuals) and the same individuals have been contacted again, as far as possible, for the subsequent waves of the survey. Individuals who had left their original household were followed and if they had joined or formed new households, the members of these households were added to the survey. New members coming into the original households, including children who reached the age of 16, were also interviewed. Cohort Panels can be considered as a specific form of panel study that takes the process of generation replacement explicitly into account. A cohort is defined as those people within a geographically or otherwise delineated population who experienced the same significant life event within a given period of time. Researchers select an age group, or some subset of an age group, and then administer a questionnaire to a sample or to the whole group. Thus, one or more generations are followed over their life course. The interest is usually in the study of long term change and in individual development processes: such studies typically re-interview every five years. If, in each particular generation the same people are investigated, a cohort study amounts to a series of panel studies; if, in each generation, at each period of observation, a new sample is drawn, a cohort study consists of a series of trend studies (Hagenaars 1990). Birth-to-Twenty (Bt20) is an example of a South African cohort study. For seven weeks between M arch and June 1990, 3273 children were born in the metropolitan area of Johannesburg-Soweto and enrolled into a long-term birth cohort study that will follow them and their families for the next 20 years. The overarching vision of Bt20 is to understand the holistic determination of child and adolescent health and development within Johannesburg-Soweto. This complex study continually impacts on current thinking about youth, and is committed to scientific research that makes a difference. The study documents and explores the socio-economic, socio-political, demographic and nutrition transition that is underway within South Africa and its impact on children and their families (Richter, et al, 2004) Retrospective studies All the data types discussed so far have been recorded with reference to fixed and predetermined time points. But, for many processes within the social sciences, continuous measurement of qualitative and quantitative variables seems to be the most suitable method of empirically assessing social change. When data are recorded in a continuous time, the number and sequence of events and the duration between
15
