Insee Analyses · June 2021 · n° 65

The French Labour Force Survey is overhauled in 2021: rationale behind the renovation and impact on the measurement of employment and unemployement

The Labour Force Survey is a continuous survey: the sample is spread over all the weeks of the year. Each weekly sample is associated with a so-called reference week, in relation to which the respondents describe their employment situation. To avoid recall bias and to ensure quality data, the collection period is very short: until 2020, respondents had to respond within two weeks and two days after the reference week. With the redesign, this period is extended to three weeks, which is still below the European ceiling.

Another particularity of the survey is that it is a housing panel. INSEE interviews the inhabitants of the sampled dwellings six quarters in a row, in order to estimate changes more robustly and to produce longitudinal analyses. Until 2020, INSEE interviewed households face-to-face in the first and last interviews and by telephone in-between. With the new survey, the first interview remains face-to-face, but for the reinterviews, the Internet is proposed as an additional response mode to the telephone. In the general case, the first three days of the collection period are reserved for the response by Internet. From Thursday of the first week of collection, the INSEE interviewer can call the household to conduct the survey by telephone. The collection website remains accessible to the household as long as it has not responded and at the latest until the end of the third week of collection. Note also that the sixth interview, which was face-to-face until then, becomes a reinterview like the others.

This multi-mode protocol is intended to encourage responses from households, which are increasingly using the Internet in their daily lives, and to lighten the load on interviewers. However, even with reinterviewing, the role of the interviewers remains crucial to obtain the response of households less familiar with the Internet or more sensitive to telephone follow-up.

During the first quarter of 2021, among the respondents whose households were reinterviewed, 41% responded by Internet, the others having responded by telephone. This result is close to those observed in the 2020 Pilot survey. The overall collection rate for the first quarter of 2021 is close to that of 2019, although slightly lower due to the health crisis conditions for the first face-to-face interview. In 2020 the collection rate for the Pilot survey and the old version of the survey were broadly comparable, both of which were affected by the difficulties associated with the health crisis.

The purpose of weighting a survey is to reconstitute the total population (not just the population that responds to the survey) by correcting for possible non-response bias (if the non-respondents have particular characteristics) and by controlling for sampling fluctuations (the sample may not be strictly representative of the reference population according to a particular criterion). To calculate the weight assigned to each respondent, auxiliary information from the sampling frame or from external data is used. As the sampling frame for the Labour Force Survey is different for metropolitan France and the French overseas departments, the weighting method differs accordingly.

Starting in 2021, in metropolitan France, INSEE weights the Labour Force Survey data according to a new method. This change of method aims first at taking advantage of the data of a new sampling frame, richer in characteristics of dwellings and their occupants. Since the fourth quarter of 2020, the Labour Force Survey sample for metropolitan France has been drawn from a new sampling frame (Fideli), following the same principles as before. This new sampling frame is very similar to the one used previously, but has new information that allows for a better understanding of nonresponse [Courrier des statistiques, July 2021]. The change in method is also intended to apply the requirements of the new European regulation, particularly on population margins, and more generally to improve on certain aspects of the old weighting method.

New information in the sampling frame to better model survey nonresponse

For metropolitan France, INSEE changed the weighting method from a one-step method (single calibration on variables in the sampling frame and on external data) to a two-step method, as was already done in the DOM: first a correction for sampling bias and nonresponse, then calibration on external data.

In addition, the calculation of weights uses new information from the sampling frame (place of birth, household composition, income, etc.). These characteristics are linked to both the response behavior and the variables of interest (being employed). The dwellings occupied by persons receiving high pensions or high incomes from work (without being in the last decile), or couples with children have, all other things being equal, a higher probability of responding to the survey. This new information complements the information already used (geographical location, occupation status, type and size of dwelling, etc.) and makes it possible to better characterize respondents and non-respondents in order to better correct for nonresponse bias.

In the DOM, the weighting method has been marginally adjusted (notably calibration by interview rank), but without being renovated in its general principle, as the sampling frame (Census of Population) has not been modified.

The rules of “usual residence” have evolved to better take into account the situation of young people

The implementation of the "usual residence concept" of individuals, which aim to manage cases of multi-residence in order to respect the scope of household surveys and to deal with double counting, have also been revised (for metropolitan France and the DOM). These rules define the population of interest, which must be weighted.

At the beginning of the questionnaire, all persons living in the dwelling, even occasionally, are identified. However, not all of them fill in an individual questionnaire and therefore not all of them participate in the calculation of the indicators. Either because they do not fall within the scope of the survey (e.g., if they reside in a community: retirement home, hostel, etc.), or to avoid double counting (e.g., a child living alternately with separated parents).

When INSEE redesigned the survey, the questionnaire was changed to better identify dual residence situations, and some usual residence rules were revised, mainly the one concerning adult students living both in housing for their studies and with their parents. In the old survey, these students were essentially attached to their parents' accommodation; this improved their response rate (parents were quicker to respond to the survey), but generated a high proportion of proxy responses (as students would frequently be absent when the interviewer visited their parents, their parents responded in their place). However, the response by a third party is of lesser quality, especially for the questions aimed at determining the activity status, which are very precise. For example, at a given age and level of education, the employment rate of dual-residence students is higher when they are surveyed in their student housing than in their parents' housing. Starting in 2021, a "mixed" solution is implemented: students in dual residence are surveyed in their two dwellings, and then their weight is divided by two.

Experience with previous redesigns of the Labour Force Survey has shown that even simple changes in the order of questions can cause breaks in the measurement of key labour market indicators. With the 2021 overhaul, which involves the questionnaire, the protocol, and the weighting method, such breaks were also likely. In order to provide sufficiently precise information on measurement breaks, INSEE took the necessary steps to carry out an unprecedented preparatory operation: a complete "pilot survey". It was designed to align perfectly with the new survey in terms of questionnaire, protocol and processing method, and would be in the field at the same time as the traditional Labour Force Survey, one year before the required changeover, i.e. as early as the first quarter of 2020. In order to reconcile the need for precision on the one hand (to be able to detect significant impacts), and the need for reasonable cost on the other hand, a quarter of the sample of the survey in production was taken to carry out the Pilot survey in 2020.

In the end, the Pilot survey has two components:

• an early switch to the new survey: a quarter of the sample, across all interviewing ranks, switched to the new survey in the first quarter of 2020 - this switch foreshadowed the switch in the first quarter of 2021 for the rest of the sample - ;

• an oversample in the old survey: introduced progressively from the fourth quarter of 2019, an oversample of households was reinterviewed in the old survey up to and including the first quarter of 2021. A complete control sample in the old survey was thus available in the first quarter of 2021, when the new survey went into production.

In the first quarter of 2021, the three quarters that had remained in the old survey switched to the new one and joined the part that had switched one year earlier. The survey sample thus returned to its “full size”.

The main objective of the Pilot survey was to provide a set of information that would allow INSEE to estimate as accurately as possible the measurement breaks induced by the new survey. With the Pilot survey, by comparing the indicators measured via the old survey and via the new survey, five quarterly measures of breaks in series were available, from the first quarter of 2020 to the first quarter of 2021. Having several quarters available makes it possible to measure the break with greater precision, but also to detect possible seasonality in the impacts.

INSEE measures the overall measurement break directly by comparing the two versions of the survey. Separate analyses of some components are also possible:

• the impact of the new weighting method can be studied in isolation, by applying the old method to the new survey data (Box 2);

• the effect of the usual residence rules can be distinguished by simulating the effect of the new rule in the new survey and comparing it to the old one; the "proxy" effect, which is closely related to this change of rule, can be measured by comparing the results of the new survey with those of the new survey in which the weights of the respondents are recalculated so that the proxy rate is set to that of the old survey;

• the impact of conceptual changes can be isolated by counting the number of people for whom the change in criteria changes their employment status (people on sick leave or people on parental leave);

• Among the other changes, it is possible to distinguish effects related to the change of questionnaire, by comparing the new and old version in the first interview, for which the protocol changes little. However, the estimate is based on smaller samples, and is therefore more fragile. Similarly, specific estimates are also possible to isolate mode effects linked to responding by internet, by separating a "measurement" effect (the fact that the same person responds differently on the internet or to the interviewer) and a "selection" effect (responding rather than not responding because the internet mode is now available). However, the estimation of these mode effects is also more fragile because it is based on assumptions that allow simulating what the response would have been in a counterfactual scenario without change. The results are sensitive to the choice of the determining variables. For this reason, these effects are not distinguished here but will be the subject of further expert work.

A "backcasting" exercise for macroeconomic indicators (from the national accounts, key labor market indicators, etc.) is a statistical operation which consists in adjusting past observations to match the "new measure" of an economic phenomenon. For the overhaul of the Labour Force Survey, this means adjusting past observations to make it "as if" the questionnaire had always been administered in the same way as in the first quarter of 2021, with the same concepts, the new weighting method and the same protocol. The objective is to create a series whose changes reflect only changes in the economic phenomenon of interest and not changes in the measurement of this phenomenon.

It should be noted that modifying a quantity in 1975 because of the possibility of responding now via the Internet may seem paradoxical, since in 1975 the Internet did not exist. However, the appearance of the Internet and its expansion should not have an impact on the backcast series. Although the hypothesis may seem very theoretical, it is the only way to guarantee the comparability of results over a long period.

A "pivot" coefficient to correct for most breaks

In practice, in the general case for a given indicator, INSEE estimates an average "pivot" coefficient over the five quarters of overlap between the old and new surveys, allowing for the transition from the old to the new measure. This coefficient measures the magnitude of the break over the overlap period between the two surveys, in a multiplicative form. It is then applied to the entire past series in order to simulate a series in the new measure. This is equivalent to assuming that the magnitude of the break is proportional over each year. For example, for women aged 35-39, employment is higher in 2020 by 2.6 percent with the new Labour Force Survey, so the coefficient 102.6 is applied each year. The same exercise is carried out for each of the activity situations, then the levels are proportionally recalibrated each year on the total population of women aged 35-39 (Chart, new backcast measure - pivot coefficient only-).

The break between the old and new surveys often differs according to the characteristics of the populations. This is why it is important to measure and understand the determinants of the breaks (primarily gender and age) and then backcast at the level of employment subsets by gender and age, rather than at the level of the entire given aggregate. This necessity leads to backcasting by gender and five-year age group. However, at this level of detail, the measurement break is based on small sample sizes and is therefore more fragile. Therefore, it is not the coefficient specific to each age group that has been applied, but rather a coefficient smoothed over three consecutive age groups. By re-weighting each year by the weight of each age and gender group, the backcasting reflects the changes in the composition of the population on the aggregate indicators.

Absences due to parental leave: a conceptual break that requires an ad hoc backcast

However, the assumption underlying the general method is not always satisfactory. In particular, in the new Labour Force Survey, people who report being employed but absent from work because of parental leave can be counted as employed regardless of the duration of the leave, provided they receive an allowance such as the French “shared child-rearing benefit” (Prepare). This conceptual change amounts to increasing the employment rate of women aged 25 to 49 by 0.8 points in 2020. However, it is not relevant to apply this increase in employment since 1975, because at the beginning of the period there were no financial incentives for parental leave. Their appearance in 1984 and their rise to prominence at the end of the 1980s led to an ad hoc retropolation, in order to have series in new measure, which better transcribe the changes in parental leave since 1975. For this purpose, external sources (other surveys, administrative sources) were mobilized. In the above example of women aged 35 to 39, this amounts to proposing a new backward-looking measure "adjusted" for the effect of parental leave. At the beginning of the period when the measures did not exist, the final series then lands between the old series and the backcast series without adjustment.

Quarterly series backcast in a second step, after checking that seasonality has not been modified by the new survey

By compiling the individual observations from the five quarters of coverage, the estimate of the break is more robust than from a single quarter. This is why the annual series are backcast first, using this "global" break. New quarterly series are then constructed in a consistent manner, i.e., in such a way that the average of the four quarters coincides perfectly with the annual series.

However, a change in questionnaire, collection mode or concept can also affect the seasonality of a quarterly series. A very different break in a given quarter from the collection year may indicate that the seasonality has changed. However, the thorough analysis of the quarterly series led to the finding that there was no discernable change in seasonality for any series. This conclusion will have to be re-examined with the addition of new quarters. One consequence of this finding is that the pivot coefficient can be identical each quarter, and thus, for the year 2020, the quarterly estimates backcast are not identical to those directly derived from the Pilot survey. Another consequence is that the past seasonality of the series has been preserved, so that the usual seasonal adjustment (SA) models have also been retained.