The Label Committee A Governing Body Ensuring the Quality of Official Statistics

Since its creation within the National Council for Statistical Information (CNIS) in 1994, the Label Committee has played a crucial role in defining the program of official statistical surveys in France. Far from being a simple rubber stamp for survey projects, the Committee certifies their compliance with state of the art standards for implementing surveys and also suggests ways in which they can be improved. Given its transversal coverage, the Committee also plays a pedagogical and advisory role, thus contributing to the dissemination of good practices and to the documentation of surveys and statistics, as well as issuing warnings when necessary.

During the 2010s, the scope of the Label Committee has been broadened: labelling now also applies to statistics resulting from the use of administrative data, when they are produced by public or private bodies entrusted with public service missions. The Committee performs this function on behalf of the Official Statistics Authority (ASP), which issues official statements on quality, following the principles of the European Statistics Code of Practice (CoP). The extension of the role of the Committee led to change its initial name, being now the Official Statistics Quality Label Committee to mark its enlarged missions.

Since its creation, the Label Committee has developed a global approach contributing to the quality of statistical operations. It has therefore supported and contributed to highlight concerns regarding statistical quality in parallel with the dynamic initiated at the European level and has gradually set up a framework to assess them.

The Committee was created in 1994 at the request of the National Council for Statistical Information (CNIS), under the name of the Statistical Surveys Quality Label Committee for Public Services. Positioned within the CNIS, its purpose was “to examine the compliance of statistical surveys projects to quality standards, after these projects have been deemed relevant by the Council, and to deliver a general interest label; to propose to the competent ministers the visa for including the survey in the official surveys program referred to in the Statistical Act of 1951 (article 2)” (note).

The aim was to improve the procedures for preparing official statistical surveys (Allain, 1995). The analysis that led to this initiative showed that the CNIS examination needed to be consolidated, as it was no longer adequately meeting certain requirements that were becoming increasingly important. In the case of companies, particularly the smallest among them, there was a need to make them accept the burden of answering statistical surveys, to distinguish statistical surveys from administrative formalities and to demonstrate their purpose and usefulness. For households, it was a case of safeguarding against reactions from respondents or complaints to the CNIL in connection with the volume and nature of potentially intrusive questions being asked in the survey.

The Label Committee was thus created within the CNIS for an initial experimental period of two years, before being made permanent. The experimentation phase led to define the initial rules for this examination of compliance using a heuristic approach, considering: compliance with existing nomenclatures, consistency of statistical units, taking account of the need to limit the burden on respondents in the sampling frame, justification of the intrusive nature of certain questions in relation to the objectives being pursued, dissemination of micro-data respecting confidentiality issues.

The French Economic Modernisation Act of 4 August 2008, which led to the creation of the Official Statistics Authority (ASP), applied a broad definition of official statistics, enshrined in the Founding Act No 51-711 of 7 June 1951 on Legal Obligation, Coordination and Confidentiality in statistical matters. The definition of official statistics (Article 2 of the Act) includes “all output resulting from:

The design, production and dissemination of official statistics shall be carried out in complete professional independence”.

The scope of the statistics disseminated by public operators” has thus been added to the scope of the surveys included in the annual programme (thus having the ministerial visa). The ASP is responsible for labelling statistics disseminated by public operators produced on the basis of administrative data. The authority relies on an expertise of the case led by the Label Committee, named since 2013 as the Official Statistics Quality Label Committee.

The Committee is thus in charge of three kinds of missions (Ouvrir dans un nouvel ongletChristine, 2016) (Box 1), since the two following missions have been added to its traditional mission of examining the compliance of official statistics surveys:

With this extension of its competences, the Label Committee became a separate entity in 2013. Its resources are provided by INSEE within the Methodology, Statistical Coordination and International Relations Directorate (DMCSI), consisting of a rapporteur and a general secretariat, as well as experts to examine the cases. The surveys and administrative statistics are examined by thematic committees, the composition of which is determined by a decree, under the responsibility of the Chair of the Committee, appointed by the Minister of the Economy, after having consulted the Presidents of the ASP and the CNIS.

The official statistics survey status implies duties for the statistical services (they have to fulfil the procedures implemented by the CNIS and the Label Committee, which requires the submission of standardised documents), but it also confers rights for the services and provides guarantees for respondents.

The services implementing surveys are therefore able to highlight the “official statistics” brand and the general interest criterion, to ensure that the communication sent allows respondents to distinguish them from the multitude of surveys or polls carried out for other purposes. The statistical survey can also be granted with a mandatory status, which acts as a lever to encourage the recipients to respond, even though, in practice, services first try to persuade people to respond by pointing to the recognition of the public interest of the survey by the CNIS (Ouvrir dans un nouvel ongletLe Gléau et alii, 2016).

Finally, the official statistics survey status also entails the responsibility of the producer to take all necessary measures to ensure statistical confidentiality during the dissemination of data and results. It gives persons or entities the assurance that this information will only be used for statistical purposes. This guarantee concerns the protection of privacy, commercial or business confidentiality. In the areas of social protection or education, for example, certain surveys conducted among administrative services do not require an official statistics status in theory, as their collection relies on persons belonging to the administration. However, granting them this official statistics status allows them to differentiate themselves from purely administrative collection, submitted to obligations stated in the French Code on the Relationship between the Public and the Administration (CRPA), which do not guarantee the statistical confidentiality.

The scope of official statistics surveys (within the meaning of the Statistical Act of 1951) is defined in a quasi-tautological manner: any survey published in the annual decree defining the survey programme is qualified as an “official statistics survey” thus being granted a visa by the Ministry for the Economy.

There are three criteria that must a priori be met in order to be included in this scope:

These criteria are quite broad. Ultimately, it is the procedure itself that delineates a posteriori the scope of official statistics surveys. In practice, the visa from the Ministry, together with a mention of the mandatory response, when applicable, is granted, by way of delegation, by the Director-General of INSEE.

The procedure for surveys to obtain approval is as follows (Figure 1): they must first obtain an opportunity or relevancy statement from the National Council for Statistical Information, which confirms that the survey is useful, that it responds to a general interest need and that it does not duplicate existing sources on the same subject. For regional surveys, the Regional Committee for Economic and Social Information (CRIES) or a regional body, duly constituted and convened by the Regional Director of INSEE, first decides if the survey is needed, before submission to the CNIS.

Figure 1. From Draft Survey... to Ministerial Approval

However, two cases of urgency are provided for, either within the framework of a special law or as a result of necessity and urgency. The surveys are then deemed to be relevant by nature, and therefore exempt from thorough examination by the National Council for Statistical Information when the deadlines are very tight. This method was used in 2020, particularly for surveys aiming to measure the economic or social effects of the health crisis associated with COVID-19 (surveys such as ACEMO-COVID by DARES and EpiCov by DREES and INSERM).

The second stage consists in obtaining a compliance statement from the Label Committee on the basis of a comprehensive instruction analysing the planned process for implementing the survey, from its conception until the dissemination of the results.

The survey then benefits from a “general interest and statistical quality label” issued by the CNIS (in practice the Label Committee by delegation), accompanied where necessary by a proposal to make the survey mandatory for respondents.

The Label Committee has gradually developed its own methods to assess the compliance of surveys to statistical quality. Over the last 25 years, it has built a body of rules that sets out the criteria expected to qualify the surveys, in particular:

In order to assess these criteria, the Committee has established a standard instruction file, which allows the requesting services to document the entire survey process, from the design phase to the implementing phase and data processing following the collection. The Committee is especially careful about the upstream consultation phase (the service must indicate and objectify how it consulted the stakeholders and researchers) and the questionnaire testing phase (to assess their length and the cognitive understanding of the questions by the respondents). The Committee also oversees the methodological aspects and in particular the sizing of the sample in order to achieve the expected precision, as well as the adjustments planned, to ensure that there is no response bias and to correct for the negative effects of non-response. In the event of any changes to a regular survey, it examines in particular the strategy put in place to check for any breaks in time series and the way they will be assessed and explained to the users. Finally, it ensures that the means provided for are proportionate to the objectives. The openness of micro-data for researchers (in compliance with confidentiality) is a crucial final criterion, since official statistics surveys are “public goods” serving general interest.

Without specifically referring to them at the outset, these criteria are similar to those defined in the European official statistics framework built since the nineties. In 2001, six quality criteria were set out by Eurostat and officially adopted by the European Statistical Programme Committee: relevance, accuracy, timeliness and punctuality, accessibility and clarity, comparability, and coherence (Desrosières, 2003). They have since been consolidated with the establishment of the European Statistics Code of Practice (CoP) and its 16 principles (Ouvrir dans un nouvel ongletEuropean Union, 2017).

Indeed, the labelling initiatives launched in France within the framework of the CNIS and the Label Committee are in line with this approach. They were also identified by European auditors during the peer review carried out in 2014 as a piece of evidence of the quality of the surveys.

The consultation initiated by the CNIS therefore represents a means of ensuring the relevance of the surveys to meet the needs expressed by users. The compliance review performed by the Label Committee ensures that the design of the surveys follows the state of the art rules in terms of both methodology and deontology. Finally, the instruction files submitted to the Label Committee often form a key element of the documentation of official statistics surveys (metadata), which ensures their transparency for users (Bonnans, 2019).

There may be cases where the Label Committee judges that the survey is indeed of general interest, but that it cannot be fully labelled ex ante as it does not immediately fulfil all the criteria set out above. However, the Label Committee can in this case issue a favourable examination statement and request the granting of the ministerial visa (Figure 1).

These special cases involve the following three situations in particular:

Some surveys conducted for the purposes of scientific or historical research or action research (for example on the knowledge of daily mobility in certain territories) can, under certain conditions, obtain a random sample from the population census or from the statistical repository on housing and people (FIDÉLI).

These surveys are often conducted within the framework of European research infrastructures that have their own governing bodies (as is the case for the European Social Survey, for example, which is conducted every two years in the majority of European countries). The difficulty for the Label Committee in examining these projects resides in the fact that its recommendations often come up against other forms of governance and supra-national specifications and are therefore rarely followed up, as they are defined upstream by an international consortium.

The Label Committee is asked by the INSEE Executive Committee to give its approval for the release of a sample, without this being included in the official statistics survey programme. “The Executive Committee has validated the fact that, for the sole purpose of obtaining a random sample, the Label Committee may issue, having verified the quality of the sampling plan, the downstream methodology and the bodies governing the project, a simple examination statement rather than a compliance statement. For this reason, the operations in question will not be qualified as official statistics surveys within the meaning of the Law of 1951”.

Ultimately, although the spirit of the examination by the Label Committee has been to ensure the compliance of methods used with the state of the art, it has developed adapted procedures fitted to different uses and it delivers different types of statements (compliance, favourable examination with a ministerial visa, simple examination without a visa).

Among other tasks, the ASP is responsible for labelling statistics produced on the basis of administrative sources and disseminated by bodies fulfilling public service roles, i.e. outside of the Official Statistical Service (OSS).

The procedure put in place for the purposes of establishing these labelling statements was defined by the ASP following a mission entrusted to INSEE’s Inspectorate-General (Ouvrir dans un nouvel ongletChappert and Puig, 2011), supplemented for its operational implementation by a guidance note from the Committee’s Rapporteur. For these statistics that fall outside of the “peri-OSS” surveys frame, the idea was to draw upon the European Statistics Code of Practice by using the principles of this Code that were relevant to the examination of these data (Box 2). It was also decided that the Label Committee will be entrusted with examining the labelling files, as is the case for official statistics surveys, this procedure being enshrined in the regulatory texts in 2013.

More specifically, the ASP decides upon the annual programme of administrative statistics that are to be examined. The Label Committee examines the labelling requests decided by the ASP, using a standard documentation file, as is the case with surveys. It meets as a specialised committee and issues a statement that is accompanied by recommendations with a view to ensuring that the statistics comply with the Code. The labelling decision is formally taken by the ASP and is published on its website, together with any associated conditions or recommendations and is published in the Official Journal.

With the survey labelling process having been initiated a little over 25 years ago, the Label Committee has developed an examination methodology that enables it to certify the quality of official statistics surveys and, more recently, statistics produced outside of the OSS by bodies fulfilling public service missions. These approaches have acted as important levers to enable statisticians to carry out their work in accordance with rules that adhere to best practices and ensure transparency. Ultimately, it is the very existence of this governance framework that constitutes a safeguard, preventing as many potential failings as possible.

Within the OSS, the labelling procedures continue to favour traditional surveys, while other data collection methods are not subject to the same level of formality. For a number of reasons (including reduced production costs), administrative sources or mass data are increasingly being used for the production of statistics by the OSS. A question could therefore be raised within the OSS regarding the expansion of the more formal qualification procedures, following the example of the labelling by the ASP of official statistics produced within the “peri-OSS” sphere.

Furthermore, with the development of digital technology, the production of statistics has largely developed outside of the OSS: there are numerous stakeholders who develop databases or datasets and disseminate statistics as a by-product of their main activity. When looking at the “peri-OSS”, few of the statistics produced are labelled by the ASP. As a result, many statistics continue to be picked up by the media without any guarantee of quality or even transparency. A new qualification service was launched in the early 2010s, involving a “calibration” approach by the CNIS for general interest statistics produced by private organisations. This arrangement has proven to be very costly, for both the producing organisations and the Committee, and has not seen the level of development anticipated.

A new qualification method was recently suggested for statistics produced outside of the OSS: the aim would no longer be to attest to the overall quality of the statistics produced, but more simply to induce the minimum transparency of their documentation (metadata). This approach, which was recently approved by the CNIS, is currently being tested. The principle would be to establish a standard documentation template, that can be used by organisations on a voluntary basis, with the result that the statistics in question could then be approved by the CNIS. The aim is to induce a virtuous circle: first, make users question the quality of the data and statistics disseminated before using them, and second, encourage organisations to document their statistical products in order to meet these expectations.

Ouvrir dans un nouvel ongletArrêté du 10 janvier 1994 portant création au sein du Conseil national de l’information statistique d’un comité du label des enquêtes statistiques des services publics et des autres services producteurs d’informations statistiques. In: Légifrance website. [online]. Modified 24 December 1997. [Accessed 24 November 2020].

Ouvrir dans un nouvel ongletArrêté du 2 mai 2013 relatif aux modalités d’organisation du comité du label de la statistique publique. In: Légifrance website. [online]. [Accessed 24 November 2020].

Ouvrir dans un nouvel ongletDécret n° 2009-318 du 20 mars 2009 relatif au Conseil national de l’information statistique, au comité du secret statistique et au comité du label de la statistique publique. In: Légifrance website. [online]. Modified 1st February 2019. [Accessed 24 November 2020].

Ouvrir dans un nouvel ongletLoi n° 2008-776 du 4 août 2008 de modernisation de l’économie. In: Légifrance website. [online]. Modified 5 July 2019. [Accessed 24 November 2020].

Ouvrir dans un nouvel ongletLoi n° 51-711 du 7 juin 1951 sur l’obligation, la coordination et le secret en matière de statistiques. In: Légifrance website. [online]. Modified 25 March 2019. [Accessed 24 November 2020].