In 2015, INSEE embarked on the construction of a new industrialisation chain for surveys: a new online collection platform (See the article on Coltrane by Olivier Haag and Anne Husseini-Skalitz in this issue), equipped with a questionnaire generator (See the article on Eno by Heïdi Koumarianos and Éric Sigaud in this issue), which itself is based on a formal description in the DDI international standard (Ouvrir dans un nouvel ongletData Documentation Initiative), a widely recognized metadata description language.

New surveys have gradually been integrated into this system. However, the writing of questionnaires in DDI in advance has quickly turned into a bottleneck, as the advantages of this metadata description language are paid for by a high level of verbosity, which increasingly overwhelmed the institute’s scarce expert resources: the back-and-forths between survey designers and programmers had been shifted to the two or three DDI experts, but not eliminated.

To make designers autonomous and end-to-end masters of the collection media creation process, they had to be allowed to write DDI without knowing it. In addition, they needed to be able to quickly observe the results of their amendments, without having to resort to an expert. The idea was born: A questionnaire design tool (Ouvrir dans un nouvel ongletIverson and Smith, 2013), within a “designer’s workshop”, making it possible to create the questionnaire templates and view them in one click, while directly feeding into the RMéS statistical metadata repository (Bonnans, 2019).

The idea of creating the questionnaire design tool, named “Pogues”, initially sought to reduce the workload of modelling questionnaires in DDI. It was soon to be enhanced by other issues then present in the area of methodology and IT: the need to demonstrate by example this notion of active metadata (Ouvrir dans un nouvel ongletIverson, 2010), which remained vague and theoretical for many, the desire to test new technical tools for the creation of web applications, or the desire to experiment with collaborative, agile and lightweight development methods. This combination of interests made it possible to quickly mobilise the resources and the skills needed to organise the launch of the operation. It goes without saying that the procedures for planning, allocation and management of resources were a little disturbed in the process.

The initial development of Pogues was mainly carried out in April 2015, during a “Ouvrir dans un nouvel onglethackathon” lasting one week, at the Ouvrir dans un nouvel ongletEuraTechnologies centre of expertise and innovation in Lille. The hackathon brought together eight computer scientists and two professionals (metadata and collection). It was preceded by a few specification sessions using agile methods and followed by another shorter hackathon with a smaller group.

A guardian angel was apparently watching over Pogues, as the first hackathon proved to be perfect in every respect: opportunity in terms of timing, the blend of skills, how it progressed, the location and even the weather... The results exceeded expectations in respect of most of the objectives:

In terms of innovation, Pogues followed in the footsteps of Eno with three innovations for INSEE:

The application continued to be enhanced in the months following the hackathon, first on the basis of voluntary contributions, then through the provision of light services on various aspects: additional developments, documentation and ergonomics. In parallel, the “organisational” positioning of Pogues was taking place: it was necessary to specify to whom this collective object that suddenly sprung up really belonged. The option chosen was to consider Pogues as essentially a metadata management application (Ouvrir dans un nouvel ongletBakkomen, Orten and Prestage, 2014; Ouvrir dans un nouvel ongletButt, Norland and Orten, 2018), in this case of descriptions of questionnaires, and that it was therefore a “satellite” of the RMéS metadata repository. This position was to prove very useful in giving a less theoretical meaning to the statistical metadata repository: Pogues, a tool intended for designers, de facto became the means of demonstrating the contribution of the RMéS repository to their profession.

The Quality Unit, within the INSEE Methodology, Statistical Coordination and International Relations Directorate (DMCSI), owner of the RMéS repository, has therefore taken charge of the future of the Pogues tool. In particular, in October 2016, it organised a working group in which survey designers were able to test Pogues in depth and express their wishes for its development, based on specific uses. This meeting made it possible to formalise a roadmap for the future development of Pogues (and Eno). It provided these tools with good visibility among managers of business and household surveys.

The Quality Unit also handled contacts with the DDI Alliance, the organisation responsible for the development of the Ouvrir dans un nouvel ongletDDI standard. The requirements of Pogues and Eno, which emerged as increasingly complex questionnaires were taken into account, needed as a matter of fact additional details or additions to DDI. It would have been easy to integrate these modifications into a version of the standard specific to INSEE. However, in accordance with the principles of the RMéS, and even though it would require more time and energy, the decision was made to contribute to the enhancement of the DDI standard itself, so that the entire community could benefit from the new functions.

A service provision started up in spring 2017, to redesign and develop the functional coverage of Pogues: integration of questions in the form of tables, increase in the number of filter question cases, etc. Internally, major architecture consolidation work led to the “official” release of an initial version of the application in April 2018. Lastly, documentation and communication resources (user and developer guides, demonstration video, logo creation, presentations at various national and international meetings, etc.) have made it possible to give the Pogues tool a good level of visibility both within INSEE and beyond.

Pogues enables the designer to structure his/her questionnaire in sequences (modules and sub-modules) containing questions and text elements, as well as to define the respondent’s questionnaire path logic and the overall constraints for completing the questionnaire (box 1).

Different kinds of questions are supported: simple single entry field (text, numeric, boolean or date), single or multiple choice, table or values can be restricted using predefined boundaries or code lists (For more details on the features of the Pogues tool, see the Ouvrir dans un nouvel ongletuser guide available on the GitHub platform).

Corresponding to each question, there are one or more so-called “collected” variables (meaning that their value will be provided by the respondent), but calculated or external variables can also be defined. The former are obtained by generally simple formulas based on the variables collected and are used, in particular, to specify consistency checks. External variables are not collected in the questionnaire but are useful for its customisation: they may be values obtained previously and recalled to facilitate completion, check evolutions or set certain elements of the questionnaire (collection wave, geographical zoning, last known population, etc.).

In addition to sequences and questions, Pogues allows users to specify different text elements: comments, instructions or help for completion, warning messages, etc. Depending on the type of collection medium that will be produced, these elements will appear in various forms, such as tooltips on web forms. Lastly, it is possible to describe some logical components for interactive modes (self administered web questionnaire or face-to-face survey):

These logical expressions are written in simple ad hoc language. Research is ongoing to study the use of a more standard language, such as VTL (Ouvrir dans un nouvel ongletValidation and Transformation Language, published by the SDMX initiative).

Pogues proposes a practical and intuitive graphical interface to describe the questionnaire and its components (Box 1). The questions and sequences can be moved using the mouse or copied, and a simplified representation of the questionnaire allows users to move around quickly. At any time, users may view their questionnaire in the form of their choosing (web or pdf), export it in DDI or produce the corresponding detailed specification document.

The questioning is not identical across each mode of collection, whether in terms of substance or form. For example, in terms of substance, an instruction may differ depending on whether it is addressed directly to a respondent or to a surveyor (Box 2). In terms of form, the dynamic functions offered by the web (online checks, dynamic filters, drop-down menus, etc.) cannot be copied identically when using the paper format. These two aspects must therefore be taken into account in the establishment of the new pathway, in which the designer must be able to be autonomous in terms of the design and construction of the questionnaires for a multi-mode survey.

The Pogues tool must give designers the option of differentiating between elements from a single mode while, to the extent possible, sharing what can be shared (questions common to all modes are described only once).

In terms of form, it is a matter of applying deterministic principles, namely ensuring that the collection method and format of the questionnaire are sufficient to determine the presentation of a component of the questionnaire. Thus, some differences are interpreted directly by the questionnaire generator: disappearance of dynamic behaviours, filters replaced by a simple text explaining the redirection, or presenting a drop-down list from the web in a simple input field.

A questionnaire and its components (instructions, question formulations, response formats, etc.) represent a set of information that characterises the statistical data collected. Thus, the questionnaire is a set of statistical metadata intended among others to be included in the devoted repository. Pogues is both a questionnaire design tool and a management tool for the metadata describing the questionnaires.

Pogues only makes sense as a metadata editor if the questionnaire components (sequences, questions and code list) can be easily reused. It is important to avoid returning to the situation that prevailed before the RMéS: the metadata was rewritten from one operation to the next, from one year to the next, or reused but adapted to the context of each survey. To that end, communication between Pogues and the statistical metadata repository must allow the search and retrieval of unit components (Ouvrir dans un nouvel ongletGreenough, Mechanda and Rizzolo, 2014), as well as their filing in the repository when the design of a stable version is completed.

To date, the development of the interface between Pogues and the RMéS has encountered technical difficulties and choices favouring other priorities; therefore, it remains functionally limited. It is now becoming urgent to complete the connection of Pogues to the RMéS repository, which should be carried out in the coming months.

The arrival of an initial version of Pogues in production in April 2018 has changed the process for specifying, conducting and validating questionnaires (from business surveys so far).

From now on, it is necessary to organise a meeting between the survey designer and those responsible for the new “collection ecosystem” (Figure 1): the administrator of the Eno tool, who is also the questioning methodologist, the administrator of the Pogues tool and the administrator of the Coltrane collection platform.

Figure 1. Under the Bonnet of Pogues

During this preparatory meeting, four important points are discussed:

This meeting makes it possible to formalise the improvements in terms of questionnaire design, the commitments in terms of functions offered by the collection ecosystem and also the functional limits.

With Pogues, it is about putting the designer back at the centre of the process. Indeed, the designer now enters his/her questionnaire into Pogues directly and, in a single click, views and validates the web rendering, the paper rendering or even the “specification” rendering (in office automation format). The time required for updating a questionnaire is greatly reduced due to the elimination of the back-and-forths between the designer and the questionnaire developer or between the designer and the DDI expert (when the Eno questionnaire generator existed, but Pogues did not yet exist). It should be noted that, at this stage, validation is carried out without customisation of the questionnaire. This stage will be carried out within the collection platform.

At the time of publication of this article, the functional scope of the various tools is still incomplete. This means some manual adjustments will still be required. However, the more Pogues and Eno increase their functional scope, the more autonomous the designer will become and the shorter the time required to update a questionnaire will become.

Nonetheless, the issue of third-party intervention for the validation of complex questionnaires remains: household survey questionnaires, which have not yet switched to the new ecosystem, are currently being updated with the help of an intermediary between the designer and the computer scientist; the question of what role that intermediary could play in the future is not yet decided.

As indicated, Pogues enables the survey manager to design his/her questionnaire, including the checks that will be performed during filling. This corresponds to the “Design variable descriptions” and “Design collection” sub-processes in the representation of the Generic Statistical Business Process Model (GSBPM: the Ouvrir dans un nouvel ongletGeneric Statistical Business Process Model is the reference model in the global official statistics community for the representation and high-level analysis of the statistical processes).

The formal questionnaire structure that is thus specified, with the associated questions and variables, can be reused further downstream in the statistical production process and, in particular, during the checking stage for the collected data. This was noted during the preliminary studies for the project to develop a generic survey processing station. It was therefore agreed that the structure of the questionnaires would be sent by the RMéS during the development stage for the checks, to be reused and thus automatically generate the screens for data editing.

It is possible to go further and envisage post-collection data editing controls being specified in Pogues, where all variable definitions are already available for this purpose. The scope of the Pogues tool would then be extended to the “Design processing and analysis” sub-process. And why stop there? Pogues could also make it possible to specify the collection process (“Design production systems and workflow”): modes, collection schedule, etc., all metadata that can activate future automated services (triggering of  reminders, etc.). Pogues would then become a sort of survey designer’s workbench, which is used not only to design the survey, but more broadly to design the survey process.

Pogues is now included at INSEE in the service offer of the RMéS repository. Due to the scope of the latter, Pogues will eventually be opened up to Ministerial Statistical Departments (MSDs). At the same time, the objective of the Coltrane collection platform is to collect as many Official Statistical System (OSS) surveys as possible so that businesses that respond to multiple surveys have only one infrastructure to use. In late 2019, Coltrane opened up its applications (such as contact management) to OSS producers who have adopted its services. Therefore, it is natural that this offering should be supplemented by opening up the Pogues and Eno tools to the MSDs that need them. The same will apply to the new online collection platform for household surveys.

In a few years, the small original Eno graphic, resulting from the Coltrane project, has, together with Pogues, launched a comprehensive and coherent information system which has become the INSEE standard for businesses surveys. This transformation, to which many have contributed, has been achieved through a combination of consistency in adherence to the strategic vision and flexibility in tactical adaptation to changing circumstances. It has been fuelled by a permanent search for innovation, at technical, methodological and organisational levels. It has sometimes been necessary to take some risks, to push the organisation a little or convince some resistance, sometimes without success, and to deal with resources that were often very restricted; however, the result can be considered to be in line with the initial ambitions. This success has largely given credibility to the approach of managing information systems through metadata, which it now seems more natural to extend to other areas such as dissemination.

Eno and Pogues have also raised important issues, which have not been fully resolved to date: how to organise the project management of cross-functional projects and the resulting tools, how to better manage technical production innovation, etc.

But the story is just beginning. Eno and Pogues are now entering a new phase of development, in the context of the platform for the online collection of household surveys. There are functional enhancements already underway and architectural improvements are planned. It is also hoped that the promotional efforts made towards national statistical institutes of other countries will eventually pay off and that the expressions of interest will be converted into reuses or effective contributions. This will probably involve professionalising the internal organisation to better document the tool and improve user support.

Above all, we must hope that the future of Eno and Pogues will continue to be built on this desire to innovate that Brian Eno, in a way, described when asked about his creative process: “Ouvrir dans un nouvel ongletThe [...] thing is to set up a situation that presents you with something slightly beyond your reach”.