1 of 23

DDI webinar: Questions and questionnaires

Becky Oldroyd, CLOSER Metadata Officer

22 May 2024

2 of 23

Outline

Introduction to CLOSER and CLOSER Discovery
Documenting questions and questionnaires
Benefits of documenting questions and questionnaires
Summary

3 of 23

What is CLOSER?

“Increase the visibility, use and impact of longitudinal population studies, data and research”

4 of 23

CLOSER resources

Discovery

Search and explore data from UK longitudinal population studies

Our Changing Society

Historical and political backdrop to the lives of study participants

Cross-study guides

Interactive guides on cognition, physical activity, and diet measures

Learning Hub

Resources for those new to longitudinal population studies

Training Hub

Advanced learning resources and training opportunities

Policy Hub

Bespoke guidance, tools and templates to help mobilise research

5 of 23

CLOSER Discovery YouTube video

https://www.youtube.com/watch?v=9mFF4hTPOrI&t=7s

…Navigating the complexities of locating, understanding and using this data can be a daunting task…

6 of 23

What is CLOSER Discovery?

The UK’s most comprehensive research tool for longitudinal population studies

7 of 23

What is CLOSER Discovery? (2)

The UK’s most comprehensive research tool for longitudinal population studies
Does not contain data (only metadata)

8 of 23

How do we document question and questionnaire metadata?

DDI allows you to document questions and questionnaires
Break the questionnaire down into its elements
Archivist: CLOSER’s in-house developed, open-source DDI questionnaire editor:

Exports DDI-Lifecycle version 3.2
No knowledge of DDI is required
Work carried out by trained but not technical staff

As previously mentioned, CLOSER Discovery contains hundreds of whole questionnaires and thousands of individual questions. So how do we create and document this metadata?
DDI allows you to document questions and questionnaires by breaking the questionnaire down into specific elements, such as the question text, instructions, responses, and the routing.
To create questionnaire metadata, we use Archivist which is CLOSER’s in-house developed, open-source web application which allows multiple users to enter and edit questionnaire metadata. It currently exports DDI-L XML, which can then be loaded into CLOSER Discovery.
Because Archivist was designed to simplify the questionnaire metadata entry process, no knowledge of DDI is required to document the metadata, meaning work is carried out by trained but not technical staff, who are trained on how to identify the questionnaire elements rather than the specifics of DDI.
So to show you what this process looks like in practice, here’s an example of entering a question in Archivist…

9 of 23

Example: questionnaire

Thank you for filling this in. Children of the 90s loves to look at the things you draw!

Things For You to Do

Question name/label

Question text

Code list

Condition/routing

Question text

Statement

10 of 23

Example: entering a code list into Archivist

So to give you an example of how some of this information would be input into Archivist, I’ll show you how a question and a response would be entered.

When entering questionnaire metadata, we start by entering the responses and then build up to the questions and then the flow of the questionnaire.

This is what it looks like to enter a response, and in this case, a code list.

First we want to give the code list a label (labelled after the first letter of each word in the code list if it is used for more than one question, otherwise it’s labelled after the question)
Enter the min and max number of responses the respondent should’ve selected aka the cardinality.
Enter the code list categories and values.
Press save.
At the bottom you can see what questions use that code list – good if you need to edit it to see what the knock-on effects are.

11 of 23

Example: entering a question into Archivist

12 of 23

Example of a question in Archivist

13 of 23

Example of a question in CLOSER Discovery

And here we have the metadata in CLOSER Discovery.

In CLOSER Discovery, questions are presented in the context of the full questionnaire which we can see a snippet of on the screen.

Being able to see the question within the full questionnaire, provides additional context and understanding of the data.

For example, a useful piece of information you can see in the questionnaire here is the routing.

At the bottom you can see that if they answered No to question A1, they skip the next question and go to question A3. This is useful when working out the sample for a particular question, and might shed light on why there are a high number of missing cases for a particular variable.

In CLOSER Discovery, questions in questionnaires are linked to variables in datasets, so if we click on question A1…

14 of 23

Summary statistics for a variable

We can then find information about the related variable, which provides further detail and context with the summary statistics.

This is particularly helpful because it allows data users to assess the quality of the data before they access it.

For example, if there’s a high number of missing or invalid responses, being able to go back to the question and see the routing might give us a clue as to why.

We recently had feedback from a researcher who said they particularly liked this feature as they would usually download the data and run a frequency analysis to get this information, so it cuts out that step in the process for them.

If the user has looked at the variable and associated question and decided they would like to access or request the data, they can easily get to the right place to do this by clicking on the dataset at the top…

15 of 23

Dataset location

16 of 23

Variables connected to questions

17 of 23

Social media: search results

18 of 23

The value of source questions

19 of 23

Example variable in Discovery

20 of 23

Importance of variable label and question text

21 of 23

Summary

Provides variable provenance - users can understand context of the data
Provides more information than the variable
Assess the suitability of the data before accessing/requesting
Increases discoverability of data
Helps to surface potentially useful variables
Variable description and question don’t always match up well

22 of 23

User feedback

“It’s a joy to be able to search various different studies…and find a study that asks relevant questions and instantly be able to see all the information I need”

UCL Researcher

23 of 23

discovery.closer.ac.uk

Visit CLOSER Discovery

Thank you

Sign up to our newsletters

Longitudinal news
Blog alerts
Twitter highlights
In the Loop

@closer_uk

@closer-uk

/closer-uk

closer.ac.uk