One of the most common questions we get about Natsal is: How do you know if people are telling the truth? The honest answer is that we can’t know for certain. That’s why the findings are described as reported behaviours and attitudes, because we can’t always know how closely they reflect true attitudes or behaviours.  

That said, the Natsal team works hard to create an environment where people feel safe and comfortable giving honest answers. From the first letter that arrives at peoples’ home addresses, the Natsal team explain why Natsal is important and how it helps shape public health and services. This helps people understand that their answers can make a real difference. In both face-to-face and telephone interviews, the most sensitive questions are answered privately through a self-completion section, so the interviewer can’t see the responses. Specific language is also used in the questionnaires and tools like the use of showcards to encourage participants to provide honest answers.   

Even with all these steps, there may still be differences in how people choose to answer. There are many reasons for this. For example, someone might answer differently to a question about sexual violence if their partner is nearby during the call, some might have difficulties remembering details and some of the response options that Natsal offers may not be relevant for everyone because sex lives are sometimes complicated. Natsal isn’t perfect, but we do everything we can to make sure participants feel respected, protected, and heard. 

On the whole, participants answer all the questions that are asked. Occasionally a participant will choose not to answer a particular question, but for most questions less than 3% of those eligible to answer will refuse to do so. 

Despite Natsal asking some very sensitive questions about sexual behaviour and sexual health, the question that has the highest level of refusal in Natsal-4 is household income. 

While some participants may refuse to answer this question as they consider it too intrusive, others may simply not know the answer as the question is about household income – especially those living in shared accommodation or younger participants living with their parents. 

It is really important to be as transparent as possible about the data and statistics underlying any analyses. This includes stating the data source (e.g. Natsal-4), who the statistics are based upon (e.g. everyone regardless of their sexual experience or just those reporting that they had had sex during a particular timeframe), any exclusions (e.g. people who refused to answer the corresponding question – especially if this is a considerable number), and the statistical techniques used (e.g. rather than just saying ‘average’ state the type of average – mean or median). 

As a result, tables and charts often end up having lots of footnotes, but this is good practice. Because all those details help to maximise transparency and enhance the reader’s trust in the reported findings. 

Surveys like Natsal can be incredibly useful for understanding what is going on in the population, but numbers alone can’t tell us everything. That’s why it’s important to recognise the limits of quantitative data (that is, data derived from numbers) and to consider how it can be complemented by qualitative data (which includes words, narratives, and art).   

While quantitative methods can show us what is happening, when and where, qualitative data adds context, depth and meaning, by helping us understand why it’s happening or how people feel about it. When used together, quantitative and qualitative methods enrich each other. 

Sometimes surveys reveal unexpected trends that raise questions that qualitative research can help explore. For example, after Natsal-3, the team conducted 126 qualitative interviews with survey participants to help them explore in greater depth topics such as involuntary childlessness and risk of sexually transmitted diseases in mid-life. Other times, qualitative research comes first, guiding the design of surveys like Natsal to better reflect people's experiences. For example, in Natsal-4, a newly developed measure of sexual wellbeing was included which was informed by findings from qualitative interviews with adults which explored how they defined sexual wellbeing.  

Natsal aims to collect data that is broadly representative of the general population. However, it is important to note that the list of home addresses used to recruit participants does not include everyone. This list includes all private residential addresses, so this means that people who are homeless or living in an institution like a prison or in residential care are excluded.  

Furthermore, some of the people who took part in Natsal-4 were recruited online from a pool of volunteers. While this made it easier for some to take part, it can also create challenges. Not everyone has the same access to the internet or feels confident using technology. Certain people, such as older adults or those with lower incomes, may be less likely to have access to a smartphone, computer, or reliable internet. Individuals who volunteer for online panels might have different experiences and needs when it comes to relationships, sexual health and wellbeing. For this reason, in Natsal-4 estimates of how common a particular behaviour, experiences or health outcomes are in Britain are based on the random sample only. For other analyses, such as exploring links between different factors, we use the full study sample while taking care to account for differences between participants, helping to ensure that the findings are robust and reliable. 

If you would like to learn more about statistics, there are a number of free courses available to you. For instance, OpenLearn’s course “Exploring data: Graphs and numerical summaries” introduces different ways quantitative data is presented graphically and how data is summarised. The course “Using statistics in your research” introduces the different ways that statistics may be misrepresented or biased.