Who Prefers Guessing to Admitting They Don’t Know? Measurement Error in Financial Literacy Surveys

Bertola G., Lo Prete A., 2025 – Journal of Economic Behavior & Organization

Indicators of financial literacy derived from sample surveys are now widely used in the economic literature to explain financial choices, market participation, and savings behaviour. But what do these indicators really capture when individuals’ level of competence is measured through answers to questions included in sample surveys?

In the article “Who Prefers Guessing to Admitting They Don’t Know? Measurement Error in Financial Literacy Surveys,” published in the Journal of Economic Behaviour & Organization, Giuseppe Bertola and Anna Lo Prete, address this issue by contributing to the literature through a framework that theoretically models and empirically accounts for measurement errors arising from the behavior referred to as “guessing”, that is the individuals’ propensity to guess when they do not know the correct answer.

The authors develop a structural model in which survey participants, faced with multiple-choice or closed questions to which they do not know the answer, may prefer to provide a random answer rather than admit ignorance. As a result, they choose not to use the “Don’t know” option when available. In this article, the measurement error is related to the probability that, by answering randomly, some individuals may guess the correct answer and be erroneously classified as competent.

The empirical analysis relies on a survey conducted by Doxa for the Financial Education Committee in 2020 and 2021, aimed at assessing the financial literacy and financial fragility of Italian households before and after the COVID-19 pandemic. This provides an interesting case study for testing the model’s predictions. In the context of financial education, “financial literacy” indicators are constructed by asking questions on topics such as interest calculation, understanding of inflation, and risk management through diversification. The Doxa survey has the advantage, compared to others, of having collected responses to the same financial competence questions in two consecutive years from a sample of over 4,000 families, thus allowing a clearer distinction between real knowledge and correct answers obtained purely by chance.

The data analysis shows that the indicators used to measure financial literacy suffer from a high degree of misclassification, due to the tendency to answer randomly rather than admit ignorance. Almost one-third of people in 2020 who had answered all three questions correctly, a necessary condition for being classified as financially literate, made at least one mistake in the following year. Among subgroups of the population, the difference between the proportion of those who are classified as literate in the first year and those who are literate for two consecutive years, about 13 percentage points, is surprisingly similar to the difference observed in the entire population, as highlighted in Figure 1.

A more accurate measurement of participants’ actual level of knowledge, which removes the guessing component from the indicators, allows for a better understanding of what lies behind the competence differences that previous studies have related to factors such as gender, age, education, and confidence in one’s abilities. For example (see Table 1 below), the more difficult the questions, the higher the probability that males will attempt to guess, thus reducing the commonly observed gender gap in sample surveys. Highly educated individuals, who may feel more embarrassed than others to admit their ignorance, and those who overestimate their financial competence, are also more likely to guess rather than admit they do not know.

Not only are truly competent individuals fewer than financial literacy indicators typically suggest, but random response behavior is also relevant for personal finance management. In fact, individuals erroneously classified as competent have more difficulty making ends meet and coping with unexpected financial needs. In a companion paper based on the same data and published in the Italian Economic Journal in 2025, the authors also demonstrate that actual financial competence (purged of the guessing component) is associated with better risk-sharing in the early stages of the COVID-19 pandemic.

Overall, the findings of this research provide practical insights for improving the design of questionnaires aimed at measuring knowledge in a given field. A more accurate assessment of individuals’ actual competencies requires the inclusion of repeated responses within the same questionnaire or across identical repeated questionnaires over time. Likewise, careful question formulation (an element that, according to the authors, is itself a source of significant measurement errors), coupled with the explicit inclusion of the “I don’t know” option, can significantly enhance the overall reliability of surveys, reducing errors caused by guessing rather than admitting ignorance.

Discussing the reliability of these tools is also crucial for formulating effective public policies and targeted interventions, especially in the field of public education. This is a very pressing issue, given that financial education has recently been included in the Italian school curricula as part of civic education, aiming to raise awareness about personal finance decisions and reduce errors in money management.

The results of this and other studies suggest that the lack of financial literacy, as measured by surveys, is also a symptom of deeper cognitive and behavioral weaknesses, which, unsurprisingly, lead to poor economic and financial outcomes. When it comes to addressing complex decision-making problems, advanced skills are required, which individuals may find too difficult and costly to acquire. Society may therefore need to make certain forms of social insurance compulsory and regulate access to, and the supply of, complex financial products.

If the problem is behavioral, a broader and deeper economic education is needed, one that goes beyond teaching personal finance knowledge. It should focus primarily on helping individuals make sound personal and collective choices, bringing attention back to the discussion of individual and collective well-being objectives that they intend to pursue.