How Survey Response Rates Affect Polling Results
by Gotham Polling
Survey response rates are an important part of understanding how polls are conducted and how much confidence readers should place in their findings. A poll may contact thousands of people, but only a portion of them may ultimately participate. The percentage that responds can provide useful information about the survey process, but response rate alone does not determine whether a poll is accurate.
Modern polling takes place in an environment where people are increasingly difficult to reach. Calls go unanswered, emails are ignored, survey invitations compete for attention, and some people are simply less willing to participate in research than others.
For pollsters, the key question is therefore not only how many people responded, but also who responded, who did not, and whether those differences could affect the results.
Understanding survey response rates helps explain why professional polling involves sampling, fieldwork monitoring, weighting, quality controls, and transparent reporting.
What Is a Survey Response Rate?
A survey response rate measures the proportion of eligible people selected or contacted for a survey who ultimately participate.
In simple terms, if a survey reaches a group of potential respondents and only some of them complete the questionnaire, the response rate describes the level of participation.
However, calculating response rates can be more complicated than simply dividing completed surveys by the number of invitations sent. Researchers may need to consider factors such as:
- Whether the person contacted was eligible for the survey
- Whether contact was successfully established
- Whether a survey was partially or fully completed
- Whether the contact information was valid
- How unknown eligibility cases are treated
Different survey modes can also produce different participation patterns. Telephone, online, text-based, mail, and in-person surveys each have their own challenges when researchers attempt to reach respondents.
This is one reason response rates should be interpreted as part of the broader polling methodology rather than as an isolated measure of quality.
Why Have Survey Response Rates Declined?
Getting people to participate in surveys has become more difficult.
Consumers regularly receive phone calls, emails, text messages, advertisements, and other digital notifications. At the same time, caller identification and spam filtering make it easier to ignore unfamiliar contacts.
Some potential respondents may also be skeptical about unsolicited surveys or reluctant to provide information to organizations they do not recognize.
The result is that modern pollsters often need to contact considerably more people to obtain the number of completed interviews required for a study.
Research from the Pew Research Center illustrates how dramatically participation has changed. Pew reports that its telephone survey response rates declined from 36% in 1997 to the low single digits in more recent years.
Yet declining response rates do not automatically mean that modern polls cannot produce useful estimates.
The more important issue is whether the people who participate differ systematically from the people who do not.
Does a Low Survey Response Rate Make a Poll Inaccurate?
Not necessarily.
This distinction is critical when evaluating polling results.
Imagine two surveys that each obtain 1,000 completed responses.
The first survey has a relatively high response rate but disproportionately attracts respondents from one particular demographic or political group.
The second has a lower response rate, but its respondents closely reflect the target population across relevant characteristics.
The first survey could potentially produce greater distortion even though its response rate is higher.
The central problem is therefore not simply nonresponse. It is nonresponse that creates meaningful differences between respondents and nonrespondents.
This issue is closely related to survey response bias, which occurs when participation patterns cause completed responses to misrepresent the population being studied.
A low response rate can increase the opportunity for this problem to occur, but it does not prove that it has occurred.
How Response Rates Can Affect Poll Results
Response rates become particularly important when willingness to participate is connected to the attitudes, behaviors, or characteristics being measured.
Suppose a political survey is intended to represent registered voters in a city.
If voters across political groups, age groups, neighborhoods, educational backgrounds, and other relevant characteristics participate at roughly similar rates, lower overall participation may have a limited effect on the final estimates.
But suppose younger voters respond much less frequently than older voters.
If younger and older voters also have different candidate preferences, the raw survey results could overrepresent the opinions of older voters.
Similar patterns could occur when response rates differ according to:
- Political affiliation
- Education
- Age
- Race or ethnicity
- Geographic area
- Income
- Voting history
- Political engagement
- Interest in the survey topic
The important question is whether differences in participation are related to the outcome being measured.
When they are, pollsters need to determine whether methodological adjustments can reduce the resulting imbalance.
Response Rate vs. Survey Participation Rate
The terms response rate and survey participation rate are sometimes used casually as though they mean exactly the same thing.
Both describe participation, but professional survey research may define and calculate different fieldwork metrics in specific ways.
For readers evaluating a published poll, the terminology matters less than understanding the underlying process.
Useful questions include:
- How were people selected?
- How were they contacted?
- How many completed the survey?
- Were certain groups more likely to respond?
- Was the completed sample compared with population benchmarks?
- Were adjustments made when groups were underrepresented?
These questions provide more information about survey quality than simply asking whether the response rate was high or low.
How Pollsters Address Low Response Rates
Modern polling organizations use several strategies to manage differences in survey participation.
Reach Respondents Through Appropriate Methods
Using an appropriate survey mode is one of the first considerations.
Depending on the population, researchers may use telephone interviews, online panels, SMS invitations, mail surveys, in-person research, or combinations of different approaches.
The objective is to give members of the target population a reasonable opportunity to participate.
Monitor Responses During Fieldwork
Researchers can examine the composition of completed interviews while data collection is still underway.
If one segment of the population is responding less frequently than expected, additional outreach or sampling efforts may be considered where appropriate.
Compare the Sample With Reliable Benchmarks
After responses have been collected, researchers can compare the survey sample with reliable information about the target population.
For example, a voter survey might evaluate whether respondents appropriately represent the population across characteristics such as age, geography, gender, education, race, or party registration, depending on the research design.
Apply Survey Weighting
Pollsters may also use statistical weighting to adjust the influence of respondents when the completed sample does not perfectly reflect the target population.
For example, if a group represents 25% of the relevant population but only 18% of the completed sample, weighting may increase that group’s contribution to the final estimate.
Gotham Polling has previously explained in more detail how survey weighting methods can improve poll accuracy.
Weighting, however, is not a substitute for good survey design. Researchers still need an appropriate sample, reliable benchmarks, quality data collection, and reasonable assumptions about the differences between respondents and nonrespondents.
Why More Responses Do Not Automatically Mean Better Results
A common misconception is that a survey becomes reliable simply by collecting a very large number of responses.
Sample size matters because larger samples can reduce sampling uncertainty. But a large sample cannot automatically correct a systematic participation problem.
Consider an online poll that receives 20,000 responses but is primarily completed by people who are especially interested in the issue being studied.
Another professionally sampled poll may contain only 1,000 respondents but use a research design intended to represent the broader population.
The larger poll does not necessarily provide the better estimate.
Representation matters alongside sample size.
This is especially important when interpreting online polls, voluntary questionnaires, social media surveys, and other forms of research where participants may effectively select themselves into the sample.
Response Rates and Political Polling
Survey response rates are particularly important in political polling because differences between groups can influence candidate support, issue preferences, and estimates of voter behavior.
Political engagement itself can affect survey participation.
People who closely follow elections may be more willing to discuss politics than people who are less engaged. Supporters of different candidates or parties may also vary in their willingness to answer surveys at different points in an election cycle.
These differences make it essential for political pollsters to evaluate the composition of their respondents rather than relying solely on the number of completed interviews.
Researchers may consider demographic information, registration data, geographic distribution, previous voting behavior, party identification, or other variables depending on the target population and methodology.
The objective is not to force survey results toward a predetermined outcome. It is to make the final estimates as representative of the intended population as the available data and research design reasonably allow.
How Readers Should Evaluate Survey Response Rates
When reading a poll, avoid treating the response rate as a simple pass-or-fail test.
Instead, consider it alongside the rest of the methodology.
A useful evaluation should ask:
- Who was the survey designed to represent?
- How was the sample selected?
- What method was used to contact respondents?
- How many people completed the poll?
- Were some population groups underrepresented?
- Was the data weighted?
- What variables were included in weighting?
- Were fieldwork procedures and limitations disclosed?
- Are the conclusions supported by the methodology?
Transparency is especially important. Pollsters should provide enough information for readers, journalists, campaigns, organizations, and researchers to understand how the data was collected and how the final estimates were produced.
The Bottom Line
Survey response rates matter because every poll depends on people choosing to participate. When participation is low, researchers have fewer responses relative to the number of people they attempt to reach, which can increase concerns about whether respondents differ from nonrespondents.
But a low survey response rate does not automatically mean a poll is inaccurate.
What matters most is whether differences in participation create meaningful distortions in the sample and whether the survey methodology appropriately identifies and addresses those differences.
Reliable polling therefore requires more than maximizing the number of responses. It requires careful sampling, appropriate data collection, monitoring of respondent composition, statistical adjustment when justified, and transparent reporting.
At Gotham Polling & Analytics, understanding participation patterns is one part of a broader commitment to producing polling and survey research that accurately reflects the populations being studied. By examining both who responds and who may be missing, researchers can provide decision-makers with a clearer and more responsible interpretation of public opinion.