Low Response Rates: What They Mean and How to Improve Them

The survey landscape has evolved significantly over the past few decades, with response rates becoming a pervasive challenge across various sectors and public opinion polling. 

This trend presents a particular challenge for associations, which rely on surveys to gather valuable insights from their members and stakeholders. 

However, while response rates can be a useful metric, they aren’t the sole indicator of a survey’s effectiveness or design quality. To ensure meaningful engagement and maximize the value of their surveys, associations must adapt their strategies, making it easier and more rewarding for respondents to participate. 

The steady drop in response rates has been driven by factors such as survey fatigue and privacy concerns. In fact, public opinion surveys have seen a marked decline in participation, with Pew Research Center reporting a drop from a 36% response rate in 1997 to just 6% by 2018. While a higher response rate is generally positive, it’s important to recognize that response rates alone don’t guarantee a survey is well-designed or free from issues. 

Here are a few key reasons why response rates are just one part of the equation:

• Survey Design: A well-designed survey should be concise and relevant. Even with a high response rate, a poorly designed survey with leading questions, unclear wording, or irrelevant content can result in skewed or misleading data.
• Response Bias: High response rates may still be plagued by response bias, where only certain types of individuals respond. This can distort the results and provide a limited view of the broader membership.
• Incentive-Driven Responses: Offering incentives may help boost response rates but may also result in lower-quality responses. Some participants may rush through the survey just to receive the reward without giving thoughtful or honest answers.
• Sample Representation: A high response rate from a non-representative sample can lead to data that doesn’t accurately reflect the association’s full membership. In contrast, a well-targeted, lower-response survey with a more representative sample can yield more useful insights.

In fact, when researchers have evaluated surveys to determine the impact of response rates on data quality, it is commonly found that there is no direct connection between the level of response and the validity of survey results. This is because, most often, the respondents who elect to participate in surveys are similar to those who decline to participate. They do so because they have time, like to provide their opinion, for the incentive or some other reason, but not because they are biased towards your survey topic (e.g., they are advocates of the association). This is not to say that response rates are unimportant, but it does reveal that we tend to overemphasize them as a measure of quality.

Surveys remain one of the most direct and effective methods for associations to collect feedback. Surveys are more than just data collection tools. They are an opportunity for associations to gauge member satisfaction, inform strategic decisions, enhance engagement and measure program effectiveness: 

In short, well-executed surveys provide a continuous feedback loop that fuels association growth and relevancy. 

But that brings us back to the question McKinley gets asked regularly: “What is a good response rate, and how do we increase our response rates?” 

For association surveys, McKinley often sees rates similar to Pew’s metric, with members or other highly engaged stakeholders responding at a rate of about 5-6% and non-member stakeholders responding at an even lower rate of about 2-3%. 

That being said, the goal should always be to balance response rate with data quality. 

Rather than focusing solely on how many people complete a survey, associations should ensure their surveys are designed to gather high-quality, actionable insights. This means paying attention to how questions are framed, how the survey is structured, and ensuring that responses come from a representative cross-section of stakeholders. 

Here are four proven strategies associations can adopt to boost participation and maintain data quality:

• Keep Surveys Short and Focused: Shorter surveys (less than 10 minutes in length) generally receive better response rates but also more thoughtful responses. It’s essential to ask only the most important questions. Not only does this improve response rates, but it also enhances the quality of the data gathered, as respondents are less likely to become fatigued.
• Personalize the Invitation: Tailoring the survey invitation to specific groups, addressing recipients by name, and explaining the relevance of their participation can increase response rates. Personalization enhances the likelihood of engagement and shows respondents that their input is valued.
• Communicate the Impact: Clearly explaining how the feedback will be used can make respondents feel like they are contributing to something important. Show how previous survey results led to meaningful changes within the association or your respective industry or field to build trust and motivation to participate.
• Leverage Member Ambassadors: Peer-driven outreach can often generate better results than institutional requests. Member ambassadors can act as trusted voices, encouraging their peers to participate and emphasizing the importance of the survey.

Ultimately, surveys remain a valuable tool for associations to gather insights, engage stakeholders, and make data-driven decisions. By balancing response rates with thoughtful design, associations can create surveys that deliver actionable, reliable insights - even in the face of changing participation trends.