Member Research: Why a Low Score Isn't Always the Warning It Looks Like

In sectors where membership is quasi-mandatory, recommendation-based metrics can mislead a board

A low member satisfaction score is not always the warning sign a board assumes it is, and a board that interprets association member research the same way a consumer business interprets customer satisfaction data is at risk of drawing the wrong conclusion from the right data. This deserves specific, careful attention, particularly given how directly member research connects to the value proposition testing discussed in our article on member value propositions.

The Methodological Caveat Most Boards Miss Entirely

Net Promoter Score, the widely used measure asking how likely a member is to recommend the association, has a limitation that is particularly relevant to associations. Where membership is effectively required for practice or employment, for example in some colleges and credentialing bodies, a low recommendation score may not signal dissatisfaction or churn risk. For associations where membership is voluntary, it often does. A member may remain enrolled regardless of how they actually feel, simply because membership is not discretionary, and recommending it to a colleague may not be a meaningful action at all when that colleague already belongs to the same body as a practical necessity. A board treating this the way a retail business treats a low recommendation score, as a direct signal of imminent loss, is drawing a conclusion the data does not actually support in this specific context. A low likelihood to recommend, in a profession where membership is effectively mandatory, may mean nothing more than that recommending isn't a meaningful concept for this member, not that they are quietly planning to leave.

The Bias Problem Every Survey Carries

Survey methodology itself introduces bias worth understanding before acting on results. Low response rates tend to over-represent members with the strongest opinions in either direction, meaning a board may be looking at the views of a vocal minority rather than the membership as a whole. The survey method itself matters too: interviewer-led surveys can produce more favourable answers than anonymous self-completed surveys, because people tend to soften criticism when speaking to someone directly, while anonymous formats tend to surface more critical feedback. A board reviewing survey results should ask about response rate and methodology before treating a single score as a reliable verdict on member sentiment.

The Governance Failure That Matters More Than Methodology

The more common and more damaging failure is not methodological. It is running a member survey, collecting feedback, and never visibly acting on it or closing the loop with the members who responded. A survey that produces data nobody uses to change a real decision does not simply waste the effort of running it. It quietly trains members that responding to future research is pointless, degrading response rates and data quality on every subsequent survey the association runs. Every question included should map to a decision the association would make differently depending on the answer. If a question would not change what the organisation does, it does not belong in the survey.

• Interpret recommendation-based metrics specifically in light of whether membership is discretionary for your sector, rather than applying consumer-business benchmarks uncritically.
• Pair any standardised satisfaction or loyalty metric with open-ended diagnostic questions, since the score alone rarely explains why members feel the way they do.
• Assess response rate and survey method honestly before treating results as representative of the full membership, rather than a vocal minority at either extreme. Ask for the response rate, how respondents compare with the membership profile, and the trend over time.
• Give respondents a clear collection notice, collect only the personal information you need, and report results in de-identified form.
• Design every survey question to map to an actual decision the association would make differently depending on the answer, cutting anything that would not change a real outcome.
• Close the loop visibly with members after every survey, communicating what was learned and what changed as a result, since this is what determines whether future research efforts actually get engagement.

Member research is one of the most valuable governance tools an association has for testing whether its value proposition and strategic direction match what members experience, connecting directly to our article on member value propositions. It is only valuable, however, when the board understands its limitations and commits to actually acting on what it finds. This is one of the practical governance topics built into our Association CEO course, alongside the papers, tools and frameworks that turn the principle into your board's actual practice. Explore the course →

Annie Gibbins

General education, not legal, financial, tax, clinical or governance advice. Confirm specifics at the relevant primary source or with your own qualified adviser. Nexus Leadership is operated by Lipstick Consulting Pty Ltd · ABN 15 619 120 482.