Check Which People a Survey Percentage Describes

A search result says that 75% of members prefer morning sessions. Before repeating that figure, find out which members the percentage describes. It might mean everyone in the organization, everyone who returned a questionnaire, or only the people who answered one particular question.

The denominator is the group used as the total when calculating the percentage. Finding it can change the sentence you should write even when the arithmetic in the original report is correct.

Follow the number to its original table

Keep the claim you found, including its wording, date and attributed source. Then look for the original report, results table or published questionnaire rather than collecting more summaries of the same number.

Search using the organization name, the report title if available, and distinctive words from the claim. Once you open the report, inspect the table title, column headings, question wording and notes beneath the figures. A prominent overall sample size may not be the base for every question.

Record the exact table or section so another reader can find it. If you can locate only a promotional summary, mark the calculation base as unverified. Do not silently borrow a sample size from a different report by the same organization.

Build a small count ledger

Consider this entirely fictional club survey. The club has 200 members and invites all of them to answer a scheduling question. It receives one valid response from each of 80 members. Every respondent answers that question, choosing one of two options: 60 select morning and 20 select evening. The results are simple unweighted counts.

The ledger is:

• Members invited: 200.
• Members who responded and answered this question: 80.
• Morning selections: 60.
• Evening selections: 20.
• Members who did not respond: 120.

The reported morning percentage is 60 divided by 80, multiplied by 100 = 75%. It describes the respondents' answers. A faithful sentence is: “In the club's survey, 60 of the 80 respondents, or 75%, selected morning sessions.”

The broader sentence “75% of the club's 200 members prefer mornings” is not established by these counts. It extends the observed answers to people whose preference was not recorded.

Keep a second calculation from becoming a second mistake

You can also calculate 60 divided by 200, multiplied by 100 = 30%. That means 30% of all invited members recorded a morning preference through this survey. It does not establish that only 30% of members prefer mornings, because some nonrespondents might also prefer them.

Similarly, the 120 nonrespondents cannot be added to the evening group. Their silence does not choose an option. Keep those 120 as unknown in this example rather than turning them into votes for either side.

Statistics Canada's survey guidance notes that people who do not respond may differ from those who do in ways that affect results. The example therefore supports a description of the returned answers, not an unqualified claim about the whole membership. Assessing whether a real survey supports wider estimates requires its sampling and analysis methods.

Check whether the question has its own base

The fictional example deliberately makes all 80 respondents answer the same question. Real reports may handle skipped questions, eligibility filters and answer categories differently. Look for the base attached to the specific result, then read how missing answers were treated.

For instance, a question shown only to current subscribers has a different audience from a question shown to everyone surveyed. A percentage among people who expressed a preference also differs from one that includes “no preference.” Preserve that distinction in your note.

Check whether the percentage is an unweighted count or a weighted estimate. Survey weights can give respondents different contributions to the published estimate. A simple division of raw headcounts may therefore not reproduce a weighted percentage. Read the methodology rather than treating that difference alone as evidence of an error.

Keep the wording as narrow as the evidence

The exact question matters too. Selecting a preferred time does not necessarily mean promising to attend, being available every week, or supporting every detail of a proposed schedule. Statistics Canada's guidance recommends examining the actual questionnaire wording and response choices.

In the fictional club example, write “selected morning sessions” rather than “will attend morning sessions.” Keep the survey period with the result so an old preference is not presented as a current measurement.

A compact research note can contain: the original claim, the question, the counted answer, the calculation base, the collection period, the source location and the remaining uncertainty. For this example, the uncertainty is the preference of the 120 members who did not respond.

If the original report does not reveal the base or method needed to interpret a central figure, state that limit or leave the figure out. The useful outcome is a percentage attached to the people and question it actually describes.