Research ·

Survey nonresponse and CSAT interpretation in outsourced support

What a satisfaction survey can and cannot show when most customers do not answer, and how to report the result honestly.

Survey nonresponse and CSAT interpretation in outsourced support research hero

Key stats

2

questions in a short survey

3

reporting figures required

1

theme acted on per month

Methodology and findings

Question and scope

This review asks what a customer satisfaction survey can show an outsourced help desk when most customers do not respond. The population is post-ticket surveys sent by email or in-app after a support interaction. The unit is one survey invitation and the response, if any. The question matters because a small set of happy or angry replies is often presented as the voice of the whole customer base.

The scope excludes relationship surveys sent outside a support interaction and excludes interviews run by a research team. It also excludes star ratings collected inside the agent chat window, which have a different timing and audience. Those boundaries keep the review on the survey most desks actually run. A finding should help a desk describe its result with the right amount of confidence.

What a satisfaction survey measures

A survey measures the customer's judgment at the moment they answer, filtered through the wording of the question. It does not measure the quality of the ticket, the accuracy of the answer, or the effort the customer spent before contacting support. A customer may rate a correct resolution poorly because the wait was long. A customer may rate a weak answer well because the agent was kind.

The measure is therefore useful as a signal about the experience, not as a verdict on the work. It becomes more useful when paired with the ticket record and with the customer's later behaviour. A single score without context invites the desk to optimize the score rather than the service. The review should treat satisfaction as one input among several.

Why nonresponse is not random

Customers who answer a survey are not a random sample of the customers who contacted support. People with an unusually good or unusually bad experience are more likely to respond, and people who are busy or indifferent are less likely. The mix differs by channel, request type, and customer segment. A response rate of one in ten can overrepresent both extremes at once.

The invitation itself also shapes who replies. A survey sent immediately after closure catches a different mood from one sent the next morning. A survey that requires a login loses customers who cannot sign in, which may be exactly the customers the desk most needs to hear from. The method decides part of the answer before the first response arrives.

Reading a low response rate

A low response rate does not make the responses worthless, but it limits what the desk can claim. Report the number of invitations and the number of responses beside the score, and describe the result as the view of those who answered. Do not say customers are satisfied when the evidence is a small self-selected group. The honest sentence is usually longer than the marketing sentence.

Look at whether the response rate changes after a process change, because that movement can be informative even when the score is stable. A drop in responses after a shorter survey may mean the invitation is easier to ignore or that customers have stopped reading. Compare like periods and note any change in the invitation method. The rate is part of the result, not a footnote.

Designing a shorter survey

Ask one question about the overall experience and one open question about what could be better. A short survey is easier to answer and easier to interpret than a long form with overlapping scales. If the desk needs a specific measure, add one question rather than five. Every additional question should have a named decision that depends on the answer.

Avoid wording that hints at the desired answer or mixes the agent and the process in one question. A customer cannot separately rate the agent and the wait if the question says the agent handled the wait well. Test the wording with a few customers or agents before sending it broadly. A clear question produces a usable signal even from a small sample.

The follow-up question

The open question often produces more useful information than the score, but it is also the hardest to summarize. Read the comments in full before grouping them, and keep a list of the exact phrases customers use. A phrase such as nobody told me what happened points to a communication gap that a rating alone would hide. The comment gives the desk a place to start.

Do not ask a second open question about the same interaction, because customers rarely answer both with care. If the desk needs a specific detail, such as whether the customer tried self-service first, make it a short choice rather than another paragraph. Every question should earn its place in the report. A shorter survey is more likely to be completed and more likely to be read.

Separating agent and process causes

A low score can come from the agent, the queue, the product, or the customer's prior experience. Read the ticket before assigning a cause, and look for a delay, a repeat contact, or a missed promise. If the customer waited three days for a first reply, the agent's tone is unlikely to be the main issue. The cause should be supported by the record, not inferred from the score.

Use the open comments to group causes, but do not treat one vivid comment as a trend. Count how often a theme appears and compare it with the sample size. A theme mentioned by three customers out of five is different from the same three out of fifty. Report the count with the theme so the reader can judge the weight.

Using verbatims without overreading

Quotes make a report readable, and they also make a single customer sound representative. Use a quote only when it illustrates a theme that appears more than once, and say how often the theme appeared. Remove names and account details, and keep the wording intact enough to be honest. A cleaned quote should not change the meaning.

Do not select only the quotes that support the conclusion. Include one comment that complicates the story when it exists. A report that acknowledges mixed evidence is more credible than one that presents a clean narrative. The client is more likely to trust a finding that shows its own limits.

Reporting to the client

Report the invitation count, the response count, the score, and the main themes in that order. State the period and the population, and note any change in the survey method. Avoid a single headline number without the denominator. A client who sees the denominator can decide how much weight to give the result.

Separate what the desk will do from what the client must decide. A theme about product confusion may need a documentation fix from the client, while a theme about slow replies may need a staffing change. Name the owner and the next review date. The report should end with actions, not with a score.

Limits of the finding

The review cannot correct for nonresponse bias without additional data, such as a follow-up with a random sample of nonresponders. It also cannot separate the agent's contribution from the customer's mood at the moment of answering. The result is a description of the respondents, not of all customers. The desk should say so plainly.

The finding is also affected by the survey tool, the send time, and the customer's channel. A change in any of those can move the score without any change in service. Keep the method stable when comparing periods, and record any change that cannot be avoided. A comparison across different methods is a comparison of methods as much as of service.

Applying the review

Take one month of survey invitations and responses, and count the responses by request type and channel. Read every open comment and group the themes, then compare the counts with the number of invitations. Choose one theme that appears often enough to act on and one that needs more evidence. The exercise shows whether the current report is telling a story the data can support.

Change one thing in the survey or the follow-up, such as the send time or the number of questions, and run it for a month. Compare the response rate, the score, and the themes with the earlier period, and note what else changed. The aim is a survey the desk can explain and a report the client can use. A modest, stable method is better than a precise-looking number with no denominator.

Sources

  1. BLS Occupational Outlook Handbook: customer service representatives - Role context for customer-facing support work.
  2. ISO 18295-1:2017 customer contact centres - Contact-centre performance measurement and customer feedback guidance.
  3. ISO 10002:2018 complaints handling - Complaints handling and customer feedback principles.
  4. GSA customer experience guidance - Public-sector customer experience measurement guidance.

Related research

Help desk staffing intake

Define the operating lane before staffing begins.

Share the queue, tools, schedule, approvals, and escalation limits. We will use them to prepare a focused help desk staffing brief.

Contact Us