Survey fatigue is real, insight from Twitter can help

"Survey finds 95% of people don't like taking surveys," could be the headline of your favourite satirical website, which wouldn't be far from the truth. Overwhelmed by the increasing number of requests for feedback, consumers are avoiding surveys all together which is impacting response rates. The telephone survey, for example, has seen a steep decline in response rates since the mid-90s, dropping from 36% in 1997 to just 6% in 2018, according to Pew Research¹. Of course, many variables are at play when it comes to response rates across different types of surveys, but the reality is that survey fatigue is real and it impacts both the volume and quality of feedback received. The insight they provide is also limited to the questions asked. So, is there another way?

I recently ran an experiment to understand if insights from Tweets could answer many of the same questions that customer surveys commonly ask. To do this I chose a consumer-focused industry⁠—airline travel⁠—and focused on a specific carrier (who shall remain anonymous). Given the disruption to the industry brought on by COVID-19, I chose to analyse the month of January 2020, as this was one of the last few months of unaffected travel.

In January 2020 alone, the airline was mentioned 70,000 times on Twitter which provided a large amount of content to sift through. In particular, I was looking for Tweets from travellers sharing their preflight, inflight and postflight experiences and was guided by the topics that bubbled to the surface organically. Here are some examples of what I found:

Preflight Experience

• Check-in: 567 mentions including app check-in, visa checks and bag drop
• Lounge: 443 mentions including food quality, shower cleanliness and staffing
• Boarding: 223 mentions including express boarding and shuttle buses

Inflight Experience

• WiFi: 34 mentions of connectivity issues or lack of service
• Entertainment system: 27 mentions of small or dirty video screens
• Food quality: 372 mentions including lack of vegan options, pricing and selection

Postflight Experience

• Baggage collection: 374 mentions including delayed, lost and damaged baggage
• Loyalty program: 438 mentions of air mile collection and usage policies
• Compensation claims: 131 mentions of confusing rules

These comments alone constituted nearly 2,600 pieces of customer feedback covering the entire customer journey⁠—insights that the airline could potentially use to improve the experience for all passengers. Crucially, they uncovered specific pieces of feedback that might not normally be surfaced by surveys due to the limitations of the questions asked. For example:

• Departures check-in: One person lamented that her elderly parent struggled to follow the prompts on the check-in kiosks.
• Onboard WiFi purchase: One traveller asked why there was no pay-as-you-go option.
• Onboard electrical plugs: Several travellers mentioned the placement and number of electrical plugs and some even provided suggestions on where they should be placed.
• Meal choices: Several passengers expressed their annoyance that gluten-free meal options had to be pre-ordered.

This analysis did not attempt to create a representative sample of customer views nor to draw any type of causal relationship between these comments and overall satisfaction with the airline. I simply set out to prove that the breadth and depth of conversations happening on Twitter can provide a real-time voice of the customer at every touchpoint in the customer journey.

The findings suggest that insights from Twitter can indeed shed light on many facets of the customer experience, providing a unique window into real observed human behaviours with several advantages over traditional insight gathering techniques.

First, the unsolicited nature of Twitter feedback circumvents one of the biggest challenges of research—the introduction of bias by the questions being asked. Secondly, as Tweets are sent at the moment people are actually having the experience, they tend to be more emotive as a result and represent a more authentic view of a customer’s true feelings. This is much different than being asked to recount your experience with the onboard WiFi a few days later via an email prompted survey. At that point, the moment is gone and you might not even bother to respond. This strength of social data should not be underestimated, for very little market research delivered today is conducted with the context in mind.

And finally, the open, continuous nature of Twitter allows you to see trends emerge over time and benchmark yourself against your competitors. It also has the advantage of allowing you to go back and analyze a topic retrospectively, uncovering feedback preserved in time, forever.

Customer experience measurement has come a long way over the past few years as new technologies and techniques have emerged to ascertain consumers’ true feelings. The reality is that it’s not just one approach that will provide the insight organizations need but rather a combination of many, used at the right time and leveraging their respective data gathering strengths. Insights from Twitter won’t fully replace surveys, but they can complement them, providing context to common metrics such as NPS while helping to focus the questions you do want to ask of your customers. Perhaps Twitter’s greatest strength then is in providing answers to the questions you didn’t even think to ask.

Interested to learn more about how Twitter can support your customer feedback initiatives? Get started at data.twitter.com.

1. “Response rates to telephone public opinion polls.” Pew Research Center, Washington, D.C. (February 27, 2019) https://www.pewresearch.org/fact-tank/2019/02/27/response-rates-in-telephone-surveys-have-resumed-their-decline/