Garbage Data

In a recent Monday Mash-Up, I shared a blog post I’d written about misleading statistics.

It struck a nerve with several readers. Margaret is one of those readers who told me the story of her run-in with garbage data.

Margaret works as a fundraiser at a nonprofit. Margaret often wonders how best to reach older donors and prospects. Some older folks say they prefer print newsletters and other “traditional” communications. 

That’s bad news for digital communication vendors who want nonprofits to use their tools for fundraising. 

Margaret received an email from one such company that trumpeted big news: “Use of Digital Technology No Barrier for Older Donors.” 

This news came from a study the company commissioned that revealed, “93 percent of matures (age 73+) said they are comfortable using digital devices.” 

In its email to Margaret and other prospects, the company insisted, “Using one or more digital channels to reach donors should be a part of your planned giving marketing program.” 

Margaret was skeptical. In her experience, many “matures” were comfortable with digital services, but not 93 percent!

“I have tried to train a few 80-year-olds on digital services,” she told me. 

So Margaret dug deeper and discovered that the survey was conducted by a consulting firm called NMI. 

Margaret told me NMI conducts its Healthy Aging Trends survey online. I visited their website and confirmed that NMI collects this data “via online research methodology.” 

So let me get this straight…

A consulting firm contacts elderly people via online channels. Those elderly people are online and able to respond to an online survey. And the researchers ask, “Are you comfortable online?”

I’m surprised only 93 percent said, “Yes.” Makes me wonder what’s up with the seven percent who are responding to online surveys but aren’t comfortable with digital services. 

So…after we get to the fine print and assess this survey, should we assume that 93 percent of ALL “matures” are comfortable with digital services? No way. 

Do we have any idea what percentage of ALL “matures” (not just the ones responding to online surveys) are comfortable with digital services? Nope! 

So what valid conclusions can we reach from this information? None, really. 

That’s the working definition of garbage data. Purposely misleading at worst. Meaningless at best. 

As Margaret told me, “For their next trick, I bet they will go to Disney World and ask people there whether or note they like Disney World.”