Columns

OPINION – What should be rated the highest in surveys

It’s the 21st century’s version of the downtown coffee shop, minus the stained counters, grouchy waitresses and post-caffeine withdrawal.
It’s the Internet, a genuine cultural pulse of our nation and frequent first stop for the gabby among us…those that begin chatting with their fingers the moment something new comes on to their radar.
This online watercooler is the perfect place to start a discussion on whether the J.D. Power Motorcycle Satisfaction Survey’s foray into naming names is healthy for the industry. Is it the right thing to do or should the renowned research and marketing firm rethink its publicizing of how the top motorcycle manufacturers fare in the eyes of the new bike buyer?
From a www.totalmotorcycle.com forum writer: “I smell an anti-HD conspiracy in this survey.”
From a www.ducati.ms/forums posting: “I find this study interesting and in general, truthful.”
From a www.kawasakimotorcycle.org/forum writer: “If I could argue the results for Kawasaki, I’d maybe give one more bubble in ‘Product’ and one less bubble in ‘Cost of Ownership.’”
Bubbles, or circles, are part of J.D. Power’s rating system, something that before this year the public attached to ratings of cars, cell phones and other popular consumer items. Now, the bubble discussion is floating amongst online motorcycle forums. So perhaps the question of, “Should this be happening in our industry?” should be set aside for the more useful question of, “Can it be better? Can the survey be more useful to consumers as well as the industry?”
After all, the survey’s results are, as evidenced by a short meander through online forums, obviously a topic of discussion. Meaning, from J.D. Power and Associates’ and bloggers’ points of view, it’s a hit.
But is it everything it could be? Powersports Business put that question to seven of the 10 manufacturers listed on the ratings, looking for feedback on the survey’s methodology. Those that responded back to us (see the cover story, page 1) shared different views.
One that struck me as something that should be scrutinized is the window of time that new bike buyers have to fill out their surveys. Currently, bike buyers have a nine-month span to respond, meaning they can fill out the survey sometime between the fourth month after they bought their new bike and the 13th month. That is alarmingly long when you consider the psyche of the average consumer.
Most people view their purchase as a good buy. After all, why would they buy it otherwise? And that feeling persists for a certain amount of time before something causes them to think otherwise.
It’s like the green, bathtub-shaped kayak that’s hanging in my garage. For the first few months after buying it, I could think of little else besides its price (a steal at $300!) and its special features (two fishing pole holders!). Ten months later, I can’t walk into my garage without belittling myself with the question, “If it looks like a bathtub, isn’t it going to ride like one?”
So you can imagine what the difference would be if I would have filled out a survey on the kayak in the first few months after I bought it compared to if I received one in the mail last week.
That brings up a justifiable concern for the industry and J.D. Power and Associates officials: For each manufacturer, how many of the surveys represent the newer buyer compared to the older buyer? And is the percentage of old vs. new the same for each manufacturer?
Right now, that can’t be determined. And to me, that’s a concern.
Could it be more useful to the industry if the J.D. Power and Associates survey would narrow the time window significantly so the new bike buyer is polled at six months after their purchase? Or perhaps, two separate studies are needed to better capture both the initial sales satisfaction and the product satisfaction that is better recorded at a period beyond six months?
Now it could be that the current methodology statistically is a wash regarding how many new vs. old surveys you get, like throwing pennies into a hat. So many are going to come up heads and so many tails.
And, J.D. Power and Associates say they aren’t specifically targeting any one group when they mail their surveys, so there certainly doesn’t seem to be any attempt to mislead.
And of course, there is the money issue. How much more would it cost to get this size of a survey (close to 7,000 responded) done if you narrowed the survey response time?
Plus, J.D. Power and Associates pointed out that if they changed the survey — now in its ninth year — that the previous data would mean less to the industry. That manufacturers could no longer in the future put one set of data up against another and have a true reflection of what is improving or decreasing in terms of customer satisfaction.
And that is a valid point. And one made by a company that is so respected by consumers that their work becomes an instant point of conversation in today’s coffee shops.
But all that still can’t lessen the concern I have over the nine-month span that the current survey has. Striving for the greatest possible accuracy, after all, should always bubble to the top. psb
Neil Pascale is editor-in-chief of Powersports Business. Contact him at npascale@ehlertpublishing.com

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button