Great discussion on how reliability data works and where it can fall short on being useful

Damn, I must be one of the few Ram owners that also has a CR account....lol.

This is a great video and does point to the issues in trying to determine when something is "reliable" or not. And like any comparison, the amount of data and the source of data is so important. Personally, I tend to look at all the different reviews, CR, JD, etc. as my data to decide for myself. It may match CR or it may not.

This also points to the difficulty of reviewers like Tim. The sheer difficulty of gathering "data". Having one of each model you are comparing at the same time, same place, etc. is much simpler but often it is not logistically possible. These long-term reviews I think are extremely helpful as you get a sense of what everyday life is like. Even driving it different seasons gives different results.
 
testerdahl said:
Alex has had a stick up his butt about CR for quite a few years. Every time a reliability report comes out from them or JD Power, he overanalyzes the hell out of it.
Click to expand...

To me CR, JD and the like are all just part of the equation. Knowing where the data comes from and how it's used is a big part of it. Surveying thousands of people and compiling the results is about as fair and equal as you can get. Will they always be truthful? Of course not, but the idea is that an equal amount says the same bad/good things.

My biggest issue with many of these "most/least reliable lists" is not knowing how far apart the numbers are. Say they survey 50K Ford and Chevy owners and the numbers are 1% of Ford owners had an issue vs. 2% of Chevy owners. The actual quantity of trucks that had a problem is small at 500 and 1000, respectively, that your odds of having issues are damn near equal. But to hear that "twice as many Chevys had problems" sounds so much worse.
 
Dusdaddy said:
My biggest issue with many of these "most/least reliable lists" is not knowing how far apart the numbers are. Say they survey 50K Ford and Chevy owners and the numbers are 1% of Ford owners had an issue vs. 2% of Chevy owners. The actual quantity of trucks that had a problem is small at 500 and 1000, respectively, that your odds of having issues are damn near equal. But to hear that "twice as many Chevys had problems" sounds so much worse.
Click to expand...
Great point and I commented exactly the same thing on Alex’s video. If you rank 5 reliable vehicles one of them will be last… BUT THEYRE ALL STILL RELIABLE! Sometimes Tim will share a ranking without any of this context although to his credit I think it’s been quite awhile since it happened. I also get that sometimes those rankings are poor info but good engagement as much as I hate that stuff.
 
mwl001 said:
Great point and I commented exactly the same thing on Alex’s video. If you rank 5 reliable vehicles one of them will be last… BUT THEYRE ALL STILL RELIABLE! Sometimes Tim will share a ranking without any of this context although to his credit I think it’s been quite awhile since it happened. I also get that sometimes those rankings are poor info but good engagement as much as I hate that stuff.
Click to expand...
People want me to declare all new vehicles are junk or a certain brand is junk. If I don't, then obviously I'm biased. If I rank them and I put one on the bottom, them obviously, I'm taking money from the other brands. When I say new vehicles are all pretty reliable these days, I get even more hate. It's all a no-win scenario.
 
I think my point is if you’re (not you specifically) not sharing reliability data, you’re just sharing reliability opinions. And that’s not worth your time or trouble. It’s unfortunate the data out there isn’t great but that’s where we’re at. I just hear how CR is biased from time to time and it just shows how far away we are from understanding this basic stuff.
 
I like to see the results from all of the reliability score lists. But I like getting the background on what the data is based on(and if editor opinion is part of the recommendation). I take all results with a grain of salt.
Unfortunately, we all have our personal biases when we listen to lists and reliability data. We’ve all had bad experiences and good experiences with certain brands. Most of us try to keep an open mind.
Unfortunately, a lot of the commenters seem to want to hate if their manufacturers looks bad and Love the results if their fave manufacturer does well.
I sure don’t buy into the fact that all new trucks are junk and the old ones were flawless. I’ve spent too much time with older trucks breaking and being more difficult to live with than newer trucks.
Keep giving us the information Tim. We will keep giving you our feedback.
 
testerdahl said:
People want me to declare all new vehicles are junk or a certain brand is junk. If I don't, then obviously I'm biased. If I rank them and I put one on the bottom, them obviously, I'm taking money from the other brands. When I say new vehicles are all pretty reliable these days, I get even more hate. It's all a no-win scenario.
Click to expand...

I think it's much more positive and agreeable than not. But truthfully, it's always been that way; it's just now you can actually receive that feedback. This new digital world is the first time that people can directly reply to any statements about anything. Writing an article in a magazine doesn't give one that simple method of replying, "Bullshit! Toyota still hasn't fixed the engine issue!". Hell, people will argue if you make the statement, "The earth is a round globe orbiting the sun."
 
Good video and the first one I've among the journalists covering vehicles that breaks down how CR works, especially with examples. I've had issues with CR for years before I ever got into the truck area. Just always seemed incomplete and biased. The bias also seems to be self fulfilling. It's possibly handy as a single point of data if the person using it understand how it works. Most don't.
 
Fightnfire said:
Good video and the first one I've among the journalists covering vehicles that breaks down how CR works, especially with examples. I've had issues with CR for years before I ever got into the truck area. Just always seemed incomplete and biased. The bias also seems to be self fulfilling. It's possibly handy as a single point of data if the person using it understand how it works. Most don't.
Click to expand...
Help me understand how CR isn’t at least “less biased” than any other source of reliability data. They’re not perfect but the rest of the opinions out there seriously lack real data as the source.
 
It's a stigma that sticks from way back when (the 80's). CR was heavily bolstering their income via donations, so they did some "click bait" video of the Suzuki Samurai which caused their donations to go up. Suzuki sued and CR settled out of court. Thus, they are seen as biased.

Plus they always seem to blast pickups as less reliable than any other type of vehicle out there.
 
Saddle Tramp said:
It's a stigma that sticks from way back when (the 80's). CR was heavily bolstering their income via donations, so they did some "click bait" video of the Suzuki Samurai which caused their donations to go up. Suzuki sued and CR settled out of court. Thus, they are seen as biased.

Plus they always seem to blast pickups as less reliable than any other type of vehicle out there.
Click to expand...

That's basically it, plus it struggles with confirmation bias and the average consumer wants to defend what they have purchased, which is why the more a vehicle sells the more popular it is and the more positive ratings it gets as shown in the video. Plus, they seem to reward the vanilla and hold on to issues long solved by specific manufacturers. They also ignore recalls...
 
Saddle Tramp said:
Plus they always seem to blast pickups as less reliable than any other type of vehicle out there.
Click to expand...
I’ve had some offline conversations with them over this exact issue.

My big complaint is there scoring system and lack of detailed information. On the scoring system, trucks always rank with the a score of 50/100. They really only have one car that gets close to 100. I’m ask them what’s the point of a 100 point scale if only one vehicle comes close?? They also seemingly only like full-size sedans that haven’t been updated in a decade and are about to be discontinued. How is that helpful?

For detailed information, I want to know what engines people are reporting having issues with. They tell me they don’t have that information. I call BS. That would be a requirement for me on any survey I’d send out. I’d want make, model, year, engine, transmission, and then ask questions on a host of areas.
 
I will say from a Publisher/business owner POV, Consumer Reports drives traffic. People want to know what they have to say maybe for no other reason than to argue with them. Anytime I put Consumer Reports in the headline of any article or video, I’m guaranteed views and comments. I find the information interesting even if I have my personal challenges with some of it. And I think any and all reliability information is helpful to some degree.

In my conversations with Alex, I just disagree with him on one thing - the value of predicted reliability. Is it questionable at best? Sure. But, Consumers need something to go off on when making a large investment. They need some sense one way or another their money is going into something they can feel good about.

Plus, predicted reliability information isn’t new or unique to automotive. You can find it in every other industry out there. Sports betting, investment advice, career advice, etc… It is all the same kind of advice just with a different name.
 
You must log in or register to reply here.
Back
Top