Depends. Third world country? Maybe not so accurate. 1st world country? Probably has lots of rules and regulations controlling calibration. In my experience, pumps are fairly accurate but variability is mostly from when they choose to auto-shutoff. This ends up coming out in the wash over the long run.
Crawford Eco Block
- Thread starter Canazes9
- Start date
Depends. Third world country? Maybe not so accurate. 1st world country? Probably has lots of rules and regulations controlling calibration. In my experience, pumps are fairly accurate but variability is mostly from when they choose to auto-shutoff. This ends up coming out in the wash over the long run.
One thing for sure, if the temp of the fuel in the ground changes then the volume will change as well. That said, the temp would have to change a lot to see a noticeable volume change for us.
Quirt
I almost only fill up exclusively at costco gas stations but occasionally will have to fill up at a random trashed california gas station (arco or some junk) Costco proudly displays their calibration labels on the pumps but some places just leave me wondering.
This im aware of. Our mornings during the summer swing from 80s to 110s.
So if auto shut off is the variable, then the most scientific way to accurately gauge mpg is:
1. pump at the same time every morning
2. pump at the same temperature
3. pump at the same exact pump at the same station
4. do not overfill and just stop at the first click
5. pump at the same speed with your nozzle
Toffees
Member
#1 rule is large sample size
Too random. Peoples driving behavior and commute vary too much.
Jetboy
New member
The variation in filling up doesn't really matter because the rest of the comparison has no validity. When you look at the data from a typical driver the variation is significant. To identify a small difference in mpg with any level of confidence it would take thousands of tanks of gas. The claimed benefit is small relative to the noise. In effect - it's impossible to test by reporting your fuel mileage. So posting it is meaningless if the goal is to demonstrate a change. If your driving were resulting in small variations about the mean mpg, sure you could do a limited number of additional tanks and have some reasonable level of confidence in a difference. But that's not the case here. When 25% variation is normal, identifying a 3% difference over time is practically impossible to do with a single vehicle. They simply don't last long enough to take enough samples.
The only way to meaningfully test the product is to do so in a controlled environment, or to have a standardized test procedure in the real world - like driving a 100 mile loop - and then doing it enough times that you can get to a statistically significant result with a control group of runs and a group testing the product. If someone actually wants to do this I can help you with the math of sample size and analysis.
The only way to meaningfully test the product is to do so in a controlled environment, or to have a standardized test procedure in the real world - like driving a 100 mile loop - and then doing it enough times that you can get to a statistically significant result with a control group of runs and a group testing the product. If someone actually wants to do this I can help you with the math of sample size and analysis.
Toffees
Member
If you don't know how to group data and use controls, I can't help you.
But it really doesn't matter so much that you control time of day, speed of fueling, etc, as much as you get a large number of samples pre-trick, post-trick, and then after removing trick again for a full comparison. Be your own control group.
#1 rule is large sample size, that's how science works.
Toffees
Member
If the sample size in the lifetime of the vehicle is too small, then why do we ***** and moan about the price of gas? Don't ignore the placebo effect as well, it has legitimate value.
Either you can see something in 10 tanks or it's not there. It only takes 20 tanks to create the comparison. (Not that most people know how to conduct a reasonable experiment, but still...)
*It also helps to know what you're comparing, whether it's mpg or gpm, but that's a different conversation.
A quick estimate is 40F swing is less than 3% change in volume which might be ‘pump error’ since the pumps measure volume not mass.
ok my mistake.. by large sample size i thought you meant a collective sample size such as fuelly a collection of many individuals..
but yes a large sample size of a sole vehicle, sole driver will be more accurate
Toffees
Member
I probably should have been more specific, and I probably wasn't clear in that I was not directing comments to you but rather to those who quote changes in fuel economy after 1 or 2 fuel ups, all of which seem to include weird variables (but i was driving around Aunty Kathy and her dogs get carsick above 55mph so I probably might have been driving more slowly...)
Jetboy
New member
The problem is that the variability in control mpg is so high compared to the signal we're looking for that it takes a huge sample to isolate a change from the noise. 10-20 tanks isn't enough especially if you want any confidence that you're not making a type 2 error, which is kinda the whole point in this case to prove that they are different. Back of the envelope - you'd need somewhere on the order of 250-1,000 samples to identify a 0.5mpg difference depending on the variation in the sample sets.
And the problem is that over that time the condition of the vehicle changes. Even at the low end you'd be driving 75k miles as a control, then another 75k miles again. If you needed 500 samples it would take 300,000 miles. It's simply not realistic to do in normal driving use. A controlled test is the only way to do it practically.
Edit: Front of the Envelope - you'd need 274 tanks of gas to have 95% confidence (.9 power factor) that the two sample means are different by 0.5mpg with a base mpg of 18 and std deviation in both sample sets of 2mpg. Just food for thought. If you had a controlled test drive where the std dev was more like 0.5, then we could get more reasonable numbers - like 15 tanks of each.
And the problem is that over that time the condition of the vehicle changes. Even at the low end you'd be driving 75k miles as a control, then another 75k miles again. If you needed 500 samples it would take 300,000 miles. It's simply not realistic to do in normal driving use. A controlled test is the only way to do it practically.
Edit: Front of the Envelope - you'd need 274 tanks of gas to have 95% confidence (.9 power factor) that the two sample means are different by 0.5mpg with a base mpg of 18 and std deviation in both sample sets of 2mpg. Just food for thought. If you had a controlled test drive where the std dev was more like 0.5, then we could get more reasonable numbers - like 15 tanks of each.
Last edited:
You guys crack me up! :whipping2
I do appreciate the significant scientific approach that several members are applying - but let's get real. I'm the only one providing information on my experience - so to run 274 tanks of fuel to make you feel comfortable in the results is nuts. I'm not even certain that you'd accept the findings then!:bash:
I'm just doing exactly what I said I would do. In the end some people will take that info and run with it while others will call it blasphemous - it's really no big deal either way.
I will run a total of 6 tanks with it on and 6 with it off. That's enough for me to make up my mind - especially if I am driving Aunty Kathy and her dogs around at 55mph because they might get carsick! LOL I love that [MENTION=144062]Toffees[/MENTION]!
Hey [MENTION=121742]Quirt[/MENTION] - you should seriously consider giving [MENTION=118321]Jetboy[/MENTION] a test mule Ecoblock. Members respect his knowledge and experience on this forum, and heck, he already provided the methods he would use. What do you have to loose?
Until the next tank......Cheers!:cheers:
I do appreciate the significant scientific approach that several members are applying - but let's get real. I'm the only one providing information on my experience - so to run 274 tanks of fuel to make you feel comfortable in the results is nuts. I'm not even certain that you'd accept the findings then!:bash:
I'm just doing exactly what I said I would do. In the end some people will take that info and run with it while others will call it blasphemous - it's really no big deal either way.
I will run a total of 6 tanks with it on and 6 with it off. That's enough for me to make up my mind - especially if I am driving Aunty Kathy and her dogs around at 55mph because they might get carsick! LOL I love that [MENTION=144062]Toffees[/MENTION]!
Hey [MENTION=121742]Quirt[/MENTION] - you should seriously consider giving [MENTION=118321]Jetboy[/MENTION] a test mule Ecoblock. Members respect his knowledge and experience on this forum, and heck, he already provided the methods he would use. What do you have to loose?
Until the next tank......Cheers!:cheers:
Toffees
Member
So you're telling me that if I want to save gas, I should just drive slower?
But will my tires rub at full lock?
Also, I've always had a small p-value. It's never hurt my confidence, just my interval.
ElectroBoy
Banned
This discussion cracks me up.
I had a 3rd Gen for 18 years and recorded every single fill up and calculated MPG in a log book. City driving ranged from 15 to 18 MPG; highway was 18 to 22. I never got the exact same result even driving the same route. There are so many variables in real life driving that a tank-to-tank, or even a 1000 tank comparison is meaningless. Unless you are seeing a huge improvement like 4 MPG consistently. The 1 or 1.5 MPG increase that I’ve seen reported is in the noise.
Jet boy is right, you need a controlled, repeatable, standardized test to get meaningful data. On a track in a lab-like environment.
And by the way, why is the 4Runner community taking up this task? It’s BS.
The manufacturer should provide test data to prove the claims.
Jetboy
New member
I'd think it's all but guaranteed that the manufacturer has done this. How could they possibly not have and still claim it does what they claim? The fact that it isn't published anywhere - you can read into that what you wish...Once they knew it didn't work - that's where they crossed from foolishness to fraud.
Last edited:
Toffees
Member
You don't think they can make a claim based on theory, or similar tests on other products or platforms? What is legally limiting them from making specific claims?
Jetboy
New member
No. Selling a product with specific claims should be based on knowledge that those claims are factual. Advertising must truthful, not misleading, and, when appropriate, backed by scientific evidence. If I sell you cancer treatment that I claim cures cancer, but I just blended up grass clippings and cat poop - even if I'm a true believer that it works, it's still consumer fraud to make that claim without some sort of basis in reality. I could say "it might cure cancer, but I haven't tested it".
The legal limit is either the EPA fines for selling an illegal emissions modification device, the FTC for false and/or misleading advertising, a state attorney general or consumer protection agency suing for false or misleading advertising, or a class action law suit from a group of customers who were defrauded by the product. The risk of testing is that when it fails - one or all of the above may follow if any of the groups cares enough about a niche product to take action on it.
Similar threads
