How to get the right division result in C# ?

Question

0.00/5 (No votes)

See more:

Hello Everyone,

I have the following issue:

C#

float increment = 0.00025f;
float sampleFreq = 1.f / increment;
[+] sampleFreq = 3999.99976

and not 4000 why ?

How may I have the 4000 expected result (not via a rounding method) ?

Certainly something stupid.

Thank you very much in advance.
MiQi

What I have tried:

I have tried with double, same result.

Posted 14-Feb-19 1:40am

SuperMiQi

Updated 14-Feb-19 7:32am

PIEBALDconsult

v3

Add a Solution

3 solutions

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Rob Philpott · Accepted Answer · 2019-02-14T02:35:00

It's a bit mathematical, but floats and doubles are floating point types which have a base 2 exponent as opposed to the base 10 we humans are used to. Some numbers can't be represented exactly in this (and also in our base 10 for that matter, think 1/3 = 0.33333333333....) and where that happens you get some accuracy loss.

This is typical of working with floating point numbers, you have to accept a margin of error and be careful with things like equality operators.

You could try using the decimal type in .NET. This uses a base 10 exponent so behaves the way you would expect it too (mostly), but it comes with a cost in terms of footprint and performance.

Dave Kreskowiak · Accepted Answer · 2019-02-14T02:49:00

Because you're using the float data type. It can't represent floating point numbers exactly. Read up on it: here[^] and here[^] .

Think about it. How you do represent an infinite number of values in a finite number of bits? You can't. Sacrifices have to be made to make a data type that can represent a wide range of values that are most commonly used and offer some degree of "accuracy".

You can use the decimal type[^] instead. It's a 128-bit type, made for higher accuracy for things like financial purposes, but has a smaller range of values it can represent.

TheRealSteveJudge · Accepted Answer · 2019-02-14T02:37:00

Solution 2

The precision of float and double is not sufficient.

decimal increment = 0.00025M;
decimal sampleFreq = 1M / increment;

Use decimal instead.

Posted 14-Feb-19 2:37am

TheRealSteveJudge

Comments

SuperMiQi 14-Feb-19 8:46am

Thank you very much that is the solution. I get now the right expected result.

TheRealSteveJudge 14-Feb-19 8:50am

You're welcome. Another one downvoted my solution. :-(

TheRealSteveJudge 14-Feb-19 8:47am

WTF? My solution is right. Thank you for downvoting!

SuperMiQi 14-Feb-19 8:50am

Hm. it was far from my intention, I accepted both comment to thanks everybody.

SuperMiQi 14-Feb-19 8:52am

ah la la, how to fix that.

How to get the right division result in C# ?

3 solutions

Solution 1

Solution 3

Solution 2

Add your solution here

Preview 0