Click here to Skip to main content
15,890,690 members

Welcome to the Lounge

   

For discussing anything related to a software developer's life but is not for programming questions. Got a programming question?

The Lounge is rated Safe For Work. If you're about to post something inappropriate for a shared office environment, then don't post it. No ads, no abuse, and no programming questions. Trolling, (political, climate, religious or whatever) will result in your account being removed.

 
GeneralRe: Excel worries me Pin
Chris Maunder21-Mar-14 3:31
cofounderChris Maunder21-Mar-14 3:31 
GeneralRe: Excel worries me Pin
Mike Hankey20-Mar-14 11:16
mveMike Hankey20-Mar-14 11:16 
GeneralRe: Excel worries me Pin
Mycroft Holmes20-Mar-14 15:01
professionalMycroft Holmes20-Mar-14 15:01 
GeneralRe: Excel worries me Pin
Sander Rossel20-Mar-14 21:49
professionalSander Rossel20-Mar-14 21:49 
GeneralRe: Excel worries me Pin
Richard MacCutchan20-Mar-14 23:08
mveRichard MacCutchan20-Mar-14 23:08 
GeneralRe: Excel worries me Pin
Corporal Agarn21-Mar-14 0:37
professionalCorporal Agarn21-Mar-14 0:37 
AnswerRe: Excel worries me Pin
Techsys Admin21-Mar-14 1:43
Techsys Admin21-Mar-14 1:43 
GeneralRe: Excel worries me Pin
BrainiacV21-Mar-14 4:13
BrainiacV21-Mar-14 4:13 
It's called "round off error", the values may have displayed as 170, but were rounded for the display. And I said round off error, because the calculations did not give a precise answer. You are aware that computers use binary, not decimal for calculations (unless told to), right? Floating point binary numbers do not map directly to floating point decimals. Think 1/3 = 0.333333333... Calculators used to have the problem that 1/3*3 would give you back 0.9999999 because they used a set number of digits. Then we took to keeping two extra digits hidden and then rounding the display. Binary floating point has similar repeating values, just in different places. 0.10 (decimal) is a repeating binary floating point value, so 1/10*10 would not give you 1 without rounding up the result.

Subtracting a "clean" value from a calculated value will generally give you remainders, just look at your result,
Corporal Agarn wrote:
-2.3E-13
That's -0.00000000000023 (if I counted my zeroes correctly) and that's a small value difference.

Close enough for bankers, or should be.

Story from my past...

A program was converted from one machine architecture to another. The difference between the two programs over a $10 million dollar total was $0.35. A meeting was called involving about ten people from upper management and the programming staff. Management demanded a reason for the difference. Explanations of machine difference in numerical processing went over their heads. As the meeting raged on, my buddy (one of the programmers) said he had to overcome an overwhelming urge to stand up and throw thirty-five cents on the conference table and shout, "Problem solved, gentlemen!" The cost of the meeting, even if a solution was found, would not have been recouped for over a century.
Psychosis at 10
Film at 11

Those who do not remember the past, are doomed to repeat it.
Those who do not remember the past, cannot build upon it.

GeneralRe: Excel worries me Pin
Corporal Agarn21-Mar-14 4:19
professionalCorporal Agarn21-Mar-14 4:19 
GeneralRe: Excel worries me Pin
Colborne_Greg21-Mar-14 5:45
Colborne_Greg21-Mar-14 5:45 
GeneralRe: Excel worries me Pin
Colborne_Greg21-Mar-14 5:44
Colborne_Greg21-Mar-14 5:44 
GeneralRe: Excel worries me Pin
jschell21-Mar-14 9:25
jschell21-Mar-14 9:25 
GeneralRe: Excel worries me Pin
Plamen Dragiyski21-Mar-14 12:20
professionalPlamen Dragiyski21-Mar-14 12:20 
GeneralRe: Excel worries me Pin
RafagaX21-Mar-14 12:36
professionalRafagaX21-Mar-14 12:36 
GeneralRe: Excel worries me Pin
obermd21-Mar-14 13:46
obermd21-Mar-14 13:46 
GeneralRe: Excel worries me Pin
John Huff21-Mar-14 21:18
John Huff21-Mar-14 21:18 
GeneralRe: Excel worries me Pin
grralph122-Mar-14 1:05
grralph122-Mar-14 1:05 
GeneralRe: Excel worries me Pin
Daniel Pfeffer24-Mar-14 23:55
professionalDaniel Pfeffer24-Mar-14 23:55 
NewsHappy New YEAR (Nowruz) Pin
Amir Mohammad Nasrollahi20-Mar-14 6:30
Amir Mohammad Nasrollahi20-Mar-14 6:30 
GeneralRe: Happy New YEAR (Nowruz) Pin
OriginalGriff20-Mar-14 7:12
mveOriginalGriff20-Mar-14 7:12 
AnswerRe: Happy New YEAR (Nowruz) Pin
Amir Mohammad Nasrollahi20-Mar-14 8:15
Amir Mohammad Nasrollahi20-Mar-14 8:15 
GeneralRe: Happy New YEAR (Nowruz) Pin
OriginalGriff20-Mar-14 8:59
mveOriginalGriff20-Mar-14 8:59 
GeneralRe: Happy New YEAR (Nowruz) Pin
Amir Mohammad Nasrollahi20-Mar-14 9:55
Amir Mohammad Nasrollahi20-Mar-14 9:55 
GeneralRe: Happy New YEAR (Nowruz) Pin
ravikhoda20-Mar-14 18:50
professionalravikhoda20-Mar-14 18:50 
GeneralRe: Happy New YEAR (Nowruz) Pin
Amir Mohammad Nasrollahi20-Mar-14 20:13
Amir Mohammad Nasrollahi20-Mar-14 20:13 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.