Lori Pearce's WebLogAll Things SDKWhy even computers get the math wrong from time to time- September 26, 2007 There was a nice little blog post on Good Morning Silicon Valley that led to another, really interesting blog post titled "Arithmetic is Hard--To Get Right." Turns out there are "...exactly 12 instances out of the 9 quintillion possibilities it Excel goes completely bonkers" and 77.1 x 850 is one of them. I tried it and did indeed, get 10000 instead of 65535. The original post is here.http://blogs.msdn.com/loripe/archive/2007/09/26/why-even-computers-get-the-math-wrong-from... |