Intel Underestimates Error Bounds by 1.3 quintillion

Do checkout Pentium FDIV bug too

Random ASCII

imageIntel’s manuals for their x86/x64 processor clearly state that the fsin instruction (calculating the trigonometric sine) has a maximum error, in round-to-nearest mode, of one unit in the last place. This is not true. It’s not even close.

The worst-case error for the fsin instruction for small inputs is actually about 1.37 quintillion units in the last place, leaving fewer than four bits correct. For huge inputs it can be much worse, but I’m going to ignore that.

I was shocked when I discovered this. Both the fsin instruction and Intel’s documentation are hugely inaccurate, and the inaccurate documentation has led to poor decisions being made.

The great news is that when I shared an early version of this blog post with Intel they reacted quickly and the documentation is going to get fixed!

View original post 2,252 more words

Advertisements

Published by

gokul

Hi!, My name is Gokula Krishnan, you can (and I prefer) call me gokul. I'm a third year Computer Science major from BITS Pilani. I'm interested in Technology, Theoretical Computer Science and Discrete Mathematics. A FOSS enthusiast, I'm one of the founders of the BITS Firefox community. I'm currently working on Big Data Analytics, Machine learning and UNIX shell programming. My not-so-geeky hobbies include playing volleyball and football and origami

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s