Home Page for the TeradataForum
 
 

 

Archives of the TeradataForum





Message Posted: Mon, 05 May 2003 @ 19:21:22 GMT





     
  <Prev Next>   <<First <Prev Next> Last>>  




Subj:   Re: Why it is rounding the last digit?
 
From:   Chuck McDevitt



IEEE 64-bit floats (8 bytes) have 15-17 decimal digits of accuracy.

IEEE 32-bit floats (4 bytes) only have around 5-6 digits of accuracy.





     
  <Prev Next>   <<First <Prev Next> Last>>  
 
 
 
 
 
 
 
 
  
 
  Top Home Join Privacy Feedback  
 
 
Copyright for the TeradataForum (TDATA-L), Manta BlueSky 
Last Modified: 30 Jun 2008