Mon, 07/31/2017 - 05:02 pm
I have a piece of code that has a series of "if" statements, and in half of the instances, a calculated value (lets call it Q) is assigned to a variable (x).
Later in the code as a boolean check, I have the code do "if (x-Q <=0)" or something along those lines. X was assigned the value Q earlier. However, when this operation takes place, abs(q-x) does not equal 0! it usually equals something between -10,10. Q by itself is a value that is something like this 7.5*10^7. So the numbers being subtracted are x(7.5*10^7) - Q(7.5*10^7) and it is not equalling 0. I'm sure it has to do with sig figs and the precision, but, without sharing my code, is there an obvious solution to these rounding weirdities?