Return Input Number to Arbitrary Precision

Two functions to take an input number and return it with a reduced precision. First function does this "by hand" second uses APMath function. Motivation for this function was to enable getting a simple increment for a set variable control from an arbitrary input, which is unlikely to be an integer. Note that precision follows APMath convention that precision is number of decimal places rather than number of digits.



 Function ChangePrecision(vNum, vPrecision, vPreserveSign)
	Variable vNum		//input number to operate on
	Variable vPrecision	//output precision (number of decimal places)
	Variable vPreserveSign	//1 == retain sign of input
	
	Variable vReturn
	Variable vExponent
	Variable vMantissa
	Variable vPwr
	Variable vSign
	
//this will cause an error with the log fcn
	if(vNum == 0)
		return 0
	endif
	
//precision less than 0 makes no sense
	if(vPrecision < 0)
		return vNum
	endif
	
//store sign of input and ensure input is positive 
	vSign = sign(vNum)
	vNum = abs(vNum)

//get mantissa of number	
	vExponent = floor(log(vNum))
	vPwr =  (1 *10^vExponent)
	vMantissa = vNum / vPwr

//change mantissa to desired precision	
	vMantissa = (round(vMantissa * 10^vPrecision)) / 10^vPrecision
	vReturn = (vMantissa) * vPwr

//if requested, change to original sign
	if(vPreserveSign == 1)
		vReturn *= vSign
	endif

//	Print log(vNum),vExponent, vmantissa, vPwr, vSign
	
	return vReturn
End

Function ArbPrec(vNum, vPrecision, vPreserveSign)
	Variable vNum		//input number to operate on
	Variable vPrecision	//output precision (number of decimal places)
	Variable vPreserveSign	//1 == retain sign of input
	
	Variable vSign
	Variable vOutput
	String sNum
	
//precision less than 0 makes no sense
	if(vPrecision < 0)
		return vNum
	endif
	
//store sign of input and ensure input is positive 
	vSign = sign(vNum)
	vNum *= vSign //same as abs(vNum)
	
//change to desired precision	
	APMath/N=(vPrecision) sNum = vNum
	vOutput = str2num(sNum)
	
//if requested, change to original sign
	if(vPreserveSign == 1)
		vOutput *= vSign
	endif
	
	return vOutput
end

Forum

Support

Gallery

Igor Pro 10

Learn More

Igor XOP Toolkit

Learn More

Igor NIDAQ Tools MX

Learn More