Cannot use MatrixOp replace

aclight wrote:

That's because the values in your wave aren't actually 0.03 (except for the ones you manually typed in).

You can see this by showing the wave in a table. Right click on the wave in the table, choose digits, and choose 16. Notice that what had previously displayed as "0.03" now displays as "0.02999999999997272".

Alternately, you cold use print/D to display a value from the wave:

•print/D Timesort[9]
  0.0299999999999727

aclight wrote:

I don't know where your data came from, but you're probably a victim of floating point error (http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html).

To round your data, you could use:

MatrixOp aa = round(TimeSort * 1000)/1000

Substitute different values for 1000 to change the precision of the rounding.

mwpro wrote:

aclight wrote:

I don't know where your data came from, but you're probably a victim of floating point error (http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html).

To round your data, you could use:

MatrixOp aa = round(TimeSort * 1000)/1000

Substitute different values for 1000 to change the precision of the rounding.

Thank you aclight! I think you are right in that I might be a victim in the floating point error. The funky data is after I use

Timesort = mod(Timesort, 1)

on the Timesort wave. What I am trying to do is chop the integer and only keep the digits after the decimal point in the new wave that is attached (this wave_decimal_day looks to be a normal wave).
I also tried

wave_decimal_day = wave_decimal_day - floor(wave_decimal_day)

which gave me the same problematic wave.

Assuming the wave with the floating point error is wave_pro,
I can use

MatrixOp aa = round(wave_pro*1000)/1000

as you have suggested to round to only two decimal points,
or

wave_pro = round(wave_pro*1000)/1000

It seems both can do the job, then my question would be if there is an advantage using one over the other? Or they are essentially doing the same thing?

I came up with

function chop1(todo)
     
    wave todo
     
    variable i
    string a
   
    for (i=0;i<numpnts(todo);i+=1)
             
        sprintf a, "%.2g\r", todo[i]
        todo[i] = str2num(a)
   
    endfor
   
end

to round, but it is extremely slow compared to if a new wave is created,

function chop2(todo)
     
    wave todo
     
    variable i
    string a
   
    duplicate/o todo newwave
    newwave = nan
   
     
    for (i=0;i<numpnts(newwave);i+=1)
           
        sprintf a, "%.2g\r", todo[i]
        newwave[i] = str2num(a)
   
    endfor
   
end

Is there a glitch if I treat the wave_pro using the two chop function? And what would be the reason for the slowness?

mwpro wrote:

What I ultimately want to do with the wave_decimal_day is to replace all the points ending in the same decimal parts with a same data entry in time, so I tried....

aclight wrote:

mwpro wrote:

aclight wrote:

I don't know where your data came from, but you're probably a victim of floating point error (http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html).

To round your data, you could use:

MatrixOp aa = round(TimeSort * 1000)/1000

Substitute different values for 1000 to change the precision of the rounding.

Thank you aclight! I think you are right in that I might be a victim in the floating point error. The funky data is after I use

Timesort = mod(Timesort, 1)

on the Timesort wave. What I am trying to do is chop the integer and only keep the digits after the decimal point in the new wave that is attached (this wave_decimal_day looks to be a normal wave).
I also tried

wave_decimal_day = wave_decimal_day - floor(wave_decimal_day)

which gave me the same problematic wave.

Assuming the wave with the floating point error is wave_pro,
I can use

MatrixOp aa = round(wave_pro*1000)/1000

as you have suggested to round to only two decimal points,
or

wave_pro = round(wave_pro*1000)/1000

It seems both can do the job, then my question would be if there is an advantage using one over the other? Or they are essentially doing the same thing?

I came up with

function chop1(todo)
     
    wave todo
     
    variable i
    string a
   
    for (i=0;i<numpnts(todo);i+=1)
             
        sprintf a, "%.2g\r", todo[i]
        todo[i] = str2num(a)
   
    endfor
   
end

to round, but it is extremely slow compared to if a new wave is created,

function chop2(todo)
     
    wave todo
     
    variable i
    string a
   
    duplicate/o todo newwave
    newwave = nan
   
     
    for (i=0;i<numpnts(newwave);i+=1)
           
        sprintf a, "%.2g\r", todo[i]
        newwave[i] = str2num(a)
   
    endfor
   
end

Is there a glitch if I treat the wave_pro using the two chop function? And what would be the reason for the slowness?

I'm not sure what you're asking.

Your two chop functions execute in about the same amount of time on my machine (a few ms difference).

In general, MatrixOp is much faster than a wave assignment statement which is much faster than a loop (such as your chop functions). There are exceptions to the rule depending on what exactly you are doing.

mwpro wrote:

What I ultimately want to do with the wave_decimal_day is to replace all the points ending in the same decimal parts with a same data entry in time, so I tried....

This might be what you're looking for:

MatrixOp/O bb = round((wave_decimal_day - floor(wave_decimal_day)) * 90000)