Time based data spliting

Share icon

Tue, 05/24/2022 - 06:22 am

Hi, all.

I have two 1D waves named "timewave" and "SP2wave", timewave is in Date/time format and SP2 is numeric, and they have the same numbers of rows. And the time range in wave "timewave" is 1 month. Now I would like to split the SP2wave into some small waves(30 minutes per wave) to better manage them. I met problems when I write the loop. I am using "Extract" to split the timeline and get the numbers of rows every 30 minutes and use "Duplicate" to split the SP2wave. But it doesn't work. 

Function splitandsave(wave timewave)
    wave timeTemp,SP2wave
    String DestwaveName
    variable index,maxindex,n
    maxindex = numpnts(timewave)
   
    for(index=1;index<maxindex;index += numpnts(timeTemp))
       Extract/O timewave,timeTemp,timewave>=timewave[index] && timewave< (timewave[index] + 60*30)
       Duplicate/FREE/R = [index, index + numpnts(timeTemp)] SP2wave, DestWave
       //Save/J DestWave as DestwaveName
       //DestWavename = "BT_"+secs2date(timewave[index],-2)+" "+replacestring(":","sec2Time(timewave[index],3)","")
    endfor
end

I'm new to Igor and new to programming. Any help would be greatly appreciated.
function splitandsave(wave timewave, wave SP2wave)
    string DestwaveName = ""
    variable startTime = timewave[0], endTime = timewave[Inf]
    variable i = 0, halfhour=30*60
   
    for(startTime=timewave[0],i=0;startTime<endTime;startTime+=halfhour,i++)
       sprintf DestwaveName, "timewave%d", i
       Extract/O timewave, $DestwaveName, timewave>=starttime && timewave<(starttime+halfhour)
       sprintf DestwaveName, "SP2wave%d", i
       Extract/O SP2wave, $DestwaveName, timewave[p]>=starttime && timewave[p]<(starttime+halfhour)
    endfor
end

 

May 24, 2022 at 06:59 am - Permalink

Thank you very much, Tony! And I have a new problem now. I need to save these waves in .txt format and in different folders. Each folder has 1 file. I generated some codes before you reply to me and save the waves. But I still don't know how to put in in different folders.

The followings are my code. If you find somewhere incorrect, please let me know.

Function splitandsave(wave timewave)

    wave timeTemp,incand_con,incand_mass_con,sonic_5Hz_data_T,sonic_5Hz_data_U, sonic_5Hz_data_V,sonic_5Hz_data_W
    String DestwaveName = "",SP2_PATH, headings
    variable index,maxindex
    maxindex = numpnts(timewave)
   
    for(index=0;index<maxindex;index += numpnts(timeTemp))
       Extract/O timewave,timeTemp,timewave>=timewave[index] && timewave< (timewave[index] + 60*30)
       Duplicate/R = [index, index + numpnts(timeTemp)] incand_con, DestWave
       Duplicate/R = [index, index + numpnts(timeTemp)] sonic_5Hz_data_U, DestWave1
       Duplicate/R = [index, index + numpnts(timeTemp)] sonic_5Hz_data_V, DestWave2
       Duplicate/R = [index, index + numpnts(timeTemp)] sonic_5Hz_data_W, DestWave3
       Duplicate/R = [index, index + numpnts(timeTemp)] sonic_5Hz_data_T, DestWave4
       Duplicate/R = [index, index + numpnts(timeTemp)] incand_mass_con, DestWave5
       headings = "incand_con"
       Newpath /O SP2_PATH, "C:\Zixuanphd\Osca2022\BT tower winter flux\Raw data"
       DestWavename = "BT_"+secs2date(timeTemp[index],-2)+""+replacestring(":",secs2Time(timewave[index],2),"")+ ".txt"
       Save/J/P = SP2_PATH/A=2/W DestWave,DestWave1,DestWave2,DestWave3,DestWave4,DestWave5 as DestwaveName    
       
       killwaves DestWave,DestWave1,DestWave2,DestWave3,DestWave4,DestWave5
       killpath SP2_PATH
    endfor
end

Regards,

mortoo

May 24, 2022 at 10:49 am - Permalink

You need to create new folder for each data set, something like:

Newpath /O SP2_PATH, $("C:\Zixuanphd\Osca2022\BT tower winter flux\Raw data\DataSet"+num2str(index))

This will create new folder on your disk for each data set called DataSetX, where X is value of index converted to string. This will then save each txt file in its own folder on drive. 

May 25, 2022 at 12:04 am - Permalink