Question about loading delimited text file

Share icon

Mon, 06/08/2015 - 11:43 am

Dear all, Can you help me with the following:

My experiment delivers results in delimited text files, such as the one attached. When I try to load the waves, I can set the column name line and the first data line in the tweaks. All fine, but when I then load the file, every other line is empty. Igor must be triggered by a carriage return somewhere or something? Does anyone have a solution for me?

I am running Igor 6.36 on Windows 7.

Thanks!! example.txt
Each line of the file after the header ends with carriage-return (13), carriage-return (13), linefeed (10).

To see this, execute the following in a new experiment:
GBLoadWave/V/T={72,72}/W=1
Edit wave0


The scroll down to point 361. You can also use Edit->Find to search for 13. You will see that the header lines are OK but the data lines have an extra CR.

It would be best to fix this in the program writing the file. Otherwise you can remove the blank lines using WaveTransform zapNaNs.

June 8, 2015 at 12:24 pm - Permalink

My guess is that the loading routine gets puzzled by the rather uncommon "CRCRLF" sequence in your data file.
Can you change the data saving routine and change it to "CRLF" or just "CR"? Old data files can be conveniently altered by a 'better' text editor like notepad++. If necessary in all files in a directory including sub-directories with a few clicks.
Igors "Load waves >" feature is able to load the data using "Tweaks", but ends up with NaN for every second data point.

HJ

PS: My computer died from excessive parallel computing and I had to restart it. Session was saved but I was late.

June 8, 2015 at 12:52 pm - Permalink

I am new to Igor coding myself, but this seems like it does the job.
#pragma rtGlobals=3     // Use modern global access method and strict wave access.
Function LoadExampleDS()
    variable preallocatesize=2000  //number of rows, will delete extra space after load.
   
    Make/T/O/N=(preallocatesize) root:LoadedData/wave=loadeddata
    Variable refNum
    Open/R refNum as ""             // Display dialog
    if (refNum == 0)
        return -1                       // User canceled
    endif
    Variable lineNumber, len, realline=0
    String buffer
    lineNumber = 0
    do
        FReadLine refNum, buffer
        len = strlen(buffer)
        if (len == 0)
            break                       // No more lines to be read
        endif
       
        if (len>1)
            buffer=replacestring("  ",buffer,";")
            loadeddata[realline]=buffer
            realline+=1            
        endif
       
    lineNumber += 1
    while (1)
    deletepoints/M=0 realline, (preallocatesize-realline), loadeddata

    Close refNum
    return 0

end

June 8, 2015 at 09:58 pm - Permalink