Cutting Data files

Share icon

Wed, 03/20/2013 - 10:43 pm

I am collecting data packaged with multiple spectra within one file. I would like to open the text file, then cut the file into sections that IGOR can read. #S is where IGOR would recognize where the start of a spectrum. I would then like to rename the waves in the first spectrum to start with "S1_", the waves in the second spectrum to start with "S2_" and so on...

I don't know where to start, so I uploaded this representative file.

Thanks.

STosH
 Np_S2PPh2_SK.txt
Start with the idea of scanning through the file line-by-line looking for lines that start with #S, and saving those line numbers in a wave.

Use FReadLine to read each line of the file; use the examples to start off.

Close the file and then use multiple LoadWave commands, one per #S section.

For each section pull the starting line number from the wave and use with LoadWave/L={nameLine, firstLine, numLines, firstColumn, numColumns }

--Jim Prouty
Software Engineer, WaveMetrics, Inc.

March 20, 2013 at 11:38 pm - Permalink

This is a difficult programming task. I will try to write some code to get you started.

What software produced this file? I want to know what name to use for the starter procedures that I will try to write.

Are the files always .txt or is some other extension also used?

Each #L line ends with this:
FLUO_I0  Pindiode  Seconds  Seconds

Why does Seconds appear twice? What names should be used for waves corresponding to these column?

March 21, 2013 at 08:25 am - Permalink

Hi Hrodstein

Thanks for the help.

The data was created by a program called SPEC, it is used to collect synchrotron data.

The wave names are always text.

I don't know what seconds appear twice. That is funny.

There should be 33 waves with the following name. They are listed on line L:
Energy Epoch Ioni1 Ioni2 Ioni3 Absorption Reference FluoW_1 FluoW_2 FluoW_3 FluoW_4 FluoW_5 ICR_1 ICR_2 ICR_3 ICR_4 ICR_5 OCR_1 OCR_2 OCR_3 OCR_4 OCR_5 DEAD_1 DEAD_2 DEAD_3 DEAD_4 DEAD_5 FLUO e-Current FLUO_I0 Pindiode Seconds Seconds

StosH
Thanks for the help

March 22, 2013 at 09:47 pm - Permalink

 

I have attached a "SPEC Loader.ipf" procedure file. It adds a "Load SPEC File" item to the Data->Load Waves menu.

This is a good example of some of the more difficult issues of file loading including:
1. Using FReadLine to parse the file to determine what is where in the file.
2. Loading multiple sections into separate data folders.
3. Loading header information using FReadLine.
4. Creating wave names based on information parsed from the file.
5. Using LoadWave to load a dynamically-determined range of rows.

For each SPEC file a main data folder is created. For each data section in the file a sub-data folder is created.

If you are not familiar with data folders, execute this:

 

DisplayHelpTopic "Data Folders"

 


Please read the comments in the file for details.

You'll note that the section data folders are names Section0, Section1 and so on. I suspect there might be a more informative way to name these data folders. Should they be named after the #S tag. This would give, after cleaning the names up:
S_1_loopscan_2_1_0
S_2_akscan_E_2_472_1_0
S_3_akscan_E_2_472_1_0

But I just noticed that the #S tags are not unique. For example, there are two sections with "#S 3 akscan E 2.472 1.0".

Update 2019-06-04: For Igor Pro 7.00 or later, there is a revamped "SPEC Loader.ipf" at https://www.wavemetrics.com/node/20935.

 
Use this for Igor6 only. For Igor7 or later, see https://www.wavemetrics.com/node/20935

March 23, 2013 at 04:28 am - Permalink

Thanks hrodstein.

I will work through this scirpt, but at the onset, it looks great.

STosH

March 29, 2013 at 07:01 am - Permalink

I posted a revamped version of "SPEC Loader.ipf" to  https://www.wavemetrics.com/node/20935.

Here are the major changes that I made:

1. Use FGetPos instead of FStatus which is very slow. This requires Igor Pro 7.00 or later.

2. Handle names containing a single space such as "Two Theta".

3. Replaced LoadWave with homebrew loader based on FReadLine in order to support comments embedded in data blocks. See See "Standard Data File Format" in spec_man.pdf.

4. Data is now loaded as double precision instead of single precision.

5. Changed data folder names from Section0,Section1,... to Scan1,Scan2...
​

June 4, 2019 at 09:52 am - Permalink