Selectively deleting columns from a 2D wave

Share icon

Wed, 09/29/2021 - 04:47 pm

Hi, 

I have a 2D wave where each column corresponds to a molecular formula. Some formulas contain isotopes. Hence, the columns are a mix of non-isotopic and isotopic formulas. I also have a 1D wave with all column labels i.e. all formulas. What I need to do is to eliminate all isotope containing columns from the 2D wave with the exception of 6 isotope columns that I want to keep.

For this purpose, I was able to write a code that eliminates all isotope columns from the 2D wave. The isotope formulas have a "j" in them so I just called "grep/INDX/Q/E="j" my2Dwave" to get the W_Index and deleted them going right to left in the 2D wave. But I just couldn't figure out how to create 6 exceptions in this kill. 

What I tried to do is create a 1D wave called "indexnote" that contains indices of these 6 formulas. Then, under a for loop, I want to use Findvalue/I=W_Index[i], indexnote. If V_Value is 0, delete the column. If it is 1, save it. However, for some reason Findvalue/I=X, the program wants X to be written in integer form. So replacing X with some actual number let's me to compile, but if I use the syntax with W_Index[i], it throws an error. I don't understand why. 

Kindly advise how I could do this operation. 

Thanks a lot! 

Peeyush 

Correction: if V_Value is -1, delete the column, else keep it. But the issue still stands since I can't get FindValue/I=X to accept replacing X as a cell value in a wave i.e. W_Index[i]. Only if I replace X with an actual integer, that the program compiles. 

September 29, 2021 at 04:52 pm - Permalink

This should not happen. As far as I know, only the wave to be searched should be of type integer for FindValue/I. For example, this compiles and executes just fine:

Function Test()
    Make/I/O testvals = round(100*sin(x/8))
    Make/D/O index = {1,2,3}
    FindValue/I=(index[1]) testvals
    Print V_value
End

If you show us the (not too long) relevant code then we can have a look what might be wrong.

By the way, I have the feeling that this problem could be solved without the FindValue since you are using grep. As always with regex, if you know the magic passphrase it will hand you exactly what you want. Unfortunately, the magic passphrase usually involves finding just the right string of ingredients. If you tell us what your exceptions look like or have a small example, then someone might have a suggetions.

September 29, 2021 at 10:08 pm - Permalink

Hi chozo, 

Thanks a lot for your response. Sure, kindly see my code below. The ////////// is where the code is not compiling. The error I get is- Use '/N=number' or '/N=(expression)' (assuming flag 'N'). If multiple dimensions used (i.e., Make/N), use /N=(n1,n2...).  Kindly advise. 

Function killisotopes()
setdatafolder root:PMFinput:
string/G NOfamily             //THIS CONTAINS SPECIFIC ISOTOPIC FORMULAS THAT NEED TO BE SAVED FROM DELETION
wave/t speciesmasstext_out    //THE 1D WAVE THAT CONTAINS ALL COLUMN LABELS OF THE 2D WAVE (ISOTOPES AND NON-ISOTOPES)
variable i,j
variable numlist=itemsinlist(NOfamily)
make/o/n=(numlist) indexnote
indexnote=nan
for(i=0;i<numlist;i+=1)       //BUILDING INDEXNOTE WAVE THAT CONTAINS INDICES OF ISOTOPE COLUMNS TO SAVE (LISTED IN NOfamily)
Findvalue/Text=stringfromlist(i,NOfamily)/TXOP=2 speciesmasstext_out
indexnote[i]=V_Value
endfor

grep/INDX/Q/E="j" speciesmasstext_out   //CREATING W_INDEX WAVE CONTAINING INDICES OF ALL ISOTOPE FORMULA COLUMNS
Wavestats W_Index
wave W_Index
string list=Wavelist("matrix*",";","")   //THESE ARE 2D WAVES FROM WHICH COLUMNS HAVE TO BE DELETED
numlist=itemsinlist(list)
for(j=0;j<numlist;j+=1)
for(i=V_endrow;i>=0;i-=1)   //DELETING COLUMNS FROM RIGHT TO LEFT IN 2D WAVES

Findvalue/I=W_Index[i],indexnote    ////////////////////THIS IS WHERE THE ISSUE EXISTS//////////////////////////
if(V_Value==-1)                     //TRYING TO SAY THAT IF W_INDEX DOESN'T MATCH WITH ANY VALUE IN indexnote, DELETE THE COLUMN
deletepoints/M=1 W_Index[i],1,$stringfromlist(j,list)
endif

endfor
endfor

string list2=Wavelist("species*",";","")  //THESE ARE 1D WAVES FROM WHICH ROWS HAVE TO BE DELETED
variable numlist2=itemsinlist(list2)
for(j=0;j<numlist2;j+=1)
for(i=V_endrow;i>=0;i-=1)   //DELETING ROWS FROM BOTTOM TO TOP in 1D WAVES

Findvalue/I=W_Index[i],indexnote    ////////////////////THIS IS WHERE THE ISSUE EXISTS//////////////////////////
if(V_Value==-1)
deletepoints/M=0 W_Index[i],1,$stringfromlist(j,list2)
endif

endfor
endfor
setdatafolder root:
end  

 

September 30, 2021 at 02:48 am - Permalink

You need to add this after the Grep command:

WAVE W_Index  // Created by Grep/INDX

Next, the error message that I get on the two FindValue commands is "Use '/N=number' or '/N=(expression)' (assuming flag 'N')."

Also, you can not use a comma between flags and main parameters. Use a space instead.

So try this:

Findvalue/I=(W_Index[i]) indexnote

 

September 30, 2021 at 03:51 am - Permalink

Thanks hrodstein! The program compiled after I removed the comma. 

Just one more relevant question. 

Is it possible to apply grep on characters within a string that is in a cell of a textwave? 

So for example, I have a 1D wave where each cell has text that may look something like "NNM_20210101_AB10_Xabc". However this is not a fixed format but each cell text will have "_X" in it wherever it may lie in the string. What I want to do is to find out the location of "_X" part in each cell and then make changes such as deleting all characters after _X, or manipulate X itself etc. 

Could grep give me the location of "_X" within each cell text of the wave? 

September 30, 2021 at 05:54 am - Permalink

First, I have to wonder whether some parts of the for-loops could be streamlined. But, if what you have now works, the added effort may be more trouble.

As to the grep, this might help get you started ...

Static StrConstant gstr = "(.*)_X(.*)"
Function mygrep_OnString(string sstr)
    string fstr, estr  
    SplitString/E=(gstr) sstr, fstr, estr  
    print sstr, fstr, estr
    return 0
end

 

September 30, 2021 at 06:36 am - Permalink

Thanks jjweimer. It'd be great if you could please advise on how I could streamline it further. I want to learn to try to avoid loops as much as possible. 

September 30, 2021 at 09:27 am - Permalink

So using hrodstein's comment, I was able to compile the program.. but it still debugs when I try to run: 

Attached is the error message I am getting. Kindly advise how to fix this. Thanks a lot! 

 

Sincerely, 

Peeyush

 
errormessage.jpg

September 30, 2021 at 09:29 am - Permalink

Here are some thoughts.

Function operateOn_WavesInFolder()

    // define the folder location
    // (this can also be passed via a string to the function call
   
    DFREF theFolder = root:PMFinput
   
    // now, rather than stepping into the folder, just reference it
   
    svar/SDFR=theFolder NOfamily
    wave/SDFR=theFolder/T spmto = speciesmasstext_out
   
    // ...

    return 0
end
Function doOn_AnArray()
   
    SVAR/SDFR=theFolder NOfamily
    wave/SDFR=theFolder/T spmto = speciesmasstext_out
   
    variable numlist = ItemsInList(NOfamily)
    Make/FREE/N=(numlist) indexnote = NaN

    // use implicit indexing on an array
       
    indexnote = set_Match(spmto,NOfamily,p)
   
    // ...
   
    return 0
end

Function set_Match(twave, matchlist, pp)
    wave/T twave
    string matchlist
    variable pp
    FindValue/TEXT=(StringFromList(pp,matchlist))/TXOP=2 twave
    return v_value
end

 

September 30, 2021 at 10:25 am - Permalink

maybe take a look at SplitString:

string s1, s2, s3
splitString/E="(.*)(_X)(.*)" "NNM_20210101_AB10_Xabc", s1, s2, s3
print s1, s2, s3

  NNM_20210101_AB10  _X  abc

Edit: SplitString was already there in jjweimer's response earlier. I didn't notice.

September 30, 2021 at 10:42 am - Permalink

Hi jjweimer, 

Thanks for the ideas and suggestions! 

Few quick things: 

string/SDFR is throwing unknown flag error

stringfromlist(p,NOfamily) is throwing error that p is being used outside of a wave assignment loop

September 30, 2021 at 12:37 pm - Permalink

I finally got the code to work by converting both indexnote and W_Index to text waves via duplicate and num2str commands, and then using Findlevel/TEXT on it. With this, now I can save targeted columns from deletion. For some reason, Findlevel/I= just didn't work with numeric indexnote and W_Index waves

September 30, 2021 at 01:09 pm - Permalink

Apologies. I posted quickly without testing. I've revised it. The first offending error should be SVAR/SDFR not string/SDFR. Otherwise, I've exposed the p index directly in the function call (and this *should* fix the second warning).

September 30, 2021 at 01:40 pm - Permalink

you could compose a regular expression to match with exceptions.

it might look something like this:

// match words other than one or two that contain the letter o
"(^(?=.*o).*)^(?!(?:one|two|three)$)"

Also, you can use multiple expressions with grep:

•make /T foo={"one","two","three","four"}
•grep /E="o"/E={"(^(one|two)$)", 1} foo
  four

use replacestring to generate the regex from your exception list.

September 30, 2021 at 02:29 pm - Permalink

Please have a look at my initial posting. To get FindLevel/I to work you need to provide an integer wave. So you would need to first convert W_index to integer. It seems you have already switched to another way of doing it, but just as a note for future endeavors.

September 30, 2021 at 05:19 pm - Permalink