selecting waves in a function doing a linear combination fitting

Hi

I am using a linear combination of multiple contributions (2 to 4) to fit data. They all have the same x scaling. This is an example for a fit using 3 waves:

Function LinearCombination3C(par, y, x) : FitFunc
    Wave par, y, x
    Wave C1, C2, C3
    MultiThread y = par[0] + par[1]*C1(x) + par[2]*C2(x) + par[3]*C3(x)
end

I have two other functions that use 2 or 4 C waves, i.e., LinearCombination2C and LinearCombination4C.

I am then calling these functions in another function that includes a prompt where user can select input data, range, constraints, and C contributions that will be used for the fit. Based on the C waves selected (or not) in the prompt, the function then runs the corresponding "LinearCombinationxC" function shown above.

FuncFit/X=1/H="1000" LinearCombination3C tempcoeff tempinput[lrange,hrange] /D=tempoutput /R=tempresidual /C=T_constraints

What I want to do is be able to tell Igor which contributions to use to run the linear combination, rather than having to create waves called C1 C2 etc. and I can't figure it out. As it is right now, the first contribution I use to run the fit must be named C1, the second C2, etc. So if I want to use C2 and C3 to run the fit, I have to rename those to C1 and C2... How can I define which contribution/coefficient waves to use in the fit?

Thanks

There are two ways to do what you want, one conceptually easier and the other more robust.

1) Use a global string containing a list of standards waves. In your function that calls FuncFit, something morally equivalent to this:

Wave c1, c2, c3  // or whatever
String/G MyStandardsWaves
MyStandardsWaves = ""
MyStandardsWaves += GetWavesDataFolder(c1, 2)  // full path to wave, just in case
MyStandardsWaves += GetWavesDataFolder(c2, 2)  // full path to wave, just in case
MyStandardsWaves += GetWavesDataFolder(c3, 2)  // full path to wave, just in case
FuncFit ...

Then your fit function might be modified like this:

Function LinearCombination3C(par, y, x) : FitFunc
    Wave par, y, x

    SVAR MyStandardsWaves
    Variable nwaves = ItemsInList(MyStandardsWaves)
    Variable i
    y = 0
    for (i = 0; i < nwaves; i++)
        Wave w = $StringFromList(i, MyStandardsWaves)
        MultiThread y += par[i]*w(x)
    endfor
end

2) Use a structure fit function. The standards waves can then be an array of wave references in the fit structure. Read about structure fit functions: DisplayHelpTopic "Structure Fit Functions"

This is harder to implement, should run faster as you don't have to do the look-ups on the global string or the waves, and it is more self-contained- no globals involved.

Global variables are essential sometimes, and should be avoided whenever you can. Because they are persistent, they can lead to hard-to-discover bugs. Sometimes, though, the persistence is exactly what you need, as when you are trying to store the last value of some parameter you used to re-use that value in the next invocation.

In this case, there is another way to do it, but it is difficult. It is a borderline case for a global variable. It used to be the only way to implement a function like this.

In reply to by johnweeks

EDIT: It works! I put the prompt in the function calling the fit function and used global strings. Thanks John!

 

Okay I went through the code and I think I get it. Your code is going through all c waves in the folder and depending on how many there are, it changes the function to add more terms. 

That is faster than what I was doing, but this is not what I want to do. I want to be able to select which wave to use to do the fit. Check this code so you can get an idea of what I want to do:

Function LinearCombo(par,y,x) : FitFunc
    wave par, y, x
    wave c1, c2, c3, c4
    string c1name, c2name, c3name, c4name, waveliststr
   
    Make /O/D/N=5 tempcoeff
   
    waveliststr = Wavelist("*", ";", "DIMS:1")
   
    prompt c1name, "First contribution", popup, waveliststr
    prompt c2name, "Second contribution", popup, waveliststr
    prompt c3name, "Third contribution", popup, "none;"+waveliststr
    prompt c4name, "Fourth contribution", popup, "none;"+waveliststr
    doprompt "linear combination", c1name,c2name,c3name,c4name
    if (V_Flag)
        return -1       // User canceled
    endif
   
    wave c1 = $c1name
    wave c2 = $c2name
    wave c3 = $c3name
    wave c4 = $c4name
   
    if (CmpStr(c3name, "none")==0)
        MultiThread y = par[0] + par[1]*c1(x) + par[2]*c2(x)
    elseif (CmpStr(c4name, "none")==0)
        MultiThread y = par[0] + par[1]*c1(x) + par[2]*c2(x) + par[3]*c3(x)
    else
        MultiThread y = par[0] + par[1]*c1(x) + par[2]*c2(x) + par[3]*c3(x) + par[4]*c4(x)
    endif
end

Issue is, I want to do this for a batch of y waves (I use 2D waves), so I cannot put the prompt in the fitting function. I have to put the prompt in the other function that invokes the fitting function.

 

I guess I could use the wavenames obtained from the prompt function as global variables, and use them in the fitting function?

My code was skeletal, not intended for production. It was only intended to illustrate the idea.

Yes- no prompt can be used in the fitting function. That simply won't work, and if it did, it would drive you crazy answering a bazillion dialogs, several for each fitting iteration. But you can use my version of the fitting function that will fit any number of components. The Prompt statements need to go into the function that calls FuncFit, as you have already figured out.

I just noticed that in my original answer, I forgot the ";" required between list items...

Here is a skeleton of a function to invoke FuncFit with a variable-length list of component waves (and includes the required semicolons 🤭) :

Function DoLinearComboFit()

    string c1name, c2name, c3name, c4name
    String waveliststr = Wavelist("*", ";", "DIMS:1")
    prompt c1name, "First contribution", popup, waveliststr
    prompt c2name, "Second contribution", popup, waveliststr
    prompt c3name, "Third contribution", popup, "none;"+waveliststr
    prompt c4name, "Fourth contribution", popup, "none;"+waveliststr
    doprompt "linear combination", c1name,c2name,c3name,c4name
    if (V_Flag)
        return -1       // User canceled
    endif

    String/G MyStandardsWaves=""
    MyStandardsWaves += c1name + ";"
    MyStandardsWaves += c2name + ";"
    if (CmpStr(c3name, "none") != 0)
        MyStandardsWaves += c3name + ";"
    endif
    if (CmpStr(c4name, "none") != 0)
        MyStandardsWaves += c4name + ";"
    endif
    FuncFit ...
    // code to report results, etc.
end

This one has abandoned my original attempt to make it data folder-proof, but that can be added back with just a bit of extra code.

I have to wonder about two other approaches.

Method 1

Make ONE function that should use all four waves. Use a statement before the function call to set three waves to zero (or NaN) when you want to fit only one component, two waves to zero (or NaN) to fit to only two components, and so forth.

Method 2

Again, make one fit function for all four waves. Use a HOLD string to force the representative coefficients C_j to zero (held) for those components that should be excluded.

In reply to by johnweeks

johnweeks wrote:

That simply won't work, and if it did, it would drive you crazy answering a bazillion dialogs, several for each fitting iteration. But you can use my version of the fitting function that will fit any number of components. The Prompt statements need to go into the function that calls FuncFit, as you have already figured out.

Yes I figured it out the hard way hahaha. Got out of the loop after keeping the enter key pressed for about 2 min

In reply to by jjweimer

Method 1

Make ONE function that should use all four waves. Use a statement before the function call to set three waves to zero (or NaN) when you want to fit only one component, two waves to zero (or NaN) to fit to only two components, and so forth.

If I understand correctly, you are suggesting that components could be eliminated simply by adding a component that is all zero, that is, a component that doesn't add anything.

I think that is guaranteed to cause a singular matrix unless you hold the coefficients that apply to those components.

Method 2

Again, make one fit function for all four waves. Use a HOLD string to force the representative coefficients C_j to zero (held) for those components that should be excluded.

That would work. When it is possible to avoid holding coefficients, I prefer to avoid it. It's really easy to forget to hold them and the result is a singular matrix in this case. A singular matrix error is hard to diagnose.

In reply to by johnweeks

johnweeks wrote:

Method 1

... If I understand correctly, you are suggesting that components could be eliminated simply by adding a component that is all zero, that is, a component that doesn't add anything.

I think that is guaranteed to cause a singular matrix unless you hold the coefficients that apply to those components.

Oh. I'd not considered this.

johnweeks wrote:

Method 2

That would work. When it is possible to avoid holding coefficients, I prefer to avoid it. It's really easy to forget to hold them and the result is a singular matrix in this case. A singular matrix error is hard to diagnose.

I think in review that another issue arises. The OP is pulling the wave names in a prompt dialog. Those names may change from experiment to experiment. In this case, the approach to brute-force build the fit function call first is better. Otherwise, when the component waves are always the same number (1 - 4) and always the same waves regardless of the data set, I'd lean toward using the hold string approach and set coefficients to zero. In the latter case, I would also hard code the wave names in the function and replace the popup dialog with a panel having four checkboxes plus a "Do Fit" button.

I guess this is the software programming equivalent of saying "horses for courses".

I had to look up "horses for courses" :)

Sure- in this case a lot has to do with personal preference. The easy way has drawbacks, the way to get around the drawbacks requires the poor fellow to do a lot of reading in the Igor manual...

You only know how long it will take to develop something after it is finished, and then it's too late. I often have the experience of using something quite often, think that it would be better to take the time to automate it, but I'm only going to do it this once. After doing it ten times, I automate it and then never do it again.