Multithreading wave read issues
Thu, 09/12/2019 - 02:49 pm
Hello,
I have a genetic algorithm program that works well, but I want to make it more efficient using multithreading and I rewrote the code to include multithreading, but I am running into problems reading waves when using multiple threads. To simplify what is going on I wrote a simpler program to demonstrate the problem. This simple code is supposed to pick two random waves (named "indX" where X is an integer) that exist and write them into temporary waves for further processing. Here is the code:
variable tgID=ThreadGroupCreate(4)
make /o/d/n=500 indA0,indA1,indA2,indA3,indB0,indB1,indB2,indb3 //create temp waves
ThreadStart tgID,0, CrossoverThreadTest(indA0,indB0)
ThreadStart tgID,1, CrossoverThreadTest(indA1,indB1)
ThreadStart tgID,2, CrossoverThreadTest(indA2,indB2)
ThreadStart tgID,3, CrossoverThreadTest(indA3,indB3)
// Wait for all threads to finish before starting another batch of threads
do
Variable threadGroupStatus = ThreadGroupWait(tgID,100)
while (threadGroupStatus != 0)
end
threadsafe function CrossoverThreadTest(indA,indB)
wave indA,indB //temp waves for each thread
string loopstr1,loopstr2
variable a,b
a=floor(20+enoise(20))
b=floor(20+enoise(20))
print a
print b
loopstr1="ind"+num2str(a)
loopstr2="ind"+num2str(b)
Duplicate /o $loopstr1,indA
Duplicate /o $loopstr2,indB
end
when I run "crossoverthreadtest" by itself it functions perfectly. However, when I run it in the multithreading function ("breedtest") I get the error: "While executing Duplicate, the following error occurred: expected wave name".
It seems that when multithreading the program isn't able to see the "indX" waves, but doesn't have any issues when it is single thread. I am assuming this has something to do with how multithreading handles directories and waves but couldn't find anything in the help files. Does anyone know how to make this work?
Also, I am creating and passing separate temp files to each thread to be written too. Is there a more efficient method? Thanks.
Please execute the following command:
DisplayHelpTopic "Thread Data Environment"Note this paragraph:
Your CrossoverThreadTest would only have access to the two waves passed as input parameters.
You might be able to pass a wave reference wave (Make/WAVE ...) to the threadsafe worker function, but as you've written the code even if the Duplicate worked the destination would be the private data folder of the preemptive thread but since you are not using ThreadGroupPutDF and ThreadGroupGetDFR I think you'll just otherwise lose that wave you create.
September 12, 2019 at 03:39 pm - Permalink
In reply to Please execute the following… by aclight
Aclight, Thank you for the information. Based on your comments it sounds like I should use ThreadGroupPutDF and ThreadGroupGetDFR. I have been looking at the Help topics on these and it isn't totally clear how to best implement it for my goal. Can you provide a code snippet showing how I can do this?
September 12, 2019 at 04:34 pm - Permalink
I'm not sure if you saw this topic or not, but it does have an example:
DisplayHelpTopic "Input/Output Queues"I don't have a clear idea about what you're trying to do, since the example you gave doesn't run and doesn't make sense to me. For example, you're not actually using what would be the output waves of the worker function (the result of Duplicate) and your worker thread doesn't actually do any analysis.
I'm not sure that using the input/output queue is actually necessary in your situation.
September 12, 2019 at 04:48 pm - Permalink
In reply to Aclight, Thank you for the… by sleepingawake86
variable tgID=ThreadGroupCreate(4)
string ListA = "indA0;indA1;indA2;indA3", ListB = "indB0;indB1;indB2;indb3"
string loopstr1,loopstr2
variable a,b, i
for(i = 0;i < 4;i += 1)
make /o/d/n=500 $StringFromList(i, ListA),$StringFromList(i, ListB) //create temp waves
a=floor(20+enoise(20))
b=floor(20+enoise(20))
print a
print b
loopstr1="ind"+num2str(a)
loopstr2="ind"+num2str(b)
ThreadStart tgID,i, CrossoverThreadTest($StringFromList(i, ListA),$StringFromList(i, ListB), $loopstr1, $loopstr2)
endfor
// Wait for all threads to finish before starting another batch of threads
do
Variable threadGroupStatus = ThreadGroupWait(tgID,100)
while (threadGroupStatus != 0)
end
threadsafe function CrossoverThreadTest(indA,indB, sourceA, sourceB)
wave indA,indB //temp waves for each thread
wave sourceA, sourceB
Duplicate /o sourceA,indA
Duplicate /o sourceB,indB
end
September 13, 2019 at 02:18 am - Permalink
Here is an example that I believe does pretty much the same thing as your original example, but uses the (arguably?) simpler MultiThread syntax. It also preserves the waves created by the Duplicate operation in the worker function.
Make/O/D/N=40/WAVE indWaves // equivalent to ind0, ind1, etc.
indWaves = NewFreeWave(4, 500) // double precision waves, 500 points
Make/O/D/N=(4,2)/WAVE indABWaves // equivalent to indA0, indA1... in col 0 and indB0, indB1... in col 1
Make/O/FREE/N=(4) junkWave // Only needed to make it possible to use MultiThread instead of the more complicated ThreadGroup stuff
MultiThread junkWave = CrossoverThreadTest(indWaves, indABWaves, p)
// At this point, we don't need junkWave
WaveClear junkWave
// Do something with the waves in indABWaves
end
ThreadSafe function CrossoverThreadTest(sourceWaveRefWave, destWaveRefWave, destRowNumber)
WAVE/WAVE sourceWaveRefWave
WAVE/WAVE destWaveRefWave
Variable destRowNumber
variable a,b
a=floor(20+enoise(20))
b=floor(20+enoise(20))
WAVE loopWave1 = sourceWaveRefWave[a]
WAVE loopWave2 = sourceWaveRefWave[b]
// NOTE: If we don't use the /FREE flag with Duplicate below,
// when this function is called in a preemptive thread the waves
// created by Duplicate are automatically killed when the thread
// returns. But if they are free waves, they aren't killed until
// the last reference to the wave is lost. Since we store the
// wave reference in destWaveRefWave, the wave isn't killed
// when the thread returns.
Duplicate /o/FREE loopWave1, indA
destWaveRefWave[destRowNumber][0] = indA
Duplicate /o/FREE loopWave2, indB
destWaveRefWave[destRowNumber][1] = indB
end
September 13, 2019 at 09:10 am - Permalink