Fast calculation of linear correlations

Hello, 

I have to calculate linear correlation coefficients for a 2D matrix with large dimensions (e.g. 30000 rows and 2000 or more columns). This can take a while to run and I have a couple such matrices . I am wondering if there is a faster way to run correlations in IGOR. Please advise. 

Best, 

Peeyush

I don't fully understand your problem; when you say "correlation coefficients" of a matrix do you mean covariance?

If so, 

DisplayHelpTopic "MatrixOP"

and look under Functions for covariance(w).

Hi s.r.chinn, 

Yep, I mean Pearson correlation coefficients to check how well the individual columns linearly correlate. Thanks for the tips!

Best,

Peeyush

I have a form of Pearson correlation in my Image Profile tools to align rows or columns. I use Correlate/AUTO for that, which seems to be reasonably fast. I also implemented fitting to determine substep offsets. Here is my code from that tool. in1 and in2 at the two columns to compare (I loop over all columns to compare against the first):

static function SpectrumCorrelationShift(wave in1, wave in2, variable sub)
	Duplicate/Free in2, work
	Correlate/AUTO in1, work
	WaveStats/Q work
	variable error = 1, result = 0
	variable optLoc = sub ? V_maxLoc : (V_maxRowLoc - x2pnt(work, 0))	// if sub-shift is active, return in scaled wave units
	if (V_sum != 0)
		if (V_maxRowLoc >= 3 && sub)
			try
				CurveFit/M=0/Q gauss, work[V_maxRowLoc-3,V_maxRowLoc+3]; AbortOnRTE
				wave W_coef
				if (numtype(W_coef[2]) == 0 && W_coef[2] > optLoc-3 && W_Coef[2] < optLoc+3)
					result = W_coef[2]
					error = 0
				endif
			catch
				if (V_AbortCode == -4)
					variable cfError = GetRTError(1)
				endif
			endtry
		endif
		if (error)
			result = optLoc
		endif
	endif
	return result
end