Non-negative matrix factorization

In developing the hyperspectral browser project I have been playing a bit with various methods of dimension reduction, including principal components analysis and non-negative matrix factorization. Here is the base algorithm for NMF. For my application this gives interesting results with a randomly seeded input, but the solutions it finds are always local minima. It looks like there are some interesting methods for adding constraints to this algorithm, but I haven't yet been able to figure out any of the more advanced algorithms. If anyone has experience with these, please let me know! 

Here is the simple version. The input is a 3D wave, and the output components describe the layers dimension of the input data.

// the rows of matrix H contain the estimated components
// the output matrix NMFcomponents maps the proportions of components
function NMF(wave w3D, variable components)
	variable rows = DimSize(w3D, 0)
	variable cols = DimSize(w3D, 1)
	variable layers = DimSize(w3D, 2)
	variable pixels = rows * cols
	variable mincycles = 1000, maxcycles = 2000
	
	Duplicate/free w3D w_input
	Redimension/E=1/N=(pixels, layers) w_input
	
	w_input = max(0, w_input)
	
	// rows of W are pixels of w3D, columns are z-dimension of w3D
	Make/O/N=(pixels, components)/Y=(WaveType(w_input)) W
	Make/O/N=(components, layers)/Y=(WaveType(w_input)) H
	W = abs(gnoise(1))
	H = abs(gnoise(1))
	
	int i
	variable err_old = Inf
	variable start = ticks
	
	for (i=0;i<maxcycles;i++)
				
		MatrixOP/O W = W * ((w_input x H^t) / ((W x H) x H^t))
		MatrixOP/O H = H * ((W^t x w_input) / ((W^t x W) x H))
		
		if ((ticks-start)/60>10)
			Make/N=(rows, cols, components)/O NMFcomponents
			NMFcomponents = W[p + q * rows][r]
			DoUpdate
			DoAlert 1, num2str(i) + " iterations completed.\r\rContinue?"
			if (v_flag == 2)
				break
			endif
			start = ticks
		endif
				
		if (i >= mincycles)
			MatrixOP/O err = sqrt(sum(sq(w_input-(W x H))))
			if(err[0][0]/err_old > 0.9995)
				break
			endif
			err_old = err[0][0]
		endif
	endfor
	
	Make/N=(rows, cols, components)/O NMFcomponents
	NMFcomponents = W[p + q * rows][r]
	
	return i
end

 

Hi Tony,

I assume that you are aware of the MatrixFactor operation with /out=2.  Considering that the results are non-unique (subject to constraints), have you tried to compare the results of your algorithm with the built-in operation?

 

AG

I didn't know about the built-in operation. I'm trying initially to reduce data sets of say 1000 rows and 500 columns to just 3 components. The built-in operation, which I am sure is much more robust than my naive two-line matrix multiplication, complains about non-convergence and reseeds the random inputs a few times before giving up.

Looking at the components output from MatrixFactor (factorBmat after MatrixFactor gives up), they are mostly not as nice a solution as the ones I typically get from my test code, and MatrixFactor is much slower. What I mean by nice is how much they look like the components used to create a fake dataset.

I'll explore a bit more by varying the adjustable parameters for MatrixFactor.

Hi Tony,

Can you please email me the examples you tried so I can look into them in more detail.

AG

Forum

Support

Gallery

Igor Pro 10

Learn More

Igor XOP Toolkit

Learn More

Igor NIDAQ Tools MX

Learn More