VulkanCompute - run GPU compute shaders on your waves

My benchmarks are encouraging, especially when precompiling the shader:

•DemoGPUvs1CoreAndMultiCoreCPU()
  n=200000, iters=2000, threads=32
    GPU        =    366.6 ms
    CPU 1-core =  20815.3 ms  (56.8x slower than GPU)
    CPU N-core =   1074.9 ms  (2.9x slower than GPU, 19.4x faster than 1-core)
    MSE GPU vs CPU = 2.15023e-12,  MSE MT vs 1-core = 0
•DemoGPUPrecompiledVsMultiCoreCPU()
  n=200000, iters=2000, threads=32  (GPU time excludes one-time compile)
    GPU (precompiled) =     12.8 ms
    CPU N-core        =   1030.1 ms  (80.3x slower than GPU)
    MSE GPU vs N-core CPU = 2.15023e-12

This is for an Intel i9-13900K (3.00 GHz) CPU with an RTX 3070 Ti (8 GB) GPU

Even though XOP wasn't written for macOS, I wanted to see if Igor 10 could use this XOP on an M4 with Crossover. It seems to be working:

•DemoGPUvs1CoreAndMultiCoreCPU()
  n=200000, iters=2000, threads=10
    GPU        =     88.5 ms
    CPU 1-core =  33312.0 ms  (376.3x slower than GPU)
    CPU N-core =   4766.7 ms  (53.9x slower than GPU, 7.0x faster than 1-core)
    MSE GPU vs CPU = 2.30319e-12,  MSE MT vs 1-core = 0
    
•DemoGPUPrecompiledVsMultiCoreCPU()
 n=200000, iters=2000, threads=10  (GPU time excludes one-time compile)
   GPU (precompiled) =     10.6 ms
   CPU N-core        =   4695.9 ms  (444.0x slower than GPU)
   MSE GPU vs N-core CPU = 2.30319e-12

On my 4-year-old Windows machine featuring an Intel Core i9-12900K processor and NVIDIA GeForce RTX 3090 graphics card:

•DemoGPUvs1CoreAndMultiCoreCPU()
 n=200000, iters=2000, threads=24
   GPU        =    152.1 ms
   CPU 1-core =  23277.4 ms  (153.1x slower than GPU)
   CPU N-core =   1556.6 ms  (10.2x slower than GPU, 15.0x faster than 1-core)
   MSE GPU vs CPU = 2.15023e-12,  MSE MT vs 1-core = 0

•DemoGPUPrecompiledVsMultiCoreCPU()
 n=200000, iters=2000, threads=24  (GPU time excludes one-time compile)
   GPU (precompiled) =      9.4 ms
   CPU N-core=   1547.1 ms  (164.8x slower than GPU)
   MSE GPU vs N-core CPU = 2.15023e-12

Even after leveraging all 24 CPU threads, the GPU is still over 10× faster. Once the GPU kernel is precompiled, the ratio grows to ~165×, while maintaining essentially identical numerical results.   Keep in mind that most CPU computations ran in DP while the GPU ran in SP.

I also have a Windows ARM laptop with this hardware, so it's running Igor 10 under emulation via Prism.

Snapdragon(R) X 12-core X1E80100 @ 3.40 GHz CPU
Qualcomm(R) Adreno(TM) X1-85 GPU

It nearly choked on the single core CPU stress test, with the GPU coming to the rescue:

•DemoGPUvs1CoreAndMultiCoreCPU()
  n=200000, iters=2000, threads=12
    GPU        =    143.2 ms
    CPU 1-core = 110566.6 ms  (771.9x slower than GPU)
    CPU N-core =   7968.1 ms  (55.6x slower than GPU, 13.9x faster than 1-core)
    MSE GPU vs CPU = 2.16207e-12,  MSE MT vs 1-core = 0
•DemoGPUPrecompiledVsMultiCoreCPU()
  n=200000, iters=2000, threads=12  (GPU time excludes one-time compile)
    GPU (precompiled) =     14.6 ms
    CPU N-core        =   7891.9 ms  (540.9x slower than GPU)
    MSE GPU vs N-core CPU = 2.16207e-12

Your comparison CPU-GPU seems unfair to me: for CPU you are using Igor code, not a C code like for the shader.  

Did you try to make comparisons with OpenCL code (via OpenCL XOP) ?  I often use this XOP so I can compile exactly the same C code for GPU and CPU.  For heavy single-thread calculations on CPU  I installed the Intel's OpenCL framework, where the compiler apparently does a lot of aggressive optimizations (partial loop unwiding for multicore CPU's etc.), I did not study the details but for my apps the OpenCL code is in general a lot faster than any Igor code.   I might prepare a demo/benchmark if anybody is interested.

Hello Pawel,

I agree that the comparison is not perfect. A better comparison would have been an XOP that computes the result in C using a single thread. However, considering that a typical user is more likely to implement the operation using Igor instructions rather than write a custom XOP just for a multiplication, I believe the comparison I provided is still useful.

I did not explore OpenCL because I was looking for a more general and modern solution. OpenCL does have at least one advantage that comes to mind, but it is not supported as broadly. Then again, given NVIDIA's trajectory over the past year or so, one could argue that platform portability may become less of a concern in the future.

AG

Hello AG,

I  was curious so I wrote an OpenCL code equivalent to your Heavy_GLSL function, here is what I obtain on my i7-12700 (12 physical cores):

  Igor loop (your CPUCompute_Heavy function):  24520.4 ms
  Igor multithread (your MTCPUCompute_Heavy function) :  1937.4 ms
 OpenCL (your Heavy_GLSL function) , single thread :  279.2 ms
 OpenCL, 12 threads :  76 ms

I attach my  experiment below.

Interestingly, running the same code on my RTX3060 I obtain :

OpenCL GPU, single thread :  34815.8 ms
 OpenCL GPU, 4000 threads :  12.3 ms

Pawel

I resend my demo pxp file with minor corrections.

Pawel

Hi Pawel,

Thanks for providing these performance numbers.  

It is amazing how much work can be done by 3800 cores.  Although it is getting more difficult (and expensive) to get the high end graphics cards, they clearly outperform the CPUs and most of the built-in graphics hardware available these days.

AG

they clearly outperform the CPUs and most of the built-in graphics hardware available these days.

True for SP,  but in DP the CPU often wins (unless you buy a special compute-grade GPU ?).   On my RTX there is at least a factor of 10 slow down in DP.

FWIW the nightly build of Igor 10 now takes about 7% less time to execute the CPU-1 core part of DemoGPUvs1CoreAndMultiCoreCPU due to some loop optimizations I just merged. For simpler wave assignments (think "wave1=x") the decrease is closer to 40%. The CPU-N core part is also likely a little faster though I haven't measured that configuration and the difference may not be measurable.