misplaced subscript and superscripts in IgorTex

What platform (Windows or macOS) and what Igor Pro version?

You might try this ...

\Z16\[0\$WMTEX$ \alpha_0^2 \$/WMTEX$

\Z16\[0\$WMTEX$ \alpha \$/WMTEX$\[1\B0\M\X1\S2

The bracket captures the baseline default. Otherwise, you may resort to one of these other options.

a) Create your own superscript and subscript spacings using other Igor Pro kerning options.

b) Create the LaTeX symbol that you want in an off-line application and copy+paste the result as a graphic (picture) annotation. On macOS, I can recommend LaTeXiT

c) Again on macOS, use the LaTeX to Picture package posted at https://www.wavemetrics.com/node/22092

To be clear, the two packages are LaTeX Pictures (built in to Igor Pro directly) and LaTeX to Picture (a package at https://www.wavemetrics.com/node/22092). The former works on both macOS and Windows. The latter only works on macOS.

Thank you for these suggestions. Still, I prefer to stay vectorial, and the solution I found, though inelegant, consists is typing

\$WMTEX$ \alpha^2 \$/WMTEX$

as one text box, and place by hand the subscript 0 in a second text box. I does the job...

The font that's in effect when $WMTEX$ starts can change the character heights that affect the super and subscripts.

Arial is the usual default font for the experiment and for a graph.

Here's what it looks like on my PC:

I had trouble finding a font combination that looks as wrong as your first situation, but came up with a graph font of Microsoft Uighur that didn't look too great:
 

Here's a procedure file that allows you to adjust the subscript and superscript offsets.

Open it in your experiment and choose Graph->Packages->Show Igor TeX Settings.

Offsets near 0 may not make any difference, keep going until you get around 1.0.