Let GG be a reductive complex Lie group (or algebraic group), VV its finite-dimensional complex representation, and HH a vector space of Hermitian forms on VV (HH has an obvious GG-action). By reductiveness there is a projection map H→HGH \to H^G to GG-invariant Hermitian forms on VV. I wonder whether the image of a positive definite Hermitian form is also positive definite.
It is true in compact situation: if GG is a compact real Lie group instead, and VV and HH as before, then the projection map H→HGH \to H^G can be constructed by integrating over GG, so it certainly preserves positive definiteness.
On finite-dimensional VV, the integral cannot converge, because if it did it would give a continuous homomorphism of GG into the corresponding compact unitary group acting on VV.
– paul garrett
Oct 20 at 18:56
@Paul, for complex reductive GG the map H→HGH \to H^G is given algebraically, not by integration — as the projection on invariants, existing by complete reductibility of representations. Or maybe I understand you wrongly?
Oct 20 at 19:08
Ah, well, however the GG-invariant(s) might be obtained, if there is a non-zero GG-invariant hermitian (not symmetric…) positive-definite form on a finite-dimensional complex vector space, then GG maps to the corresponding (compact) unitary group. So I guess the genuine conclusion is that there is no such invariant subspace except in a trivial repn (where GG acts by the identity).
– paul garrett
Oct 20 at 19:20