Update... Doing my research a bit yielded this tidbit I was missing before.
1. Mix the gyro rate INVERSLY with your throttle.
2. At high speeds, you want little to no correction. (Like a purely anti-wind device)
3. Near or below stall speed you need crazy mad gyro sensitivity and gain.
Still it makes sense what I was seeing now. You can probably set a gyro to auto hover, but you can't fly it like that horizontally or normally.
You really would need to set the sensitivity based on speed / inertia. I can see issues with sensitivity spikes while "coasting" through the air at a high speed...
I can also see fixing those with a curve too. (Zero throttle is 0 gain, and next point is 100 gain at 25%. Then maybe 35 gain at 50% and 5% at 75, with another 0 on the opposite end at 100%.)
Gaaaaaahhh! Just getting more curious...