You should not have to compensate with the program output levels too much.
Are you using the amps effects loop with I/O 3? If that is the case, then yes it should be line level.
The best way you can troubleshoot this is by loading to “THRU” presets to get a clearer idea of where the signal is getting louder. The H90 should not be changing your signal level at all with 2 thru algorithms and all of the gains set at 0db, but you should have Program mix at 100% for this test.
