The following may assist you (I am a Logic user so I am giving you YouTube resources to view content relevant to Ableton)
Regarding the RME RME UCX ii the following You Tube link will explain how to configure Ableton to send and return to an effect processor like the H9000:
With regard the H9000 the following section of the user manual explains the way to set up Analog inputs and outputs on the H9000 front panel:
2.1.9. Analog References
The Analog Reference Levels screen allows you to set the levels for the eight analog inputs and outputs present on the DB-25 connectors. This allows the H9000 to be installed in a wide variety of environments, and interface at an optimum level with professional and some consumer equipment.</i>
Each pair of analog inputs and outputs on the DB-25 connectors can be separately selected between -10 dBV and +4 dBu.
Use the cursor buttons to highlight the specific pair of inputs or outputs you wish to adjust. Options include:
• Analog In 1/2 • Analog In 5/6 • Analog Out 1/2 • Analog Out 5/6 • Analog In 3/4 • Analog In 7/8 • Analog Out 3/4 • Analog Out 7/8
Once you have highlighted the appropriate pair of inputs or outputs, press the Enter button to switch to the alternate operating level.
Note that the rear panel XLR analog connectors are a duplicate “mult” of channels 1-2 of the eight- channel analog DB-25 connectors. As a result, any reference level changes made to analog input or output 1-2 of the DB-25 connector will also affect the XLR inputs and outputs.
or in Emote
3.3.8. Setup View
Please refer to Section 2.1: Setting up Your H9000 for information about these settings.
Finally in Totalmix you can specify -10db Inst or +4db Line level as the input level for the channel you are feeding in from the H9000. As a rule they should be matched as either -10 or +4db for both the RME RME UCX ii and the H9000
Hope this helps!