Fascinating question. I totally 'get' why you would want to set up it up this way. The bad news is: I couldn't manage to figure out how to pull this off, using the aux switches alone.
It'll cost you one MIDI cable and some configuration time. Let me know if you're *not* using MIDI for anything else, and you're open to some setup and experimentation. I've tested the idea here using the PitchFactor and another device, and it seems to work.
The basic concept is to use the Aux switch to *only* control MIDI BPM in the Timefactor. Then, the Modfactor is controlled by the TF MIDI Clock (or not); depending on whether the preset is saved with TMP: ON or TMP: OFF.