Hello,
Whether or not the latency is an issue is mostly up to the player and if it bothers them. If you say that you are not sensitive to it, then my best suggestion is to try it out and see if you like it.
Typically, pitch based effects are placed pre distortion as that will ensure the tracking is working best, but there is nothing wrong with using them post distortion if that works for you.
You seem to have a good grasp of your options and how everything could potentially work, so I would highly suggest doing some experimenting on your own and figuring out what works for your setup.
