I had similar thoughts, and have 3 of the factors.. Before getting the third I did some latency tests. I measured 2.25 milliseconds, consistently, from analog in to analog out. I tested this using both the timefactor and the pitchfactor, recording a drum machine click, capturing the input and output with y cords to pro tools, and counting the samples between the waveforms. I did the same test with the TC repeated too, and came up with 1.65 ms, which matches their published spec, and verifies the test validity.
I don't think that 2.25 milliseconds would be a problem, until you got to 4 or so pedals.
As far as the multi conversions go, I have delay pretty much on all the time.. the other factors i use for effects off and on occasionally, and so do relay bypass with them when I'm not using them. So there is no conversion or latency for those pedals unless they are on.
