When one of the Factor pedals has a problem with the ribbon connector inside the pedal, “jumping around” means random and erratic switching between all algorithms/parameters/etc. From your description, it sounds like the expression pedal you are using is not compatible with the TimeFactor. I had an expression pedal that never allowed the values to “settle” too. It was frustrating. My solution was to use the recommended Ernie Ball Jr. 25k Volume Pedal which works well. For what its worth, I do not think your TimeFactor is broken but you may have to try a new expression pedal. If you have the same problem with an Ernie Ball Jr. 25k Volume Pedal (try one out at a music store if you can) than the ball is back in Eventide’s court so to speak.
