so – I am having a hard time duplicating this behaviour.
I dont use ableton, but setting up with that preset.. the panning follows my tempo perfectly.. no matter what I do with the tempo.
In your project.. is your TEMPO light in sync with the project throughout? Is this actually a clock issue?
Also.. are you monitoring through your computer..maybe there is some latency going on?