The devil is of course in the details, but any time you have a significant delay change, however caused, there will be audible artifacts. Delays are like that.
Crossfades and mutes are among the ways to deal with this, but a discontinuity in the audio is always going to be an issue.
