Been lost in coding details for the last few weeks, but today, I finally gave a try to the first trick you've suggested: interpolation.
Honestly, it's baffling… works incredibly well once you find the right setting, in my case I used a lerp with a t of 0.11.
Wondering if I'm even going to leave it like that and just undo my previous changes because it's such a silver bullet!! ?
