Thank you!
Think it's physics being a bit random and definitely found a bug on the left paddle specifically I'll fix.
Friend showed me you could put animator callbacks in physics to make them better sync with physics but forgot left/rigth paddles were separate prefabs and to slap that fix on left paddle :>