The voice coil is still dealing with back pressure from the box since the cone is still attached to it's air space/receiving back pressure. So it really isn't like dealing with IB in any way shape or form.
Inverting the speaker is changing the phase 180 degrees WITHOUT adding any "delay". If you flip polarity on the sub then you change the signal 180 degrees AND delay by one wave length. It's easier to demonstrate than talk about.