Let us examine this shall we? Pulley is .4" smaller diameter, power is lost, and belt is making noise...

And the question is should I use a smaller belt?

I'll just leave that there...