New parts arent always good parts or another way to say it is, they aren't a for sure thing.

I'd start checking things that can make noise. Remove one of the belts, drive it. Then remove the other and see if the noise is still there. Yes you can drive without the water pump for a few minutes on a cooler motor, it won't harm it. It's possible it's the power steering making the noise or the transmission pump making noise. I'd check both fluid levels while your at it.