Yes it's actually the brake booster itself. At the end where the metal arm comes out to the brake peddle. It's not audible because of the foam stuffed in there. Put an air compressor to it and you can tell.

I feel like an idiot, the softer peddle should of been my main hint. But so many things were new to me, GXP brakes all around, smaller booster, SS lines, etc.. Plus I should of pulled the hose off after the car was running and noticed there was so 'suction' left in the booster. Off to the wreckers today.