I had this exact same problem. Just fixed it. Most likely, your vacuum control valve solenoid is bad. It's the small grey box that all the vac lines are hooked to.

Part# is 15-72182 . I got it from AC Direct for $56 shipped.

If you're pulling it off another car, just follow the vac lines. It's screwed into the blower motor housing. You don't need the rubber manifold that the lines go into, just the grey box.