My 97 GTP is doing exactly this and the cooling capacity went majorly down. Previous owner had it charged 15 months ago with dye, and the compressor is green coloured. My AC manifold shows low refrigerant. I know the seals are shot in the compressor now, and as I have two other cars with A/C I'll use one of the other ones on super hot days.
I'd hazard a guess to why this happens - when low on refrigerant cooling only happens on one side of the evaporator. So the cooling effect will feel different side-to-side because the airflow is not split evenly due to turbulence.