Although I don't work with optical-grade resins often, I do work in plastics and would like point a few things out. I apologize if some of these things seem obvious, but I think it will help some people.
Headlight lenses turn hazy/white/yellow due to oxidation. No need to go into the chemistry, but it's basically plastic's equivalent of metal rusting. There are a number of reasons a plastic might oxidize, but in general, radiation (heat, UV from sunlight, etc) makes the process happen much faster. Light doesn't cause the oxidation per se, but damages the plastic in a way that enables oxidation.
Generally, the OEM's make headlight lenses out of polycarbonate. They generally prefer it over acrylic because polycarbonate is tougher and more difficult to crack/break. Unfortunately, polycarbonate has a much greater tendency to oxidize. Although there are additives to make polycarbonate more resistant to oxidation, these additives also tend to reduce clarity and are not appropriate for an application such as a headlight.
To get around this issue, OEM's generally put a thin UV-resistant layer over the lens. This reduces the amount of radiation that accelerates oxidation in the lens.
Now, let's go back to the "metal rusting" analogy. If you have a rust spot on your car, do you just sandpaper the rust off and call it good? Hopefully not, as the conditions that caused the rust will simply react with the fresh metal and rust again. Ideally, you would clean the rust, then properly paint and coat it to prevent the rust from returning.
You can apply similar logic to your headlight lens. Once the coating is reduced or gone, your lens oxidizes. If you simply clean off the outside of the lens (likely removing any remaining coating at the same time) then you will probably remove the oxidized plastic and have a clear lens again, but the conditions that caused the oxidation will cause it to haze over again.
So what can be done about it? Like others in this thread have mentioned, you can add another protective coating. You will need one that will be hard and resist all the particles that strike the lens during driving. Finding one that lasts as long as the OEMs apply probably isn't going to happen, so you will likely rely on a coating that should be periodically re-applied. The rest of the action will be preventative. Keeping your headlights out of the sun and weather as much as possible will help protect the coating.
