First you should use some goo gone, because it looks like there's still some adhesive on there.

After that give the area and or whole trunk a nice wax, that should take care of a good amount of it.

From my experience with debadging a few cars, it looks like the rest of the car has sun fade, not necessarily a bad paint job. It should get to a point where it's not noticeable at all.