Yeah, you can do it without losing much coolant. But since a lot of them get gunked up, I would suggest pulling the old one and letting the coolant that comes out drain into a drain pan. Then clean out the sensor hole to make sure there is no debris in the way that could prevent the sensor from doing its job correctly (and showing a false light). Then install the new sensor.
Just my advice.