2005 Pontiac Grand Prix: Looking for an opinion on the problem

About a month ago, I noticed that my temperature gauge was going higher than normal when I was at a stop light or going through a drive-thru. I checked my antifreeze and it was a little low so I topped it off. The problem went away for about 2 weeks and then it start to creep up again while at a stop. Again, I checked the antifreeze and was a little low. I added more (about 1 quart) and again the temperature stayed normal for about a week. I checked for leaks and did not find any. I had to add about another quart and it was fine for about 4 days and started the same cycle of the temperature going up at a stop. I checked the radiator fans at this point (with it just filled) and noticed that the fan on the drivers side was running when it was supposed to (with AC on or when temperature rose) but the fan on the passenger side came on for a short time and then went off. It looks to me from looking at the manual that both fans are supposed to be either off, on slow or on fast. I found that there are 3 fan relays. I pulled them out and checked them with a multimeter and all seemed to test okay. I also looked at oil and it doesn't appear to have antifreeze in it. I see no white smoke coming out of tailpipe and there is no sweet smell that I can smell.

Any advice on what would cause this would be appreciated. I was going to change all 3 relays out but not sure if that makes sense or what the problem would be causing the fan to not stay on and also the loss of antifreeze. Any suggestions would be appreciated.