So it turns out it is quite easy to remove the grills once you know the trick. It took me less than 5 minutes to remove both:
1) Remove the top two screws that sit above each of the grills (takes a T25 screwdriver). You will also need to pull out the rubber thing that covers the screws. It slides right now and is not attached with any glue or anything.
2) With the screws removed, take a flat head screwdriver, slip it below the top of the bumper where you just removed the two screws. Because the screws were removed, you should have room to insert the screwdriver. There is room to look under the bumper and you can see all three tabs at the top of the grill that hold it in place. With the flat head screwdriver, unlatch each of the three tabs.
3) With the top three tabs released, you can pull the top of the grill out a little bit. Now you can insert your hand into the grill on both the left and right side of the grill. You should be able to feel the tabs on each side with your hand and unlatch them by pulling the grill out a bit.
4) There are only two tabs left now on the bottom. Using the same method above, put your hand in and you should be able to feel the two tabs on the bottom and pulling the grill out you can release these latches also.
After that, I did some sanding and painting and here they are:
