I replaced the exhaust manifold gaskets while doing the headgaskets in my 04 V6. I didn't have any leaks before, so I didn't bother checking if they were warped.
I'm guessing you could use a straight edge from corner to corner (top left to bottom right, top right to bottom left) and a thickness gauge to see if they have any significant gaps. Might be hard based on the shape and you may have to fully remove the manifolds.
If gaskets are cheap I would do that first. Taking them off is pretty easy, unless your manifold nuts have rust-welded to the studs like the passenger side of mine did. Here is how I would do it *note that there I may be missing some steps since I didn't do it exactly this way*:
* Before buying any parts, examine all 12 exhaust manifold nuts (6 on each side) and figure out if they will actually come off. One side of mine had so much rust that I couldn't get the nuts off. The manifold, stud, and nut were all rusted together and there wasn't enough room in the engine bay to do anything about it.
* If you think you can get them off, start a few days early and spray down all the nuts with penetrating fluid every day. Driving may warm up the manifolds and help the fluid get in, but I've heard that exhaust leaks can also damage the manifold so that may be something to consider.
* Start by removing the manifold nuts. This is pretty straight forward.
* Then unclip the 02 sensors on each side. These are just downstream of the manifolds at the back of the engine and can be hard to reach.
* Find the bracket near the end of the manifold pipe, but before the connection to the intermediate pipe. I think it's connected to the body on both sides. Unbolt the bracket so you can move the manifold off the studs. I don't recall, but you may also have to unbolt the manifold where it meets the intermediate pipe. There is two bolts on each side and a donut gasket on each side that will have to be replaced if you do this part.
* Pull the manifolds off the studs and remove the old gaskets. You may want to try to clean the mating surface on the cylinder head before installing the new gaskets. I would use carb cleaner and a razor blade if necessary.
* Put everything back together.
If you need to fully remove the manifolds you will probably have to remove some things to get them out. On the passenger side you will need to unbolt the trans fluid dipstick (and probably the oil dipstick) at the cylinder head. Worst case, you may also need to remove the power steering pump. On the drivers side you will have to move the wiring harness and maybe the generator or A/C compressor.
Hopefully this helps you.