I pulled the roof rack on my T4R a few years ago, form much the same reason you mention (esp. wind noise at the moonroof caused by the drag). It made a negligible difference, and I wound up putting it back on after a year or so because it does occasionally come in handy.
Sorry, didn't take pics of it, but I'd say it looked slightly better with the rack on. It's pretty easy to remove. I don't know where the holes from the mounting holes lead to (and didn't want to find out the hard way) so I filled all eight with silicone caulk.
Perhaps think about just removing the crossbars, the rails are going to cause minimal drag, and leaving them keeps the appearance about the same.