Looking to get something that will perform great, but that will not break the bank. Something that is easy enough to toss in a small case or the pocket. I want the camera to have decent low light ability so I want something with a faster lens, would rather not have to rely on the higher ios. Doing a quick bit of looking I am leaning towards the Canon G15, but I'm still open for suggestion. I would like to keep the cost to $500 or less (unless one is just a bit more money and what you gain is really worth it). I do not care about a swivel LCD screen.
Ok, i have more time now to give a better answer (i think). The biggest obstacle for you (or anyone) is...can you part way with $650 for the Sony point & shoot (RX100)?? If not, then this post is irrelevant. Sony RX100 is damn expensive...and one of the main reason why I hesitated in buying it. Lets be honest...$650 for a P&S borders on being ridiculous. No horseshoe. No real viewfinder.
Lets talk about the Canon G15. According to DxOMark, it is about the same level as Canon S110. Here is their conclusion:
"This is a good result overall and when considering the G15’s other features such as the flexibility of the new high-speed zoom and reduction in size over previous iterations it’s a extremely well-balanced and attractive package. For consumers the choice remains whether to spend more on an equivalent micro four-thirds camera with a number of lenses to match the same kind of flexibility or whether to restrict that versatility somewhat by selecting a more compact model such as the S110. Alternatively,
you could consider the G1X or Sony RX100, both offer superior low–light performance with only a small premium to pay in both size and price."
Frankly, if you want Canon, then either get S110 or G1X. G1X has a huge sensor, but very slow lens and uber expensive. S110 is compact and gives good picture quality, on par with G15. In addition, S110 is reasonably priced.
What the Sony RX100 has that is unique is damn good picture quality (on par with G1X) and equally portable as S110. Picture quality should easily beat G15 under similar conditions.
Now, the only real P&S that has a chance against the RX100 is the Panasonic LX7, which is probably
YOUR best choice if you want to stay below $500. LX7 has a small sensor (same as G15), but its lens (Leica) is damn amazing...allowing it to almost match RX100 in IQ. That is very impressive given the small sensor. Leica also makes a clone of the LX7 but price it much higher (for the name). The LX7 is surprisingly slightly bigger than RX100.
LX7 DPreview:
http://www.dpreview.com/reviews/panasonic-lumix-dmc-lx7/
LX7 Cameralabs review:
Panasonic LX7 review | Cameralabs
Very good camera!! RX100 and LX7 were my finalist...very close. I chose the RX100 simply because it is slightly more compact and big sensor. And in general, with most professional reviews, the RX100 came out slightly on top in terms of IQ. But, RX100 is also around $200 more expensive (market value).

And yeah, the F1.8 Carl Zeiss® Vario-Sonnar T* lens are simply amazing!! (Note: Sony also has Carl Zeiss lens on their cheaper P&Ss, but they are different class [Vario-Tessar].)
IMHO, i would dump the Canon G15 and get LX7, unless you really need zoom...similar price, same sensor size, and much better lens on the LX7 (by renowned Leica). But, if you're willing to spend $650, then i would get RX100, bar none.
Hope this helps.
EDIT: from 1st page of this thread is a comparo by DPreview:
http://www.dpreview.com/articles/2367736880/roundup-enthusiast-zoom-compact-cameras
This should give you most information that you need.