Rocking a 1x with a 48t and 11-32 cassette in Colorado. Perfect for flattish racing and the foothills / reservoirs in the Front Range. Not ideal for climbing. Usually after a hour of climbing at 60rpm or so, I’m cooked.
It’s possible to get up the bigger passes this way, but it’s not fast. See State Bicycle’s ‘Climbing up a Mountain on a Fixie with a Pro’ to imagine how that looks. I often turn down group rides in the mountains because it’s just too slow. Really need to go up them at my own pace, so that stinks sometimes.
Philly area has hills for sure. I’d keep the 2x. Also, it can be better from a drivetrain efficiency side for racing. You definitely would want a 2x in Manayunk 
. In South Jersey, I would have no qualms about ditching the 2x and would probably be rocking a 50t or 52t chainring.