An ISM really needs 25mm of setback to be set up properly for road riding, IMO.
Is there a lot of weight on your hands? You might need to tilt the saddle up every so slightly from your current position.
Also, the further forward you sit on the saddle, the less stability it provides. I suspect this is your issue, not weight balance. Especially while going downhill. You might want to sit a little further back on the saddle.
On another note, make sure you don’t have the saddle too high. Just a few MM can make a big difference in terms of saddle sores, because of how wide the saddle’s nose is.