I sometimes use both good earplugs, as well as over the ear muffs. This gets things much quieter than just earplugs, but you have to be OK sleeping on your back. I never go the whole night like this, but it gets me to sleep when the room is too loud otherwise.
You can get ‘slim’ earmuffs that work pretty well, but they don’t get quite the NRR that full size ones do - it’s a comfort/NRR tradeoff.
Edit: You mentioned being a side sleeper - one thing I tried for a bit was to take one earmuff off so I could lie on my side. I have ended up getting used to falling asleep on my back, so I don’t really do that anymore. I think the Harbor Freight earmuffs that I had I could take one side off.