Given the age of the drivetrain (4ish years?), it should be fixable by normal maintenance. It is hard to tell what is wrong, with older, well-used drivetrains the answer is often that everything is worn out. Especially if your cassette and/or chainrings are worn out, once you replace a well-worn chain, it is very common that your shifting will go haywire. To be honest, I’d just replace everything — cassette, chainring, inner and outer cables — and use original Shimano parts.
That happened to me when I got a new-to-me used hardtail with an XTR 9000 drivetrain. I was stupid and only ordered a new chain, and then once I replaced it, my chainring and chain sounded super unhappy. That was fixed by getting a new XTR chainring. Like others have said, you should also be looking at inner and outer cables, etc. And that is assuming that your rear mech is in good condition.
However, That’s less sexy than getting a brand new drivetrain, but unless some parts are damaged or you haven’t kept up with maintenance at all and parts have worn prematurely, you should get it to working order.
Going 12-speed is also an idea. Just keep in mind that you’ll at least need an XD driver-compatible or Microspline-compatible hub, but perhaps a new rear wheel. So things might cascade. I think the previous owner of my bike also boxed himself into a corner like that. The bike had a complete XTR drive train, XTR hubs on nice Stans wheels, etc. You’d be tempted to keep the wheels, but I don’t think the XTR M9000-series hubs are neither Microspline-compatible nor XD Driver-compatible. So either you buy a Rotor 12-speed cassette or get a new wheel.
I don’t think you can go wrong with either Shimano or SRAM. If you like to have a good power meter, I’d go SRAM, no question. They also have two GX cranksets, one made of aluminum and a more fancy carbon version. If you want electronic shifting, go SRAM because you got no choice. Otherwise, it is quite even. With mountain bikes, you can replace brakes and drivetrain separately, so your brake preference does not play a role here, fortunately.