Is managing recovery/fatigue just by TSS accurate enough?

Presumably factoring in a duration component, yes?

In fact, you could probably create a Gross Workout Exertion Score by graphing in-the-moment RPE and calculating the area under the curve.