I assume continuing core and upper body gym workouts during recovery week wont interfere with recovery too much.
Nope - keep at it - I have stopped leg weights now I have started SPB but will keep a regime of planks/side planks/ press ups etc 3x week even though I am not in the gym again until after the UK time trialling season. If I stopped core work I couldn’t hold my tt position. You could maybe cut the weight/reps but I’d still get in there - what else are you going to do in your week off? Even when racing I like to up my walking in the summer - burn some calories - chill out - look at the scenery I never notice when cycling or don’t see on the turbo! 