Rest weeks and recovery rides as a concept in general are something that as a relative noob I have struggled with.
The benificial adaptations we train for occur only during rest and obviously the more training or harder the workout the more rest you will need to let your body and mind repair.
I’m probably wrong but recovery rides to me, at least most of the time, seem silly. If it’s rest your body needs then rest - simple. You’re not 100% resting or even close to it if you’re pedalling a bike in any way. Go spend that time with your family or read a book or something.
As for rest weeks within plans, yes I have found them to be a struggle every time. Recently I hit my scheduled rest week during a sustained power build plan. I was loving the plan at that point and I was improving every day. I hit rest week and while I didn’t stick exactly to the plan I cycled super easy all week with maybe one sweet spot workout in the middle of the week to keep my sanity. The following week getting back into the hard workouts of the plan I felt like I was hitting a brick wall. One week I was buzzing with energy and purpose, the next I was demoralised.
I’ll never religiously adhere to plan-scheduled recovery weeks ever again, instead I’ll keep doing what I’m doing until my own body, or mind, or performance tells me to stop.