What intensity are you riding at on these long rides?
If you’re trying to maintain upper z2 for the entire ride you will be spending a significant amount of time in Z3. That leads to exhaustion.
At a point in the ride you will notice that, for the same power, your HR is higher than normal. This is HR decoupling. Pushed to the extreme, your HR will be extremely high for a Z2 effort.
For example - In an upper zone 2 ride I sit at 220W, HR is about 150. At around 2hrs my HR will drift up to, say 155. Once that happens I drop my power until my HR comes back to where I expect it to be. I ignored it once and my HR hit 180 at my zone 2 intensity.
Over time the heart rate decoupling point will come later into the ride. I pushed it from 90mins to 2hrs last base season.
In summary, you can’t expect to maintain the same Z2 for prolonged periods. It may be called the “ride all day” zone, but in reality that’s not entirely possible.