The longer the event, the more your body will favor fatty acid oxidation for energy, regardless of whether glucose is available. Fat adaptation is one of the many nebulous “endurance adaptations” that comes from long consecutive hours in the saddle. This generally happens over time with long rides. It’s possible you’re not as developed in this area. (Though usually multiday bike packers have this fairly developed, so your case is a little perplexing to me.) Furthermore, prolonged sympathetic tone will divert blood away from your gut, leading to GI dysfunction and potential malabsorption.
Honestly, I’d perhaps do 1-2 high intensity sessions per week then stack all of your Z2 into one really long endurance ride on the weekend. Then throw in some long Z3/sweet spot intervals at the end of those rides. I did this years ago when I was building my endurance base up for Unbound XL.
Something else I’ve been learning is that FTP myopia can be a distraction for ultra distance. The theory goes that if you raise your FTP, then you’ll raise your Z2 (primary zone for multi day stuff). But I’m beginning to question that logic looking at my historical data. I’m not sure it’s automatically proportional. FTP++ is nice, but don’t neglect your TTE, Z3, and occasional super long rides.
EDIT: Added a link to the study.