Fixed TTing - can you have too much specificity?

I am starting the preparation for next year’s TT season. I have a dumb trainer and P1 pedals, and I normally use my endurance road bike on the trainer. However, my targets for next year are TT related, and I TT on a fixed gear! So not only is the position different (as expected), but the cranks are significantly shorter (165 vs 172.5) and I have no gears (or freewheel) on the TT bike.
I can probably gear the bike to be able to do intervals around threshold or sweet spot, but I doubt I could hit the power required for many workouts that require varied power outputs just by varying my cadence. The trainer has 10 resistance levels, but they vary from super easy to impossible, so the gaps between levels are huge!
Am I better off leaving the endurance bike on the trainer all winter and just doing fixed TT specific stuff in the outside in spring ?