Used to commute 2-3 days a week, 75 miles round trip each time. It wasn’t really hard, but it let to 14 hours away from home, so I think it’s a lot about how you can fit it in. If you can do it as a commute with work, like two hours in the morning, two hours home, you can likely cover the majority of the riding with that (say about 5x60 miles/week) Then a few long weekend rides to top it off.
Don’t worry about your fitness, after the initial shock to the system, it’ll go through the roof and you won’t have issues with long rides.
Good luck!