A couple things. Well made carbon’s fatigue life should be infinite, I believe. It is robust enough that they make airplanes out of it; those planes are admittedly getting inspected frequently and with equipment not available to the bike industry, but they are carrying a lot of fuel and people and going much faster. So, the material itself is not necessarily fragile. It’s very sturdy. It’s true that you can crash it and damage it in non-obvious ways. The same could be said of aluminum - I was at a cyclocross practice once, and we were sprinting, and one guy’s aluminum frame just tore in half across the downtube. Most likely that was precipitated by a prior crash.
In the unfortunate event that your frame breaks, then you can take heart that actually, carbon fiber is pretty repairable. I don’t mean you should do it in your garage. I mean that a skilled repairer can cut out just the damaged bit, and repair that area, and it will be more or less as strong as the old tube. On a metal bike, you would have to cut out the entire affected tube and weld or braze a new one in. I’m pretty sure that for aluminum, the requirement to heat treat the frame renders this completely uneconomical. For ti and steel, it’s a thing, but it’s not exactly cheap.
One bona fide area of hesitation I have is around the fully integrated cockpits. They look slick and they save you watts. However, servicing the front end of the bike, e.g. replacing the headset bearing or even changing your stack height, is a lot harder and can involve a lot of disassembly and reassembly. For the proprietary stuff, you aren’t guaranteed that replacement parts will be available down the road. I don’t know how the industry is going to evolve on this. It mainly applies to the top end racing models right now, though.
The other thing is that for indoor training, you want a cover to catch your sweat, and you want to wipe your bike down. Sweat will cause galvanic corrosion with any aluminum inserts, and there will always be a few of those. Of course, you’d want to do that to a bike of any material.