Altitude detriments?

For people training at altitude and racing at sea level, is there a disadvantage on the muscular strength and endurance side? My thought process is that due to less oxygen, the power numbers at the same effort are always lower. I know that people from altitude say that racing with more oxygen is easier, but that is not saying that you’d be as fast as you would if you trained ideally at the race altitude. To keep things simple, let’s not consider live high train low, as that is not an option for many people.