Frame - it is the bit that can’t be upgraded. Thinks to look for:
- Geometry, and specifically does it suit YOU.
- Tire clearance - does it offer enough to clear the tires you might conceivably want to run.
- Manufacturing quality/bottom bracket tolerances. There are some manufacturers out there that are pretty sloppy in this regard - typcially this will show up in regular consumer reports of "creaking bottom brackets.
- Standard components - look at things like seatpost shape/diameter, headset diameter, intergrated handlebars, rear wheel spacing that might restrict you replacing/upgrading components in the future, or lock you in to just that manufacturer.
As others have stated, there is little functional difference between groupsets these days. But better groupset bikes sometimes come with better wheels. Sometimes it makes sense to get a 105 bike and use the money saved to get a good pair of wheels (the stock wheels can become your gravel or winter wheels). At the Ultegra di2 level, they can often come with brand name deep section wheels, which can be good value if that is what you want. At the Ultegra mechanical level, they often come with better but not great wheels, so you can pay more for the bike and still be lusting after some bling carbon hoops. (This will vary between brands).