I’ve had the Fenix 6x for a long time now, which replaced my 5 (which I sold to a swim buddy), which replaced my 4 (from which I extracted tremendous value). In other words, I’ve been using the Garmin watch for a while.
Here’s my take: if you are running, particularly distances and especially trails and ultras, the Garmin Fenix or the Suunto are your go-to watches. If you’re are running “only” marathon and below, the Garmin Forerunner fits your need.
If you are running and swimming, whether pool or open water, the Suunto is out (my knowledge of the Suunto is admittedly dated, so apologies if I’m wrong here, but you’re not considering Suunto anyway and you shouldn’t, I just included it here for comparison, like the Forerunner), but the Forerunner is in (I think). You can load swim workouts onto the Fenix, which is actually pretty awesome as it counts your lengths and tells you when you’re on your last length (especially helpful when you’re doing 500s, 800s, or 1000s… but for me also helpful when I’m doing 200s).
If you’re cycling and using the watch, then you’ll “suffer” from using the small screen that is the watch, but that may be ok for you, especially if you’re only interested in say two fields, ok to scroll with small buttons, but mostly it’s for recording. But you said that’s not the purpose of the device, so we don’t care. The Garmin ecosystem for a long time now integrates your Garmin head unit data with your Garmin wearable, which is nice when you’re looking at the body batter, work load, etc. (Personally, I find the Body Battery to be weirdly accurate.)
If you’re doing tris, then the Fenix or the forerunner (?) are great but I believe the Fenix provides more features than the Forerunner, but check that.
Ultimately the comparison to the Fitbit is what you want to use the device for. The Fenix does great mapping for running (road or trail), which I’ve used often. The former when I’m in a new city, typically for work, and want to get out for a run and yet make it back to my hotel, and the latter typically when I’m on holiday and exploring a new place and either want to know where I’m going or at least know where I’ve been and how to get out or for an ultra. I don’t think the Fitbit has the swim function covered and certainly not as well as the Fenix. Pre-covid I was swimming 3xweek for 2500-5000m each session and completely reliant on my Fenix to execute a workout and know where I was in the workout (it also detects strokes, which is cool). If you’re not swimming, then I’d suggest looking at the Forerunner. If you just want the (useless) data for daily life (ie not swimming, not cycling, not running, not the gym, not skiing, not rowing, all of which the Fenix tracks plus loads more), then the Fitbit might be for you.
I hope that helps.