I think you and a few others are way overemphasizing the impact of tire size on handling of a properly designed gravel bike.
A gravel bike is designed to be ridden on gravel roads 99%+ of the time. Single track is an edge case and if you are going to be riding singletrack, other than on the smoothest of smooth single track, a gravel bike is the wrong tool for the job (I know from experience riding my Diverge; singletrack is painful on that bike with any sized tire).
Similarly, on technical rocky gravel, this is an edge case where you are riding so slow that +/- 5mm of tire size does not matter when it comes to handling. The bike is just not suited to it and any tire size is going to be painful. If anything, bigger tires help because they can absorb the rocks better.
Or maybe I’m just not sensitive to these small changes. If I put 47mm tires on my Diverge, I’d be able to ride everything I do today just as well as I do with 38s.