Surely most people’s main goal is not to get faster, though? Getting faster is a usually a subordinate goal to something else - so that you can win a race; complete a ride before the cut-off; beat your friends in town sign sprint; etc.
If your reason for wanting to get faster is so that your outside riding is then more fun, then sacrificing that outdoor riding at the altar of getting faster is a bit self-defeating.