What determines how quickly you’ll gain muscle is largely genetic, but also how much commitment you have to eating. The training is the relatively easy bit.
I went from 77kg to 131kg in 4.5 years, but it wasn’t healthy. No drugs, but a fairly focused lifestyle, an unhealthy vegan diet (not all vegan diets are unhealthy, but that one was) and weight training that was heavy enough that I still live with niggling injuries from it, 20 years later.
If you look at seasoned, serious lifters, they all have tonnes of injuries, so I’d caution against that. 80kg at 185cm is good for a balance between strength and fitness. It’s certainly not too late to gain more muscle if you want it, but I’d say that you’re in good shape.