With a shorter lever you need to apply more force to output the same power at the same cadence. So if your body wants to keep your force and power constant, it needs to raise your cadence instead. Also, because the circles your feet have to inscribe are smaller, you require less kinetic energy to make them go around in circles.
I heard that some people do the opposite change, they go for longer crankarms on e. g. time trial or tri bikes to slow their cadence a little.
I’m not saying smaller cranks are more efficient for that reason, just explain why in some athletes you might see a change in cadence.
Same here. I went for shorter cranks, because with 172.5 mm cranks, my thighs were often massaging my stomach when I was in an aero position. When doing the last fit with my old road bike, the owner of my LBS was kind enough to take off a pair of cranks from a bike in the show room that were 165 mm long, and heureka, it instantly felt so much better.