Rubbing wheels!

3mm is very tight but 5-6mm should have been fine. Have you checked spoke tension? Maybe that could have been low if the wheels were new and worn in a bit. Riding in dirt or sandy conditions can also be the cause especially with the felt.