You can indeed, though gains might start to drop rapidly after a few months. According to my copy of “Faster Road Racing”:
These more relaxed periods typically last from three to six weeks, but you may want to schedule a longer period over the winter. … During your recovery period your running should be free of the discipline that is required when you are in training.