That is a hard call to make.
I recently switched from 43s to 47 and am loving the extra cushion on the dirt. In my opinion it would be mostly a matter of comfort, although I think that such narrow tires may be a bit hard to ride on the single track section.