If you don’t have the confounding variables available to assess or add to your statistical model then your results may still be affected by them regardless of sample size.
This is why you often hear people say that correlation doesn’t equal causation.