Living in High Altitude and Traveling To Lower Altitude Before "A" Event

I currently live in Breckenridge Colorado and am training for the Breck Epic, a 6-day MTB stage race ranging from 30-50 miles a day. I have a family event in Wisconsin (way lower altitude) the week before the race where I’ll be spending up to a week there. I’m concerned about “deacclimating” during this period before returning to CO for my race? Does anyone know if this is a thing? How long does it take to deacclimate? How many days would you recommend to return before the race starts? Any other advice or experience with this please let me know.

Thanks