🎉🎉🎉 Introducing AI FTP Detection 🎉🎉🎉

[mailman]

That’s ok I do too. I was just confused about what the point we were debating was.

[Nate_Pearson]

@kosmo886

Will this work with unstructured outdoor workouts overtime if the garmin file links to TrainerRoad?

YES!

[MI-XC]

@ambermalika @Nate_Pearson
As suspected the FTPd was off. On initial rollout it appeared accurate at 317. However after week 3 & 4 of SusPBHV1 the FTPd kept decreasing and this morning it said 313. If it was still 317 I would have probably skipped the Ramp Test.

Maybe 2.3% difference (313 vs 320) is within the acceptable margin, but FTPd as of now seems not quite dialed.

[Nate_Pearson]

@tomclune

I’ve been watching my eFTP every day since the feature came out, which was about a week after a ramp test to start HV SSB I. I have been rating all workouts as medium or hard. (Except recovery z2 rides which are Rated easy. Am also substituting long z2 rides on Sundays.

Ramp test was 229. eFTP was 228 for many days. then was 227 for a couple of days. Then jumped to 237 and is 236 today.

My conclusion is that the AI prediction has a fair bit of jitter and is only weakly correlated with the most recent workouts.

It wasn’t meant to be shown daily. We’re likely going to put a 14-day gate on it so you can only see it once per 14 days.

Here’s the thought right now:
Show you your FTP, and then show that for the next 14 days anytime you click that button whether you accepted it or not. We’d tell you when you can see your new FTP.

[Nate_Pearson]

@MI-XC

According to TR, the goal of eFTP is to match what you would get on a ramp test, not necessarily a “real” FTP. It’s so it can schedule appropriate TrainerRoad workouts, nothing more, nothing less.

The goal is to get you into the right workouts for you, so that you get faster, and crush your enemies.

It’s the combination of Ramp Test + Progression Levels that makes this powerful.

[tomclune]

I understand. Just cannot resist the urge to reverse engineer what the AI has distilled.

[jjmc]

@tomclune

I understand. Just cannot resist the urge to reverse engineer what the AI has distilled.

haha…we need to form an official TR red team for rollouts. I want on…

[killroy123]

I would imagine that Machine Learning makes this almost impossible but IANAS

[tomclune]

Where I would expect AI to be hard to predict will be in outlier scenarios. But consistently following a standard plan should (might) have a simple low dimensional model (with reduced accuracy). E.g. something roughly linear week-to week within a block, but with different slopes for different blocks/people. And maybe noise or asymptotes within a week. Or maybe it would have been mostly flat with the big change happening in recovery week. Admittedly neither seems a good match for the limited data here.

[Odonata7]

I have been checking my FTP detection over my recovery week and seeing it provide 286/287 depending on the day. Given I am starting a specialty block this week I thought I would hit the ramp test today, particularly since it is 10 more weeks before a ramp test is back on my calendar. Seeing this within 1% gives me confidence, disappointing as I thought the AI number might have been low, but it is moving in the right direction so I am happy.

Admittedly, I kind of like to suffer, so look forward to the ramp tests!

[jjmc]

@tomclune

Where I would expect AI to be hard to predict will be in outlier scenarios. But consistently following a standard plan should (might) have a simple low dimensional model (with reduced accuracy). E.g. something roughly linear week-to week within a block, but with different slopes for different blocks/people. And maybe noise or asymptotes within a week. Or maybe it would have been mostly flat with the big change happening in recovery week. Admittedly neither seems a good match for the limited data here.

I’m sure there is coherence in the output, but it’s just on a longer time scale compared to short time windows where noise is created by model over-fitting.

[Nate_Pearson]

@jjmc

I agree. I don’t know if there is any data-driven model anywhere that isn’t going to over-fit to some degree. What you want is that to be a minimal amount of noise that can be smoothed out. If fitness is increasing, you’ll capture that variation only over a longer time scale.

It seems like you really get this stuff, @jjmc.

Overfitting is one of the main issues that we were aware of when we built this. (Overfitting is when our model exactly learns what it was trained on.)

IE, if we just used SSBLV as training data, then it would likely pick up such a big signal from that data set that we’d “overfit” the data and have a model that doesn’t work super well.

We’re really lucky in that we can pull complete histories of training data. So we get someone’s entire career including the time before there was even indoor training, a time when they pause their TR subscription, a time when they do just one custom workout a week and 1 race per week, etc.

But even with all that, we can overfit the data if we’re not careful.

What we really want to see is how the FTPD is doing over 3-6 week periods when folks are having high compliance and consistency with an AT plan. If numbers are dropping off of that, it will be a natural question as to if that is legit or an issue.

Yes, and we don’t even have to wait that long. We can look at the next workout outcomes and survey responses. IE if everyone failed their next workout OR everyone marks their next workout with a lower RPE compared to those who didn’t do AI FTP Detection then we’ve got work to do.

The opposite is true too. If we get a higher pass rate and RPE survey responses that we think are appropriate for the workout’s PLs, then we’ve likely improved things and should push forward.

[Nate_Pearson]

@rocourteau

It tells you what an average rider with your workout history would typically get as a ramp test result. This puts the estimate somewhere near the center of the correlation distribution. You won’t get a different number if you’re a low-tester (or a high one).

This isn’t quite accurate. I wouldn’t say “average rider”.

You do make a good point though. If you’re a high or low tester, you should probably get a more “accurate” result for AI FTP Detection compared to the ramp test.

And by “accurate”, I mean getting you into the appropriate workouts to make you faster.

[TonyLPowell]

Just sharing:

Starting SSBBLV1 today after a couple of rounds of Pol Base and did the FTP detection.

I got nothing.

I mean that literally. 0 watts.

It’s okay though. I was kinda hoping for that anyway.

[tomclune]

Completely agree.

[jjmc]

@Nate_Pearson

Yes, and we don’t even have to wait that long. We can look at the next workout outcomes and survey responses. IE if everyone failed their next workout OR everyone marks their next workout with a lower RPE compared to those who didn’t do AI FTP Detection then we’ve got work to do.

The opposite is true too. If we get a higher pass rate and RPE survey responses that we think are appropriate for the workout’s PLs, then we’ve likely improved things and should push forward.

YOU don’t have to wait that long…but us “red teamers” without insider knowledge have to wait longer!

[rocourteau]

@Nate_Pearson

I wouldn’t say “average rider”.

From the quotes that indicate “personal biometrics like age” are taken into account, I deduce that segmentation along other variables (such as age) is also fed to the model. So indeed not an “average rider”, but an average of those sharing the same biometrics.

I’m just guessing at the contents of a black box.

[Nate_Pearson]

@bobmcstuff

In practice it’s going to be based on the same tech though isn’t it - what the detection does is look at what you’ve been doing and make an estimate based on that, what prediction will do is the same but with the planned future work.

I guess a key difference with the prediction is trying to factor in an amount of non-TR work which may not show up in people’s forward TR calendar (unless they’ve scheduled recurring club rides etc but even then that’s just a TSS number). And will have less data to work with since apparently the FTP detection looks at “a range of data” likely including HR which you obviously won’t have for planned work (only the planned power trace).

You are correct. And the farther you go out the less certain we can be. Another limitation with the current approach is we are assuming you’re going to do the work.

We are going to try to make it so we can figure out how likely you are to actually do the work, which would be like .

So when we first launch AI FTP Prediction we’ll need to tell people to use a bit of constraint. But…what’s also interesting is the model doesn’t always give you a higher FTP if you have more work planned.

[JimA]

@Nate_Pearson

The goal is to get you into the right workouts for you, so that you get faster, and crush your enemies.

I just want to say, I love this!

While the ‘crushing enemies’ part is entertaining, it’s the “get you into the right workouts” that’s money.

[mailman]

This^

This was my thought/personal experience so far.