Adaptive Training -- How long does a beta usually last?

It seems to be taking a long time to work out the kinks, no?

2 Likes

Feels more like an alpha, as features are changing. A public beta is considered by all my software friends to be a release candidate that would benefit from more widespread testing.

Hoping the final feature list is getting closer, and it leaves beta. But not a good idea to rush it to market.

3 Likes

Based on experience with Machine Learning at work, I would expect AT to remain in beta for significantly longer. TR is trying to solve several problems at one time, and the way they’ve structured the use of AT has - in my opinion only - made their life harder. That is, if TR had constrained the initial use of AT to just people who are only:

  • Using plan builder
  • Doing all rides indoors on the trainer via TR app

Then I think TR would have made faster progress on the base use case - AT being able to adapt plans in a way that improves performance. Now the whole definition of how you measure (or even what to measure) for “improve performance” is a whole other kettle of fish.

But the way I see AT being used is very unconstrained:

  • Mix of indoor / outdoor TR workouts
  • Mix of structured workouts / unstructured rides (or at least not structured in a way that is easy for AT to identify)
  • Mix of TR created workouts with “approved” rating & workout creator workouts with dubious / inconsistent rating

Throwing all of these into the mix has, again in my opinion, created a lot of noise that is going to make it much more difficult for the TR AT team to figure out if the way that AT is learning even solves the problem they are trying to solve: use AT as a way to adapt TR Plan Builder plans in a way to make them more effective (again, how is this defined) for the N=1 case

Edited to correct my bad typing

10 Likes

I could be wrong as I am definitely not TR staff, but to me the only thing holding it back in Beta is the inability to process unstructured outside rides (even the structured ones I think rely on your survey input).

Beta definitions shift greatly depending on project and industry. One (relatively) famous example is the Gmail open beta. I think it lasted something like 5 years and had many millions of users before it finally left beta

In other places betas are as described by @bbarrera - a near-final build that is really looking for stress testing and painting the final corners of functionality

All that to say - I wouldn’t be surprised if AT is in ‘beta’ for at least another 3 months, possibly 6. Total guess, could be changed tomorrow

Ultimately, whether or not it is deemed beta is relatively meaningless from a user perspective at this point. That may change if they decide to roll back functionality, but for most users, and the most likely path forward for the application - beta or officially released is irrelevant

4 Likes

I would also add being able to adapt it to the triathlon plans, which I’m sure it’s a relevant part of their userbase. Right now, it would swap my brick runs (designed to be 15-20 min long) for much longer runs, as it doesn’t understand what a brick session is. Similar for the rides also, when I’m supposed to ride continuously at tempo, it would swap it for workouts with rest periods inbetween.

3 Likes

There’s still a long way to go. I think for many (most?) people it’s not a fully workable product even as a beta until it can handle unstructured rides.

3 Likes

It’s pre-RC, but should be close to the final, as your second part states. there’s usually a planned number of spins after going beta, unless the world breaks. The software schedule is entirely mapped out, with internal integration points.

A RC would then be called RC and post beta. But, if the Beta is good, then the other steps can be collapsed.

Sometimes it would be Beta-1, Beta-2, Beta-3, RC-1, RC-2, GM (with the old non-PC term Golden Master for the master disc/image). Don’t know what the new term is for GM.

I thought Nate said that wouldn’t be in the release as it shouldn’t hold it up, but it’s something on the long pole.

Perhaps something new came about.

From an external perspective, either the TR teams are extremely small (given the bugs and regressions found as well as the cadence of releases pushed) and the testing is too limiting, or they are a bit more loose than formal in structured development cycles.

4 Likes

I’m not following a triathlon plan so I missed that, sorry.

Provided you’re doing 2-3 indoor TR rides a week (or functioning TR outdoor workouts, wherever that is these days), then I think the ability to consider unstructured rides is a bit of a red herring for a large % of users.

That’s basically how I have been using AT and it worked out great for me. E.g., one VO2 and one Threshold workout a week is relatively normal on TR plans - if they happen to get easier cos I have done loads of VO2 on club rides etc, then my survey responses change and AT bumps up the next workouts.

There’s a couple of times where TR suggests workouts that are way easy just because I hadn’t done anything in TR in those zones recently. Especially Endurance. There’s 2 ways around this - manually associate TR endurance rides with your unstructured outside rides in order to keep the PL up, or just pick a harder endurance ride from the Alternates…

I personally don’t have a big issue with picking alternates from time to time but I gather some people do. Obviously, the people on here are likely to be in the vocal minority so I am not sure if forum feedback on AT is really representative of the user base, A lot of users will just pick it up and use it.

7 Likes

realizing I’m not an edge case, I did an entire base-build-specialty cycle with AT and it worked flawlessly. I don’t really do unstructured stuff, so on a couple of occasions I did (a fast group ride and a century) I didn’t get progression level changes, but in my opinion it’s inconsequential unless you’re actually doing a lot of work on particular energy systems and then you get served really easy workouts relative to your abilities. The easy step is to try a level 4 workout and help AT with setting a better baseline level. With the current fix to outdoor workouts to auto-pass, I can’t imagine too many reasons not to keep up training. And for those who just ride for fun and don’t want to do structured training, well that’s the tradeoff, either go out and do whatever or stick with a plan.

5 Likes

except for the recent regression to not update PLs if its a custom workouts.

How many TR users do high-volume full based/build/specialty cycles, and almost all on the trainer? Maybe you are an edge case?

1 Like

I actually meant that I am an edge case, not not lol

But I’ve done plenty of my workouts outside this year too, even prior to implementing the current fix, and I had no issues running workouts and getting credit.

1 Like

I’m with @trpnhntr on this, I assume most TR users are probably on AT now so changing it to from beta to “full release” doesn’t really change anything because even if it’s out of beta you can be sure it’ll be updated/changed/improved for a long time to come. The AT process is simply too new for the coaching/training industry, so the more AT data they have to analyze the better they can refine the AT progression.

However I can’t wait for progression levels to figure out unstructured group rides/races (obviously a very tough challenge), and for the levels not to be so sensitive to small-ish FTP changes.

Having done some alpha and beta testing for a very, very big software company, a public beta need not be close to a release candidate. Apple and Microsoft release various types of betas of their software. I would liken TR’s closed beta to the developer betas Apple distributes of its OSes. Of those, some get designated Public Betas. But early public betas need not be close to the release candidate. When I was testing for Apple, a release candidate was almost always almost identical to the golden master (i. e. the final initial release) unless there were showstopper bugs, i. e. the finished product released to all customers.

Google famously released quite a few features and products as beta even though other companies would just call them initial releases. Gmail started as a public beta in 2004. If memory serves, it stayed a “beta” for many years. Then there is software that is not considered beta, but extremely broken (btrfs comes to mind).

Drawing from my experience, the closed beta felt like developer betas: broadly speaking, the features were there, but you ran into lots of edge cases. The open beta right now feels like a public beta with some important features being delayed.

Most notably is the algorithm that scores outdoor rides and outdoor workouts (at least those done with Wahoo head units). They can screw up PLs of people who ride a lot, and there is no good solution right now, but to override AT when necessary. I reckon that this algorithm might also be used to score custom indoor workouts. (Last time I checked, the workout editor is still based on Adobe Flash, which has been deprecated years ago and looks like it has been designed for Windows XP in 2007.) However, you can mitigate this by getting to know AT and overriding it when it makes sense. I don’t find that super hard, to be honest, and the progression levels are on average a very good indicator of how hard a workout taxes you in a particular power range.

I wouldn’t mind if AT stayed in public beta for quite a while longer: it is a feature that can only get better if it is used and tested by a wide array of athletes — including people like you who like to rock their custom workouts. :slight_smile: (I reckon that alone makes AT of much less use to you than to me.) And your criticism that you don’t find AT useful for you is totally fair. Fortunately, the solution to your issues is also clear … but unfortunately, isn’t on the horizon yet.

IMHO my main beef with TR’s software is elsewhere: beyond the workout screen (which does its job very well), I just don’t find their apps very good. One big miss is the lack of analysis tools: how did my performance change over the course of one or several seasons? How does my performance compare the same time last year? The closest is the list of dates and FTP numbers, but I don’t think anyone can parse this without creating a graph. They should also integrate some basic nutrition info (optionally, of course) so that you can track how much you drank or ate during workouts and compute liters per hour or grams of carbs per hour automatically. Things like sleep tracking and the workout editor should be built in. Integration with iOS’s Health app. Similarly, I think you should have some visual marker if you took a break during a workout. Oh, and proper calendar management inside the mobile apps.

4 Likes

Yes. And I’ve got a long wish list too. My fundamental issue is that I found a better macro approach, or base/build/specialty if you will, and no amount of AT is going to fix that. I’m absolutely certain TR’s approach works for others, but I only care about my performance. The season planning discussion in HighNorth guide (that I bought today) had an interesting discussion on season planning.

4 Likes

IMHO TR puts some effort into displaying how your fitness develops over time, AT could be very useful at one point. Progression levels could track your progress. And you can enter the FTP by hand, so you can use any testing protocol you’d like.

And you might still benefit from AT if you pick your workouts for your custom training plan from TR’s catalog. Progression levels should still give you a good indication of your progress, and be a good indicator of what a good alternate workout is.

Yes, TR has had the ability to show how fitness increases or decreases over time. Its called a power curve and was rolled out when performance analytics was launched over 29 months ago.

Lets look at how my fitness dropped after switching from self coaching to using TR training plans:

and then increased by switching from TR to another approach using an off-the-shelf plan (purchase once, use over and over):

That is 7 months of FasCat off-the-shelf plans with some mods based on feeling. Really I stopped training the second week of June, most of the gains on that chart are from 5 months of switching to FasCat approach.

And some improvements on short power too, after switching from TR to FasCat approach:

Thanks for the pep talk, but the data and feelings are crystal clear. I’m sure adaptive training would help, thats not the point. Using other analytics I can post my heart rate data, and this new approach has put less strain on my heart.

Thanks but I’ll stick with what is working, as I’m stronger and healthier by following a different road.

I know about the power curve, but it provides only very coarse grained information (not least because it is based on seasons). Two big problems that come to mind: (1) you will have to wait for a while for it to be populated with meaningful data and you have close to zero granularity in the time dimension. And (2) what if your goals change from one season to the next? To give you an idea what I mean here, last season I opted for the Crit Plan to change things up and I was hoping to compete in a few crit races (I did one so far). As a result, I got a massive boost in FTP and could hold 117 % FTP for almost 7 minutes without going all-out. But I predictably I wasn’t able to hold my (relative!) power level for as long as I did before. That wasn’t because I was failing, that was by design. In absolute terms, I had the best season ever. How do I extract that kind of nuance out of my power curves? What if I want to check progress mid-season? Furthermore, it is currently not available on the iOS app.

All I am saying is that what TR has now is not, hmmm, good. A chart with progression of PLs on top of FTP changes could add very useful information here. But so far they haven’t attempted that. Heart rate data could be very useful to show how fit you are. (I can tell how fit I am by seeing how quickly my heart rate recovers after a hard interval to my VT1ish heart rate. If it is less than 1:30 minutes after a few intervals, I’m in good shape.)

I think you misunderstand my arguments: I think the tools TR makes should also serve people like you who are probably using Training Peaks in addition to TR (I’m guessing here, so feel free to tell me that I am wrong).

I used to use TP, but found their system of their trifecta of metrics antiquated. It was less a measure of my performance and more a measure of how hard/easy a training week would probably feel like. Their race and ride analysis tools were nice and you could track metrics like mood and sleep, but in the end I wasn’t using it enough to really justify the cost. Plus, I felt like they weren’t doing anything with the data, I’d have to parse them manually.

1 Like

I’m not buying the “power curve, but …” argument. My entire power curve dropped after switching from self-coaching with Carmichael Training Systems approach over to TR plans. Then my entire power curve increased after switching from TR plans to FasCat approach. I’m faster, stronger, and healthier. Without question, except perhaps in forum-land :man_shrugging: In the second chart above I didn’t even train 2-5 minute power and saw a huge increase over TR.

I’ve got a WKO license (paid $170 Sept 2019) and can extract all kinds of interesting and nuanced stuff. Except for things like yesterday, wherein at the start of my 4th week back after a 5-week off-season (during which I was sick twice), my 2 hour benchmark zone2 workout is back to the peak of my fitness in May. But the same happened when I was self-coaching in 2017, and I have a hunch why.