I think you are missing the point. You are right that many of us don’t religiously follow a plan. That’s a choice. IMO if the AI can’t deal with that, it’s simply broken.
What I’m not saying is that is that I’ve got a better plan than TR AI offers. Or that I think it’s estimates when clsely following a plan are wrong. My issue is that the AI just can’t deal with complex situations and gives bad esitmates. Most of us have been around long enough to know when we’re getting fitter, If the AI says we’re getting weaker when hard measurements show improvement it’s just broken.
Absolutely, but there is a clear group of people who attack everyone having a problem and come up with a reason everything is the end user’s fault and it gets old, so people just give up and stop posting or flame out and get banned. Followed the plan exactly? “Well duh, why didn’t you change your FTP or pick alternates?” Chose alternates or modified your FTP? “Well duh, why didn’t you trust the plan?” Trusted the plan? “Well duh, you’re your own coach!” See a poll that shows 40% think things are headed in the wrong direction? “Well duh, the squeaky wheel gets the grease”. If it had been 90% happy, “See, the poll proves it’s just a few miscreants making trouble”. It’s always the user’s fault with these types and at times it feels like “They told you to reject the evidence of your eyes and ears.”
That is not how surveys work. When companies/organizations/people want to know how their customers feel, they don’t have to ask ALL of their customers. They do a sample population. There are calculotors out there that can help you determine the size of your sample for confidence in results, but in short, the size of the sample is much smaller than you might think. For market research, a sample of 100-400 people provides a good baseline for feedback and making directional decisions. As of now, 186 users have answered.
Now - the obvious criticism is that this poll was not sent to a random sample. This poll is to forum users who may skew a certain way compared to your average TR user. But this is still solid feedback and I would not completely dismiss if I were TR.
I’m probably one of only a few, but I canceled my sub a few months ago because I think the AI features made it worse. I still frequent the forum because there are knowledgable members and I like the discussions. But I wasn’t happy at all with the app. I ride outside mostly and it never seemed to work well with outdoor rides and workouts. The workout suggestions were all over the place sometimes and didn’t make any sense. I had a friend that’s a coach look at some of the progression and he was baffled what it was trying to do. I felt like the workout surveys had way too much weight into the upcoming workouts sometimes, especially with VO2, which would get rated Very Hard or All Out and I’d always get given way easier workouts the next time, even when completing the workouts just fine. I also was not a fan of the daily FTP prediction that would change wildly day to day seemingly untied to what I was doing. Swings of 20W week to week when I haven’t had a 10W swing in my FTP in years.
I first started TR back in 2018. I’ve seen lots of changes over the years with it, mostly good. I’ve made huge gains, going from around 200W up to 350W FTP. Up until a few months ago it always made sense. But with the AI features, I just felt like I spent more time wondering what the heck was going on more than training. I was questioning every workout change and because I’d get changes to my plan multiple times a day, it was just too much. And I wasn’t getting any stronger.
It isn’t statistical significance that you need to worry about, it is whether it is a representative sample. It isn’t. That isn’t to ding people who e. g. have run into bugs (like the outdoor AI FTP bug) or for whom it doesn’t work, it is just that these people are more likely to post here and to have responded to the poll.
I’d be cautious with such conclusions. People for whom it is working are unlikely to post. Some of the bugs are genuine, but often only a small percentage of users are affected.
Correct, and I don’t get the impression they are. It is a combination of all sorts of factors, including seeing
rare bugs that are unlikely to surface with a small beta user group,
athletes who are statistical outliers.
Addressing the later especially is difficult as I reckon improvements for one group may degrade the results for everyone else.
Your response summed it up for me as well. I have used TR the last few years and has been super helpful. I also did it in conjunction with another plan I followed but seemed to work well together. My rides are a combination of both indoors and outdoors. I tried the new version released this year but could not wrap my head around having two different FTP’s and my TR FTP jumping all over the place. I find the forum really helpful to read with lots of great input but did cancel my sub a couple months ago as the new system just hasn’t worked for me.
But that’s part of my point. In the last 5 years, how many new features had this many separate threads dedicated to its problems? I would wager there are more threads related the AI features than every other major feature release combined. In the past, you might have a few threads after a feature is released, then it quiets down. But with the AI release, it’s been multiple new threads every week, week on week. It’s literally never stopped. That’s a big indicator. I honestly don’t remember the last time I came to the forum and didn’t see an AI-related problem within the top 10 most recent threads.
Since essentially all of TR’s features are enabled by “AI”, I don’t see that as a good measure one way or the other.
TR AI has fundamentally changed how TR’s software works. It is probably the single biggest update it has ever had. Moreover, what the marketing department calls TR AI is a whole host of different pieces that have to work together.
As a result I have seen a whole host of support threads, e. g.
how to use TR AI properly and struggling to adapt,
people not being aware that certain features exist (and perhaps even predate TR AI),
people fighting with TR AI instead of adapting or switching features off,
bugs,
UI and UX problems (such as (not) showing FTP projections to users),
people struggling with what FTP really is
Overall, TR AI has made TrainerRoad more complex in many ways. It really pays off to know how it works and to find new ways to use it. But if I look at the individual pieces, I don’t think it has gotten “worse” than before. Plus, there are bugs that make the product worse for the group of people that run into it.
When TR mentions numbers, then it seems TR is, statistically speaking, a significant improvement over AT. But even if, hypothetically, it works as advertised for 95 % of athletes, 5 % of a user base that is well in the tens of thousands is still a very significant number.
I’m not denying that people are having legitimate issues with TRAI. And I completely get your point about “trust the AI then!” vs “set your FTP manually then!” etc. Total mixed messages, you get no argument from me there although personally I have indeed found that I live in both worlds a bit (trusting the AI but also making tweaks) and it has worked incredibly well for me.
My beef is with those that insist upon arguing in bad faith, claiming the issue is always TR’s fault when there are indicators of other potential causes. The blame door swings both ways. It might be worth keeping in mind that those of us speaking from the positive side have been having good, great or in some cases phenomenal results all the while seeing people come through claiming that TRAI is “broken” or that it should be removed or scrapped altogether because they don’t like it. They claim they are “mostly following the plan” and “for some reason” it’s throwing out results they don’t like. Then they show their training history and we all have to scramble to figure out what their definition of the word “mostly” might mean because their calendar is a dogs breakfast. It would be funny if it wasn’t so infuriating.
As I alluded to previously, the biggest victims of this kind of bad faith behavior are those of you who are doing everything right and still not getting results. It muddies the waters and makes everyone cynical to the real issues that need to be addressed. I’m not saying TR is perfect or that those of you struggling are lying or even wrong. I’m just saying that if you’re going to get out the bullhorn and demand justice then at least be willing to be open to suggestion or advice because we have learned from experience that a lot of the time it has been user error.
All that said, I wouldn’t put a lot of faith in a poll that’s run in the #1 place you’d expect to find people searching for help with problems as its unlikely to be a reliable indicator of the overall user base.
I really hope they can get this bug sorted quickly for those that are affected. It must be extremely frustrating to have a large part of the season affected by this.
I’m now another bad example too because I’ve given up after 6+ months of trying. I’ve gone back to using a different tool and, so far, I’m happier. I just regret that I bought in to the early hype and signed up two people for a full year. That’s on me, but I’m definitely bitter about it, which does admittedly skew my perspective.
You get no disagreement from me. IMHO TR should think about splitting its product into a pro and a regular version where the pro version has more knobs to turn.
Even if you assume for a moment the product has no bugs and works for everyone, the fact that you can pick TR AI’s aggressiveness for different workout types introduces a level of complexity it hasn’t had. Ditto for having to select between (not) allowing TR to ramp its endurance rides in time rather than just intensity. Or the effects of showing users predicted FTPs, what is motivating for some is demotivating for others.
On the other hand, TR AI has given me a lot of flexibility on how to use it. E. g. I can create a simple training plan without Plan Builder by dropping in workouts-by-type and let TR AI figure out the rest. That has worked better for me than Train Now in the past, and I will go that route when I start training again. (My wife has had to deal with a serious health condition, and I am currently not training.)
I think you misunderstood my point: ML and AI have become regular tools in the arsenal of programmers, and I merely see those issues (usability and bugs) as part of a major new software release, not a ML-specific issue.
For that matter I have to admit that the experience I’ve had since the beta skews my perspective too. I’ve never argued for TRAI just because I want to be right. I’ve argued for it because I believe it works and don’t want people to walk away just because they have misapprehensions about it or for them to give others misapprehensions about it.
It sucks that you lost 6 months of training in the turmoil and I know you’re not alone in that.
There is a significant difference on how these issues are fixed in an ML environment.
Previously, a bug/issue would be narrowed down to a line of code, a function, an algorithm, etc. and it could be fixed by a programmer. The scope of the change could be small or large, but the programmer can understand the scope, the fix can be validated, any unintended consequences can be resolved, and when the product is QA’d it can go live and the issue is solved (with high probabilty). Perhaps days to fix a bug, then a few weeks to go through QA and intercept a release.
In this new ML world it’s not generally about the code any more - typcally there literally is no line of code to go and fix. ML is probabilistic/statistical and there’s a huge pile of impenetrable maths between inputs and outputs with huge amounts of obscure parameters. The structure of the ML calculation is understandable (layer and matrix multiply etc.), but it is near impossible to introspect why the ML makes any particular decision because it is buried in some set of parameters and some set of deeply nested calculations, and ultimately could come down to something in the training data, some bogus correlation, or hallucination with no way to isolate that one effect.
If you are lucky, you might be able to just fix something in the system prompt (stop talking about goblins!!) but these may be band-aids not real fixes.
So fixing an issue requires a lot of experimentation/iteraton - data santization, retraining the model, tweaking the model training objectives, increasing the training data set, more and more training time, or subtle manipulation of the prompts. The even harder part is doing this without changing other behaviors creating unrelated issues. Unlike a surgical code fix, it is much harder to define the scope of the changed model. You basically have to re-QA everything, measure metrics before and after across bulk data sets, and chase down a bunch of out-liers … when that finds yet more issues you have to iterate. If you converge then you can release. If you don’t converge, you have to start over.
This is why it now takes months to fix issues rather than days. And even when an issue is fixed, it is very hard to be sure that you haven’t introduced a worse problem that will only be exposed later at scale …