Sorry, working late. its 1:30am.
Nate also said in the announcment video (in Feb) that his system processed outside workouts, and that was coming soon
Feels more like an alpha
I think a lot of the early steps were very much alpha, if not development. But as far as my team would be concerned, this wasnât ever a beta test, and isnât currently, and the main reason for that would be that the purpose of a beta test is to determine if it works, and not that âwords are spelt correctlyâ and âdoes the survery list the right orderâ, but does it actually work, as in, does my FTP actually improve when following adaptive training at a greater rate than if i wasnât, and also the users werenât set an expectation of was expected of them during the beta test, these are prerequisits for us, but I work in software in the healt industry and mistakes arenât as simple âftp has gone downâ, this to me (but again it could be my training), this is about throwing untested code at users, calling it a beta, and seeing what happens
For me the beta test would have been âdoes this improve ftp at a faster rate a standard planâ as far as I can tell, this hasnât been tested (there is another thread about ftp going down)
For me the beta test would have been âdoes this improve ftp at a faster rate a standard planâ as far as I can tell, this hasnât been tested (there is another thread about ftp going down)
Well everyone is different, so the test should be for the majority rather than 100%. Additionally as I mentioned above, the people posting on here are likely to be the vocal minority (I guess there must be tens of thousands of people using AT now, and how many are posting on here?), so it would be surprising if the feedback on here was totally representative.
Personally I had great results using it this year, setting all time PRs etc across all durations, and I think this was mainly due to better compliance with a smoother progression - previously in the fixed plans there used to be a few progressions in Build which would totally wipe me out, then Iâd lose motivation and basically stop following the plan. But I am not making threads about it
I totally agree, you should be very careful about analysing a forum, only the angry people post, but two things
- You post is a cop out, you are saying that all is well, because the thousands of people that using AT, only the angry ones are posting, which dismisses the angry ones as irrelivant, without evidence, not everybody that is angry posts either, so the best feedback is no feedback ?
- That wasnât what I wasnât what I was talking about, so you are dismissing my post with âhappy users donât postâ, I was talking about the aim of a beta test would be to detemine if the product work âdoes it improve users ftpâ, and by your own post, this hasnât been done. because the âthe happy users havenât postedâ
Feedback from a beta test should never be âhow many people post in a forumâ
Oh no, that wasnât what I meant, sorry!
I was only really saying that just because a few people have threads saying theyâve had issues doesnât invalidate the beta - no training approach will work for 100% of users, surely. I totally agree that you canât validate a beta just because angry people arenât posting on a forum! The evidence will be in the data, which we donât have access to, but TR does - would be cool if they could post something, even high level.
I was only really saying that just because a few people have threads saying theyâve had issues doesnât invalidate the beta
Oh I totally agree, but I am saying (suggesting) is that this was never a beta, because it didnât set user expectation, or reporting
The evidence will be in the data, which we donât have access to, but TR does
Iâm sorry but that is another dismissive statement, and canât be checked or questioned, it was used on people who said that the progression rates of TR plans was to high (look in my history) and when DJ posted his video, why there was so much feedback, because so many people were dismissed
And only using your data will be miss leading (if you only look at patients who have died from cancer, 100% of people die from cancer), TR data can quickly be skewed by people not using strava, triathletes would have a different progression rate than cyclist, but TR donât gather swimming/running data (or other sports), do TR actually employ statisticians ?
Itâs just a easy comment to dismiss other people, and doesnât fit into what should be required in settting up a beta test, and setting user expectations, and that the user behavior is consistent during the beta test (do they upload to TR)
I would also add being able to adapt it to the triathlon plans, which Iâm sure itâs a relevant part of their userbase.
Please understand: not very.
Iâm sorry but that is another dismissive statement, and canât be checked or questioned
But itâs true isnât it? We canât know how effective AT is unless TR publishes some evidence, as they hold all of the data here.
Even if you were to ask people to contribute you are going to get only a subset of data (i.e., those engaged users who post in this forum, unless you invest a lot more time into it).
To whether TR employs statisticians - I would guess yes but obviously I have no better idea than anyone else who doesnât work there.
To be clear I think they should show some evidence. At the moment we basically just have their word that people on AT do better than those who arenât on AT.
But itâs true isnât it?
Maybe, but you dismissed my argument with âTR have the dataâ
We canât know how effective AT is unless TR publishes some evidence,
Exactly (presuming they have actually looked at it), but , again, my post was entirely about is this a beta test, not if AT works, you dismissed that with TR have the data, in my experiance, thats no a guarrente of accuracy, and like I said, a dismissive comment that canât be questioned as unless âTR publishes some evidence, as they hold all of the data here.â, which wouldnât be per reviewed
I would guess
And thats me out of here
Exactly (presuming they have actually looked at it), but , again, my post was entirely about is this a beta test, not if AT works, you dismissed that with TR have the data, in my experiance, thats no a guarrente of accuracy, and like I said, a dismissive comment that canât be questioned as unless âTR publishes some evidence, as they hold all of the data here.â, which wouldnât be per reviewed
Thatâs a fundamental limitation of this test. Unless TR decides to publish something (yes itâd be great if it was peer reviewed too) we wonât get any further information.
I wasnât meaning to be dismissive but there is literally no better answer at the moment - it is all in TRâs hands. I certainly canât claim to know that AT is better than any other system, all I can say with any certainty is that it worked well for me this year.
I certainly canât claim to know that AT is better than any other system, all I can say with any certainty is that it worked well for me this year.
Never commented on AT working / no working
OK - I certainly canât say whether the AT beta has been successful and nor can anyone else outside of TR. Is that better?
No, wasnât commenting on AT Success or Success of the beta at all, in any way
After all , I am using AT
all I can say with any certainty is that it worked well for me this year
Actually you canât. All you can say is that you âimprovedâ. You donât know if this improvement was due to:
- TR making the plans easier. And even without AT, you would have improved because the easier plans didnât over tax you as much, increased your compliance, etc.
- Simply doing another year of training, you would have made the same, or better, improvements
- Since TR hasnât actually said what AT is optimizing for, you donât know if how you are viewing âimprovementâ is even what AT is trying to âimproveâ. So for your N=1, this could all be a big coincidence
- Placebo effect - your improved compliance / improvement could simply because you believed that AT would be beneficial, so mentally you approached all workouts with a different belief about your ability to complete the workout. And the âimprovementâ you are noticing is simply due to this
This is my big problem with AT at the moment: we donât actually know how TR is evaluating its performance, and more importantly, what are they trying to optimize around? Nor to the best of my knowledge has TR done a large A/B test: Randomly assign a large number of people to one of the following groups:
- Group A (vaguely the control group): 100% indoor rides using the TR app and new plans âas isâ without AT
- Group B (the AT group): 100% indoor rides using the TR app and new plans + AT
You could easily add more groups that would eliminate the strict need for 100% indoor rides using the TR app, but given that these would introduce ânoiseâ / potential confounding factors, I view this as above and beyond the 2 base groups called out above that would be the most straight forward way to see if AT actually is beneficial.
You are right, it is entirely my perception. The smoother progression in Build is probably the main factor which kept me compliant, as I have got quite a bit of experience with TR plans over the years.
Theres lots of things to like about it for me - I like the PLs and easy selection of sensible alternates etc - but nothing quantitative.
I seem to remember that when AT was in closed beta that TR were looking at bring individuals on board who fitted certain usage profiles. My assumption is that this was to do some A/B testing. Of course weâre all individuals so another assumption is that they were trying to have lots of smaller groups: TR users under one year with FTP improvements of 10 - 30% since starting; that sort of thing. Get ten or fifteen such sub-groups with around 50 in each A &B set and you can start to work out whatâs working and what isnât.
Given that many papers base their research and conclusions on fewer than twenty individuals ( a guess but I doubt itâs far out) itâs likely that TR have a better base on which to make decisions.
Thatâs why I was careful to say ânor to the best of my opinionâ. If TR did do careful A/B testing, then this would be great. And hopefully they will talk about this, even obliquely in the Q&A next week. I would especially like them to talk about what they are optimizing for:
- Compliance - hypothesize that increased compliance to plan leads to better âfitnessâ / increased âFTPâ / ???
- FTP increase - this is another whole kettle of fish, because if you used âFTâ to mean 75% of Ramp Test, then AT could increase âFTPâ by making people more anaerobic to do better on the RAMP test. Which wouldnât really increase âFTPâ - just ramp test value
- ???
And then you get into a whole other issue of what timeframe are you looking over for âimprovementâ: this season? the next 2 seasons? ???