I created an open source Strava app that let’s you see your activities on a 3D globe. The activities table is pretty useful too.
2 Likes
my “product” continues to evolve. The big thing is that I finally merged the handling of fit files and strava downloads into a central database, until today I had them being handled differently as I was learning how to work with these data and was finally able to marry them into the Django database (using postgres and storing each data stream as an arrayfield)
And just because visuals are always fun, just continuing to build my ride detail page with more summary stats. I’ll have to figure out how to get this onto a real server and see how it’ll handle additional users
2 Likes
Intervals.icu now has a public API for people who want to play with their data. There is only one endpoint so far but its a start!
3 Likes
Intervals.icu is on a server at Hetzner in Germany. One of the cheap 2nd hand ones. At least 5-10 x cheaper than anyones cloud.
Hey @hubcyclist Love what you’re doing here. Fascinating to read your updates 
I’m a data scientist & Python developer, so thought I might have a few useful suggestions/comments…
- Be careful when using stavalib. It hasn’t been updated for over for year - a good sign it’s not being maintained and will break if Strava make a change! Requests is pretty straightforward and would probably end up being more reliable.
- Storing large arrays in relational databases is generally not a great idea. You’ll run into performance issues pretty quickly I would expect. It might be better to store the time series somewhere else (e.g. in object storage, like Amazon S3, if you’re deploying it) and only storing a reference to it in Postgres alongside the workout summary stats (TSS, duration, etc.)
- Regarding deployment, you could run your own server on something like Hetzner (I’ve not used them but they’re probably fine). However, you will be able to get something up and running faster with Heroku’s free tier. Bear in mind, if you’re going to scale this to more than (at a guess) a dozen or concurrent users Heroku will get very expensive!
I hope that’s helpful!
3 Likes
Thanks! Will do have to look into these (although point 2 is still largely Greek to me lol) , right now my DB looks like this for my ridedata
Before I did this, I was just parsing the fitfile that was saved for the detailview, so maybe even less good performance-wise lol
As far as deployment, none of the free tiers would work for me because my thing has a couple of apps within it (custom user profile app and the main app). As an inbetween step I may test out on a raspberry pi zero I have apache set up, although that’s not exactly something to make public lol
What are the code bases and languages people using here? I have dabbled in some dev work but now I have all the time to kill, id like to start playing with my cycling data.
I have tried to use the Strava API with Python but never really got it to work initially. Any tips here?
Anyone using Django + React frameworks for this?
My stuff above is all Django-python and a little bit of bootstrap and bokeh for the graphs
So what you could have is a database field with a file location, where the file (e.g. a JSON file) contains the time-eries data( time, watts, heart rate, and cadence in this case). You then load the file as required. Thinking more about it I’m not sure whether this would be better. Anyway, I wouldn’t about it for the moment!
Do you mean two apps within the same Django project? Or two completely separate Django projects?
As a general point, Django is great for getting started but I wouldn’t recommend it in general. It has pretty terrible performance compared to other frameworks. Something like Falcon , Bottle, or Flask would be good alternatives.
Definitely Python for data analysis!
What problems were you having?
outside using requests, I had issues with the authentication. I was searching the web trying to use swagger.
But I think I was over complicating things for my own good…all I want is to login with Strava and get my data. - Guess requests will be able to do this?
How is this done with intervals.icu?
I’m not sure you need to bother with Swagger here, it’s just standard OAuth 2.0 authentication. I’d probably use something like Requests-OAuthlib, which uses requests and OAuthlib under the hood. I’m pretty sure Django has it’s own thing for handling OAuth 2.0.
Looks like intervals.icu uses basic authentication so you just need an api key. In fact, looks like they’ve made a right hash of their authentication… With requests in Python this is easy, you do something like
import requests
url = 'https://intervals.icu/api/v1/athlete/2049151/activities.csv'
response = requests.get(url, auth=('API_KEY', 'your-key-here'))
Note, API_KEY is a fixed user name in the basic auth 
1 Like
I wrote a post on the strava api google group on how I authenticate and refresh (because it seems half the questions on that group are about authentication). Keep in mind that I use stravalib which @occasionalathlete has rightfully suggested might not always be viable. But I have my stuff stored in a json file and I have it automatically refreshed every 6 hours.
https://groups.google.com/forum/#!topic/strava-api/9ewMamiqN2s
1 Like
Nice explination @hubcyclist 
I strongly recommend against storing secrets (the tokens) in files. It’s probably fine whilst testing on your laptop, but what you really want to avoid are secrets accidentally making their way into version control (GitHub). It’s better practice to store secrets as eviroment variables and read them from the system as required. python-dotenv is a tool for making this easier. The tokens that expire are less sensitive but still don’t need to be stored - you can keep them in memory and just request another one when you restart the app.
You could use smallint arrays for a lot of those fields and save a bunch of space. Intervals.icu stores each individual trace on the filesystem in a csv.gz file. This is important for performance in that it keeps the activity table sizes small for fast query.
Also the traces are on spinning disks with the SSDs used for the database.
You need to think about backups and the database will get very big very quickly if you store the traces in it. The traces don’t change often but the db does so you can backup the traces much less frequently.
Intervals.icu back-end is a Java Spring Boot app. Front-end is Vuejs with d3.js for charts and Vuetify components. There are all sorts of libraries for Java to do curve fitting and whatever else and its very fast, and most importantly I have been using it for 22 years now. Use what you have mastered unless it is really unsuited to the job.
2 Likes
Err what do you mean? I think basic auth over https is a perfectly reasonable choice for something like Intervals.icu.
Also a data scientist & working in Python. I’ve only skimmed through the thread and don’t have any experience with relational databases but I do have some with large arrays. As @occasionalathlete says, there are better ways. Look into hdf5. Easy to store and load. Probably all fit files should be converted into hdf5 files on import.
Also, I don’t work with pandas, numpy is just fine. However, so far I was too lazy to do what you @hubcyclist did - respect!
@hubcyclist - your work is very impressive!
Some time ago, I built an excel sheet to display a PMC, training load, etc.
Individual ride data (i.e. NP, IF and TSS) has to be entered manually, and the spreadsheet goes off and does the rest. I was working on a TiZ implementation too but got sidetracked with life.
If you are at all interested in seeing the “calendar” interface I came up with I’d be glad to share it with you. It’s a good way to view a “season” at a glance.
PS: Is your site accessible by anyone else online, or is it all local for you?
Cheers!
Thanks! Right now my site is just local on my machine, I’d like to get it public for some folks to try, as long as those folks are ok with the possibility of something going terribly wrong lol I certainly have the foundations to make it public with individual profiles and what not, I just need to find the right hosting place (for the right price lol)
1 Like