Data Project [Part 1]: Sleep Analysis
About two years ago I started collecting data about my sleep via a wearable that I purchased in early 2020.
I knew that I was sleeping badly, but I didn’t consider my sleep an impediment to my productivity or health. That wasn’t until my hands and feet started aching.
At first, this wasn’t really even noticeable. I put it down to training. I am in the habit of using 50mm grips and a climbing hangboard to increase my grip strength. I thought all of this was just post-workout soreness. Boy was I in for a surprise.
It wasn’t until late in 2022 that I started waking up sore and in a lot more pain. That slowly escalated to a point where I was waking up 6 times a night because my body was aching so badly.
If you had asked, I would have responded with, “I’m fine!”.
I decided to get all the data I could from my wearable (not something that is easy to do) and turned my sorrowful sleep routine into a data science project.
What I found was pretty shocking and would ultimately lead to a diagnosis from my doctor that was very eye-opening!
Hang on its about to get bumpy in here!
Before I get to the part of this story where I corroborate my findings and tell you how my sleep improved after all this effort, I really do have to torture you a bit with how painful this project was.
I won’t ruin your reading experience with all the technical details, but let me just say, there were over 700 records that had to be scraped from images on my phone.
I had to find a way to automate taking the screenshots, prepare the images for text extraction and tediously, repeat this process numerous times because the data that was extracted was often only partially readable or complete.
So there. It was painful. Still with me?
Good, because here is the fun part! I started my Exploratory Analysis using a pair plot. While I saw that there may be a relationship between the total steps I had taken each day and the quality of my sleep, nothing else really stood out for me. There was a lot of scatter in the data, which was to be expected. After all, I was getting up 8 times a night from time to time.
After obtaining a rough overview of the data, I decided to plot the features of this dataset using time series analysis.
What was fairly obvious was that the total number of times that I was waking up each night was far, far worse than I had anticipated.
I have had clients that have come to see me for coaching on sleep-related challenges and if I had been my own client, I would have fired myself! This is terrible!
To complete my EDA I decided to use a heatmap to look for correlations in the data. This was what I found.
It was at this point that I really began digging into my data to see if there were further relationships that might explain why I was so sore. Upon further reflection, I realised that if there had been a relationship between how many steps I was taking a day and the quality of my sleep, the opposite may in fact be true.
This prompted me to contact my physician. When I showed him this data he was able to diagnose me with Gout.
So, I’m sure you could understand why I was so flabbergasted about this. I don’t drink and it’s been 3 years of sober for me. Although correlation does not equal causation, I went for the bloodwork and he was right!
Who would have thought! Even people who don’t drink can develop Gout! :/
I’m currently being treated and the improvement in my quality of life is immeasurable. I can sit comfortably, stand for longer than 10 minutes and sleep at night.
All because I decided to take a look at the DATA! :-)
Conclusion
Thankfully I was able to translate my passion for data into an outcome that was beneficial for me. My sleep has improved and I am no longer in pain.
Don’t take your health for granted.
If something is wrong, try to find more information that you can show to your physician. I’m using a Whoop strap now and you can actually export your data into a fancy report that can be shared with your physician.
Find out more about Whoop here.
Thanks for reading!
D
