How Useful is Quantified Self?
Exploring the question of whether you can do anything with N=1 experiments
The proliferation of wearable devices such as Apple Watch and Fitbit has fueled a phenomenon known as quantified self—the pursuit of self knowledge through data. A major concern with these N=1 studies is that they have an insufficient sample size to warrant any kind of statistical inference; the common objection is that this data, although possibly interesting to look at, can’t be useful for telling us anything meaningful with any confidence.
A regulatory body like the FDA would never except the results of a clinical trial with one participant; there is too much noise and not enough signal. A drug could make 10% of people terribly ill, but this would go undetected if the one participant didn’t get sick. Conversely, if the participant did get terribly ill, we would lack the sample size to attribute it to the drug rather than randomness. We can’t know if we’re making type I errors (false positives) nor can we know if we’re making type II errors (false negatives). Why should we be much more lax than the FDA with our standards of what makes for useable data?
If we’re doing quantified self to figure out something about people in general, we should be hesitant. If there is an already available literature on the subject, this would be significantly stronger evidence than a single participant experiment. A large enough literature might make an N=1 experiment irrelevant and not worth considering.
However, if there is no available literature and no good reason to think one way or another, then quantified self can actually serve as evidence toward or against a hypothesis. If you have absolutely no prior information to lean one way or another, a quantified self experiment may be very informative.
The portability of a quantified self experiment will be determined by how alike you are with the person on the relevant variables they are studying and possible confounders. If you are part of a small group of people with a very rare eating disorder, you may want to converse with the community and look at their data. Any sort of data you could work with might help you to avoid foods which trigger the unwanted symptoms.
Of course, the person that is most relevant to a quantified self experiment is the self. Many people just want to know more about themselves and the best way to do it is through data collection. If I wanted to know if I will like spicy food, I could look at research to determine that 80% of people like spicy food and conclude that I most likely will like spicy food, or I could just try it myself.
A lot of population level statistics give good guidance, but there are also certain idiosyncrasies of a person’s life that can’t be evaluated with survey data or research like “Does hanging out with my buddy Joe make me feel better?”
People live their life with emotional ups and downs, feeling physically better or worse, increasing productivity and declining in productivity, sleeping well and sleeping poorly and other variation across all aspects. Most people just try to be guided by their intuition and fleeting memory to form associations such as “I don’t sleep so well when I eat late.” And often they are right, but many of these types of relationships go unexamined.
The benefit of QS is that it becomes explicit. The relationships are in the dataset, you just need to find them. Collecting and analyzing the data is tiresome at times, but imagine you discover a lifelong source of happiness or productivity. This is the type of diamonds in the rough that makes it all worth it. If you do find it, you can try to control more factors and get more data to verify the effect is real! This is the future of self-help.
If you like newsletters, you can try The Sample for topics that interest you. Thank you for reading. Feel free to leave a comment.