Throughout our articles, we provide advice across an extremely wide range of topics. Much of our advice can be very speculative - it is genuinely hard to know which career paths will have more impact than others over the next five decades (or more). The breadth of fields we aim to cover also means we cannot invest the same amount of time and effort in all career paths. Much like our readers have to make career decisions under great uncertainty, we believe it is valuable for us to share insights and priorities at different levels of confidence rather than avoid sharing information until we are completely confident we are not making mistakes (which we never are).
When sharing uncertain information, we believe it is critical to be transparent about the level of confidence we have in the information we are sharing. This can help readers prioritize which articles are most worth their time, and make more informed decisions on when they want to listen to our advice or their own experience (hint: we don’t think the answer is to always listen to our advice).
With this goal in mind, we indicate our level of confidence at the top of our different career path profiles. The purpose of this page is to clarify what we mean by each confidence level - allowing those interested in the details to understand just how reliable different articles are.
Finally, in the interest of brevity and clarity we boil down our confidence into a small number of categories indicating our overall confidence in the whole article. Obviously, many nuances are lost in the process - there are components of an article we are more confident in than others, our advice may generalize better in some circumstances than others, and there are different senses in which we can be uncertain about different things (for example, if we haven’t consulted with a sufficient number of experts, or if the question is inherently speculative and subjective). If you are making critical career decisions and would like more detailed and personalized advice, we’d love for you to reach out to us for one on one consultation.
Confidence level definitions
We apply the “high confidence” label only to articles we have decided to invest in significantly, base our recommendations on a wide range of expert opinion or publications, and believe the conclusions are fairly robust and are unlikely to turn out to be false or non-generalizable. We expect articles in this category to be useful even for people with experience or expertise in the field.
More concretely, we require all the following standards to be realized before using the “high confidence” label:
We have spent more than 50 hours in relevant research and consultations in preparing the article.
We have interviewed at least 7 independent experts in the relevant field/path, have asked them about the core views expressed in the article, and the views expressed represent the level of support of these experts.
The views expressed in the article concern issues for which experts and evidence are likely to be reliable and informative, rather than dependent on speculative questions for which even expert opinion is unreliable.
We apply the “moderate confidence” label to articles we’ve managed to collect significant information and recommendations for, but are still not entirely confident in their core conclusions. We believe their recommendations are useful and likely correct, but may turn out to not generalize sufficiently well, be missing nuances, or in some cases end up simply being mistakes. We believe people who don’t have any direct experience or expert knowledge on the relevant path would benefit significantly from listening to this advice, but if people have direct personal experience indicating otherwise that experience may be more reliable (and we’d love to hear about it!) The majority of our articles fall under this category.
More concretely, we require all the following standards to be realized before using the “moderate confidence” label:
We have spent more than 15 hours in relevant research and consultations in preparing the article.
We have interviewed at least 3 independent experts in the relevant field/path, have asked them about the core views expressed in the article, and the views expressed represent the level of support of these experts.
The views expressed in the article may concern speculative questions for which it is difficult to form highly reliable opinions, yet must also contain core content for which experts can reasonably gain confidence in.
We apply the “low confidence” label to articles which we have not yet prioritized for in-depth research, but have done some preliminary analysis on and believe we have useful advice to share. Based on the information we have, we believe the recommendations in the article can be concretely beneficial, yet we may only think this is true on balance of probabilities, rather than being a statement of confidence that the details are actually true. As a result, we believe they are worth listening to when lacking other sources of evidence, but we recommend readers apply their own judgement and explore other sources of information before making major decisions based on these articles.
More concretely, we require all the following standards to be realized before using the “low confidence” label:
We have spent more than 3 hours in relevant research and consultations in preparing the article.
We have interviewed at least 1 expert in the relevant field/path, which may be a member of our organization.
We have shared the article with several advisors whose opinions we value and trust, and these advisors have expressed support for the value of this article, though they may not be experts in the relevant field.
Speculative and exploratory
On rare occasions, we want to share preliminary thoughts that we have not yet properly investigated. The goal of these articles is to gauge interest, call for contributions, generate a discussion, or catalyze action which we hope will eventually lead to useful and reliable advice in some other way (rather than being an end-product to be used as advice on its own). As a result, these pieces of content do not have minimum standards for reliability.
To avoid the risk of this content being mistaken for reliable advice, in these cases we do not rely on the short confidence level label as in other articles, but rather explicitly describe the caveats and level of confidence within the article itself.