Jekyll2023-07-28T20:00:48+00:00https://dsbowen.github.io/feed.xmlDillon BowenPersonal website for Dillon Bowen, PhD candidate at the Wharton School of Business.{"avatar"=>"/assets/images/profile.jpg", "bio"=>"I'm a PhD candidate at the Wharton School of Business. My goal is to elevate behavioral science using machine learning.", "links"=>[{"label"=>"GitHub", "icon"=>"fab fa-fw fa-github", "url"=>"https://github.com/dsbowen"}]}Rethinking the relationship between age and ability2023-07-28T00:00:00+00:002023-07-28T00:00:00+00:00https://dsbowen.github.io/age-ability<p>I used to have a fairly defeatist attitude about age and achievement. Groundbreaking achievements seem to come disproportionately from younger people; what I call the <em>achievement gap</em>. Nobel Laureates typically do their prizewinning work in their 20s and 30s. Successful entrepreneurs often start their companies before age 30. Artists make their greatest breakthroughs while they’re young (often with some help from “cognitive enhancers”).</p>
<p>Based on this, it’s easy to conclude that as we get older, our abilities decline, making truly great achievements increasingly unlikely. In other words, the achievement gap is best explained by an <em>ability gap</em>. But I’ve reconsidered based on a simple statistical argument.</p>
<p>The key insight is that even if abilities don’t decline with age, we should still expect to see an achievement gap between younger and older people due to selection effects. Let me explain why.</p>
<p>First, it’s uncontroversial that achievement depends on both ability and luck. Being in the right place at the right time, having the right collaborators, getting the right break - luck plays a huge role. Second, within any field, the young people who produce the greatest achievements are selected to keep pursuing that field. An academic who does groundbreaking work as a PhD student is more likely to get a postdoc and professorship to continue researching. A successful young entrepreneur has an easier time raising funds for subsequent ventures. An artist who creates a major work early on is more likely to keep getting commissions and exhibitions.</p>
<p>Consequently, people who continue pursuing competitive fields as they age probably had both exceptional ability and exceptional luck in their earlier years. Even if their ability remains strong as they age, their luck will likely run out, resulting in less impressive achievements.</p>
<p>To be clear, this statistical argument doesn’t prove that abilities don’t decline. But it offers a powerful alternative explanation for the existence of an achievement gap based on simple and realistic assumptions.</p>
<p>It also has an interesting practical implication. If, as I suspect, we are overestimating the ability gap, then middle-aged and older adults who begin pursuing competitive fields have nearly as much potential for groundbreaking achievements - Nobel prize-winning scientific breakthroughs, unicorn companies, and magnificent works of art - as their younger counterparts. The reason this rarely happens, I suspect, has less to do with biological limitations and more to do with risk-aversion and lifestyle creep. Let’s face it, PhD students, first-time entrepreneurs, and artists have it tough. How many 30-, or 40-, or 50-somethings would walk away from a decently-paid 9-5 to toil for years sharing a 3-bed flat with two guys they met on Craiglist and eating instant ramen every night for a slim shot at greatness? But those few are are willing to try, I suspect, have nearly as good a shot as anyone.</p>
<p>The rest of this post is dedicated to mathematizing this argument, as well as making two additional points. 1) The achievement gap will be larger in more competitive fields, 2) The achievement gap is usually larger when achievement depends more heavily on luck.</p>
<h2 id="formal-statement">Formal statement</h2>
<p>Consider a two-period model in which a person achieves something with quality $x_i$ for $i \in {0, 1}$. The achievements $x_i$ are independently distributed random variable with cumulative distribution functions $F_i$ and expected values $E[x_i] = \mu_i$. The achievement gap is $x_1 - x_0$ and the ability gap is $\mu_1 - \mu_0$. Suppose the observed achievement gap is the average ability gap $\Delta = E[x_1 - x_0] = \mu_1 - \mu_0$. If the observed achievement gap is negative (the period-1 achievement is lower quality on average), then it implies the ability gap is also negative.</p>
<table>
<tbody>
<tr>
<td>But now suppose that only the top $1 - \tau$ fraction of people are selected to achieve something in period 1 based on the quality of their period-0 achievement. Formally, suppose the observed achievement gap is instead the expected achievement gap of the selected individuals $\Delta = E[x_1 - x_0</td>
<td>x_0 > F_0^{-1}(\tau)] = \mu_1 - \mu_0 - l$ where $l = E[x_0</td>
<td>x_0 > F_0^{-1}(\tau)] - \mu_0$ is period-0 “luck”. Then, a negative observed achievement gap no longer implies a negative ability gap. It simply implies that ability cannot increase by more than $l$.</td>
</tr>
</tbody>
</table>
<p>Two additional conclusions follow from this setup.</p>
<p>First, more competitive fields (i.e., with a higher $\tau$) will have a larger observed achievement gap because $l$ increases with $\tau$.</p>
<p>Second, less obviously, fields in which period-0 achievements depend more heavily on luck will also have a larger observed achievement gap holding ability constant, provided the absolute quality of the period-0 achievement must be higher to advance to period 1. Formally, consider two fields in which period-0 achievement (which we’ll now simply denote $x$ going forward to avoid excess subscripts) follows distributions $\rho$ and $\nu$ with cumulative distributions functions $F_\rho$ and $F_\nu$ respectively. $\nu$ is a mean-preserving spread of $\rho$ (i.e., $\nu$ involves more luck). If $F_\rho^{-1}(\tau) < F_\nu^{-1}(\tau)$, then the magnitude of the observed achievement gap is larger under $\nu$.</p>
<p>The condition $F_\rho^{-1}(\tau) < F_\nu^{-1}(\tau)$ will be satisfied for sufficiently large values of $\tau$ under many reasonable distributions, such as when $\rho$ and $\nu$ are normal and $\tau > 0.5$. As an aside, it is easy to show that, in the normal case, the observed achievement gap will be larger under $\nu$ for all values of $\tau \in (0, 1)$ by taking the derivative of the conditional expectation of $x$ with respect to the standard deviation.</p>
<p>Proof.</p>
<p>Because $\nu$ is a mean-preserving spread of $\rho$, $E_{x \sim \rho}[u(x)] \geq E_{x \sim \nu}[u(x)]$ for all (weakly) concave, non-decreasing real-valued functions $u$.</p>
<p>Consider $u(x) = (x - F_\nu^{-1}(\tau)) * 1{x < F_\nu^{-1}(\tau)}$ where $1$ is the indicator function. $u$ is concave and non-decreasing.</p>
<p>Then for $\theta \in {\rho, \nu}$,</p>
\[E_{x \sim \theta}[u(x)]
= \int_{-\infty}^{\infty} (x - F_\nu^{-1}(\tau)) * 1\{x < F_\nu^{-1}(\tau)\} dF_\theta(x) \\
=
\int_{-\infty}^{F_\theta^{-1}(\tau)} x dF_\theta(x)
+ \int_{F_\theta^{-1}{\tau}}^{F_\nu^{-1}(\tau)} x dF_\theta(x)
- \int_{-\infty}^{F_\nu^{-1}(\tau)} F_\nu^{-1}(\tau) dF_\theta(x)
\\
=
\tau E_{x \sim \theta}[x | x < F_\theta^{-1}(\tau)] \\
+ (F_\theta(F_\nu^{-1}(\tau)) - \tau)E_{x \sim \theta}[x | F_\theta^{-1}(\tau) < x < F_\nu^{-1}(\tau)] \\
- F_\nu^{-1}(\tau) F_\theta(F_\nu^{-1}(\tau)).\]
<p>Note that when $\theta = \nu$ the above expression simplifies to</p>
\[\tau E_{x \sim \nu}[x | x < F_\nu^{-1}(\tau)]
- F_\nu^{-1}(\tau) \tau.\]
<p>Also note that</p>
\[\tau E_{x \sim \theta}[x | x < F_\theta^{-1}(\tau)]
= E_{x \sim \theta}[x] - (1 - \tau)E_{x \sim \theta}[x | x > F_\theta^{-1}(\tau)].\]
<p>Using $E_{x \sim \rho}[u(x)] \geq E_{x \sim \nu}[u(x)]$ and noting that $E_{x \sim \rho}[x] = E_{x \sim \nu}[x]$ by the definition of a mean-preserving spread, we obtain the following after some gnarly algebra:</p>
\[E_{x \sim \nu}[x | x > F_\nu^{-1}(\tau)] - E_{x \sim \rho}[x | x > F_\rho^{-1}(\tau)]
= l_\nu - l_\rho \\
\geq \frac{F_\rho(F_\nu^{-1}(\tau)) - \tau}{1 - \tau}
(F_\nu^{-1}(\tau) - E_{x \sim \rho}[x | F_\rho^{-1}(\tau) < x < F_\nu^{-1}(\tau)])
\geq 0\]
<p>where $l_\nu$ and $l_\rho$ are the “luck” required to advance to period 1 under $\nu$ and $\rho$, respectively.</p>
<p>Note that $F_\rho(F_\nu^{-1}(\tau)) - \tau \geq 0$ because $F_\nu^{-1}(\tau) > F_\rho^{-1}(\tau)$ and the cumulative distribution function is non-decreasing.</p>
<p>This result is intuitive. The more luck-dependent the quality of the achievement, the more luck is required to advance to period 1. Therefore, given that the abilities in both fields are the same, the more luck-dependent one will exhibit a larger observed achievement gap.</p>{"avatar"=>"/assets/images/profile.jpg", "bio"=>"I'm a PhD candidate at the Wharton School of Business. My goal is to elevate behavioral science using machine learning.", "links"=>[{"label"=>"GitHub", "icon"=>"fab fa-fw fa-github", "url"=>"https://github.com/dsbowen"}]}I used to have a fairly defeatist attitude about age and achievement. Groundbreaking achievements seem to come disproportionately from younger people; what I call the achievement gap. Nobel Laureates typically do their prizewinning work in their 20s and 30s. Successful entrepreneurs often start their companies before age 30. Artists make their greatest breakthroughs while they’re young (often with some help from “cognitive enhancers”).SQLAlchemy mutable2022-03-31T00:00:00+00:002022-03-31T00:00:00+00:00https://dsbowen.github.io/software/sqlalchemy-mutable<p>SQLAlchemy-Mutable aspires to be the most powerful and flexible <a href="https://www.sqlalchemy.org">SQLAlchemy</a> database column type.</p>
<p>Its features include:</p>
<ol>
<li>Nested mutation tracking</li>
<li>Mutation tracking for <code class="language-plaintext highlighter-rouge">list</code> and <code class="language-plaintext highlighter-rouge">dict</code></li>
<li>Support for storing database models in mutable columns</li>
<li>Support for common literals and <code class="language-plaintext highlighter-rouge">datetime</code> objects</li>
<li>Support for custom mutable classes</li>
<li>Support for converting existing classes to mutable classes</li>
</ol>
<p><a href="https://dsbowen.gitlab.io/sqlalchemy-mutable" target="_blank">Read the docs</a>.</p>{"avatar"=>"/assets/images/profile.jpg", "bio"=>"I'm a PhD candidate at the Wharton School of Business. My goal is to elevate behavioral science using machine learning.", "links"=>[{"label"=>"GitHub", "icon"=>"fab fa-fw fa-github", "url"=>"https://github.com/dsbowen"}]}SQLAlchemy-Mutable aspires to be the most powerful and flexible SQLAlchemy database column type.Hemlock2022-03-31T00:00:00+00:002022-03-31T00:00:00+00:00https://dsbowen.github.io/software/hemlock<p>Hemlock aspires to be the most powerful and flexible way to create online studies, with applications in marketing and behavioral science research.</p>
<p>I spent nearly every evening and weekend for the first two years of my PhD building this tool to facilitate my research, and the research of all behavioral scientists striving to push the boundaries of our field.</p>
<p><a href="https://dsbowen.gitlab.io/hemlock" target="_blank">Read the docs</a>.</p>{"avatar"=>"/assets/images/profile.jpg", "bio"=>"I'm a PhD candidate at the Wharton School of Business. My goal is to elevate behavioral science using machine learning.", "links"=>[{"label"=>"GitHub", "icon"=>"fab fa-fw fa-github", "url"=>"https://github.com/dsbowen"}]}Hemlock aspires to be the most powerful and flexible way to create online studies, with applications in marketing and behavioral science research.Hemlock meets dash2020-09-17T00:00:00+00:002020-09-17T00:00:00+00:00https://dsbowen.github.io/software/hemlock-dash<p>I’m excited to release a long-awaited hemlock update: the ability to embed <a href="https://plotly.com/dash/" target="_blank">dash applications</a> in hemlock surveys.</p>
<p><a href="https://dsbowen.github.io/hemlock/questions/dashboard/" target="_blank">Read the docs</a>.</p>{"avatar"=>"/assets/images/profile.jpg", "bio"=>"I'm a PhD candidate at the Wharton School of Business. My goal is to elevate behavioral science using machine learning.", "links"=>[{"label"=>"GitHub", "icon"=>"fab fa-fw fa-github", "url"=>"https://github.com/dsbowen"}]}I’m excited to release a long-awaited hemlock update: the ability to embed dash applications in hemlock surveys.Smoother2020-08-06T00:00:00+00:002020-08-06T00:00:00+00:00https://dsbowen.github.io/software/smoother<p>Smoother is a statistical tool for computing non-parametric distributions by optimizing a constrained smoothing function.</p>
<p>We often want to estimate a distribution given limited information, such as its mean and standard deviation. Rather than impose parametric assumptions, smoother uses non-parametric techniques to infer an entire distribution given limited information.</p>
<p><a href="https://dsbowen.github.io/smoother/" target="_blank">Read the docs</a>.</p>{"avatar"=>"/assets/images/profile.jpg", "bio"=>"I'm a PhD candidate at the Wharton School of Business. My goal is to elevate behavioral science using machine learning.", "links"=>[{"label"=>"GitHub", "icon"=>"fab fa-fw fa-github", "url"=>"https://github.com/dsbowen"}]}Smoother is a statistical tool for computing non-parametric distributions by optimizing a constrained smoothing function.Generalized Shapley Additive Explanations2020-05-19T00:00:00+00:002020-05-19T00:00:00+00:00https://dsbowen.github.io/software/publications/gshap<p>Generalized Shapley Additive Explanations (G-SHAP) is a technique in explainable AI for answering broad questions in machine learning.</p>
<h2 id="abstract">Abstract</h2>
<p>Many important questions about a model cannot be answered just explaining how much each feature contributes its output. To answer a broader set of questions, we generalize a popular, mathematically well-grounded explanation technique, Shapley Additive Explanations (SHAP). Our new method - Generalized Shapley Additive Explanations (G-SHAP) - produces many additional types of explanations including: 1) <em>General classification explanations</em>; Why is this sample more likely to belong to one class rather than another? 2) <em>Intergroup differences</em>; Why do our model’s predictions differ between groups of observations? 3) <em>Model failure</em>; Why does our model perform poorly on a given sample? We formally define these types of explanations and illustrate their practical use on real data.</p>
<p><a href="https://arxiv.org/pdf/2006.07155.pdf" target="_blank">Read the paper</a>.</p>
<p><a href="https://dsbowen.github.io/gshap/" target="_blank">Read the docs</a>.</p>{"avatar"=>"/assets/images/profile.jpg", "bio"=>"I'm a PhD candidate at the Wharton School of Business. My goal is to elevate behavioral science using machine learning.", "links"=>[{"label"=>"GitHub", "icon"=>"fab fa-fw fa-github", "url"=>"https://github.com/dsbowen"}]}Generalized Shapley Additive Explanations (G-SHAP) is a technique in explainable AI for answering broad questions in machine learning.