Lambda School's Incredibly Naive Incentive Alignment

Lambda School is the most incentive-aligned education in the world.

It is so incentive-aligned, that their fundraising announcement was titled “Lambda School Raises $74 million for Incentive-Aligned Education”. Another blog post, tilted “Taking Baby Steps Toward Incentive-Aligned Higher Education and Job Training” includes the phrases

  • With incentive alignment, schools don’t succeed unless their students do.
  • We’ll continue to share updates on incentive-aligned education
  • Incentive alignment between schools and students is the objective.
  • As long as the underlying principles of student protection and incentive alignment hold true, that’s what matters.
  • …create an education model where the incentives of the school and student are aligned
  • …keep financial risk low for students and maintain incentive alignment.

Lambda’s big innovation is the Income Share Agreement, a mechanism that allows students to pay the bootcamp as a percentage of their salary, rather than paying tuition upfront. This ensures that the bootcamp is free to attend until you get a job, and ensures that Lambda is extremely incentive aligned.

But is it actually?

Here’s the details of the Lambda deal:

  1. Students pay 17% of their salary for 2 years,
  2. Payments are capped at $30,000, no matter how much you’re making
  3. Students only pay if they’re making over $50,000

With this information, we can graph Lambda School’s 2-year returns against raw student income:

Those flat lines represent the cap and floor. The section in the middle is where Lambda Returns grow as a function of Student Income.

But remember that these returns come out of student income. So account for Lambda payments and taxation, here’s the same chart, but now with another line for Actual Student Income.

(Source)

That huge drop at $50,000 is the point where tuition payments kick in, and Actual Student Income drops off.

This is a classic example of a perverse incentive: students would actually earn more with a lower salary [1]. As payments and taxes scale with earnings, students aren’t back to the actual income they had at a raw salary of $49,000 until they hit a raw salary of $67,000, nearly $20,000 later.

Accordingly, we can divide the graph into 4 segments, and for each, ask if incentives are actually aligned:

  • 1) Not Aligned: Actual Student Income increases, but Lambda Returns do not.
  • 2) Misaligned: Actual Student Income drops, but Lambda Returns grow sharply.
  • 3) Aligned: Actual Student Income and Lambda Returns grow together.
  • 4) Not Aligned: Actual Student Income increases rapidly, but Lambda Returns are flat.

To summarize, the actual incentive-alignment portion of this graph only happens in section 3 (when raw income is between $67,000 and $88,000). Despite their repeated claims, Lambda School is not actually incentive-aligned.

Okay, fine, but does this actually matter? Maybe all salaries fall in that incentive-aligned range, and so in practice, Lambda is effectively incentive-aligned.

We can test this hypothesis against Lambda’s table of reported student outcomes:

We want to know how many students make between $67,000 and $88,000. Using a naive assumption of uniform distribution within buckets, we get 47 students, or 26% of the total population. [2]

To make a closer approximation, we can estimate an underlying normal distribution.

The range we’re curious about ($67k - $88k) doesn’t align with the bucketing, but we can make an educated guess by estimating the underlying normal distribution:

That’s with mean $72,000, SD $26,000. This is still naive, but seems to be a decent approximation, and probably better than the uniform assumption. Using this model, we estimate that 31% of graduates are earning $67,000 - $88,000.

To sum up, Lambda School appears to be incentive-aligned with around one third of their students.

While Lambda School is not properly incentivized to improve income for the majority of its students, you might still argue that it is at least incentivized to get them a job.

This is fair, but also true of every other educational institution. Flatiron School releases data on job placement outcomes, as does General Assembly, and does every university. It doesn’t matter if you literally get paid as a portion of student income, every institution has a reputation to uphold, and a case to make for its value.

It is true that Lambda School only charges tuition if you get a job earning more than $50,000, and that’s great. For many students however, I expect this calculation to be dominated by the question of whether or not they’ll actually have a job at the end, a matter on which Lambda School has been notoriously deceptive.

[1] This concept of being punished for earning money is at the heart of the libertarian critique of taxes. It’s absolutely bizarre that Lambda has appropriated libertarian rhetoric without any of the accomplaying economics.

[2] (75 - 67) / 25 * 71 + (88 - 75) / 25 * 46

Austen Allred is Consistently Deceptive

Lambda School is a coding bootcamp that only charges tuition if you get a job. Founder/CEO Austen Allred frequently takes to Twitter, defending his bootcamp against allegations of fraud, and rebutting critics with case after case of student success.

I was initially excited about Lambda School, but have slowly grown disillusioned over time. So when they released their 2019 H1 Outcomes Report, I was excited to finally access a ground truth and put an end to all the speculation.

Instead, I found a consistent pattern of deception.

In the rest of this piece, I’ll walkthrough a number of examples, in which Allred:

  • Claims a job placement rate of 86%, when the actual number is as low as 55%, and at least as bad at 70%
  • Misrepresents graduate salaries on Twitter, despite claiming a random sample
  • Calls regulatory approval a “significant endorsement”, despite a troubled history of bans
  • Lies about having been homeless

Some of these are blatant and explicit, some are more subtle. I’ve done my best to present the facts fairly, and leave the rest up to your judgement.

1. Allred Misrepresents Student Outcomes

In early 2019, Allred took to Twitter to share student outcomes, listing the following salaries:

  1. J: $80,000+
  2. S: $80k
  3. D: $110k base, ($130k total)
  4. C: $130k
  5. T: $90k
  6. L: $140k
  7. J: $85k
  8. L: $89k
  9. A: $85k
  10. R: $75k
  11. S: $70k

A year later, when the Outcomes Report was released, it cited a median salary of just $70,000. In stark contrast, Allred’s 11 Tweets report exactly 0 students making less than $70,000. Statistically, that’s like flipping a coin and getting 11 heads in a row (probability .00049).

This seems bad, but whatever, Allred is just highlighting a few heartwarming anecdotes right? Surely he doesn’t actually claim that this is a randomly selected sample?

Except he absolutely does:

To see just how poorly Allred’s claims line up against the official Outcomes Report, we can plot the histograms side by side:

Allred claims that his examples were randomly selected, but his own statements are contradicted by the official outcomes report.

2. Allred Repeatedly Misrepresents Job Placement Rates

Earlier this year, New York Magazine reported that Lambda school had marketed itself as having an 86% job placement rate, but then released an investor memo reporting a much lower rate of just 50%.

Allred took to podcasts, explaining in an interview with Jason Calacanis:

When we talk to investors, we talk in terms of enrolled students… of those students who were enrolled, X% were hired. Whereas when schools speak publicly, they speak in terms of graduated students. So obviously those two numbers are always going to be different.

He goes on to clarify:

That 86% was somewhat out of date… so we quickly put together what the real numbers would have been, which was 78%.

Okay, so that seems reasonable enough. Case closed?

But then the official Outcomes Report was released, and contradicted Allred’s own statements:

  • Of the 448 students in the cohort, only 318 graduated
  • Of those 318, only 284 graduated on time
  • Of 284 graduates, Lambda could only reach 255
  • Of those 255, only 201 had jobs
  • Of those 201, Lambda “did not have salary data… due to our data collection methods”, giving us data for 178 students

The job placement rate for enrolled students is 201 out of 448, or 45%. The job placement rate for graduated students is 201/318, or 63%.

It actually gets worse. The original Lambda School claim was:

86% of Lambda School graduates are hired within 6 months and make over $50k a year

But according to the outcomes report, 26 of the placed graduates are making under $50,000. The actual placement rate for graduates making over $50,000 is just 55%. That’s a far cry from the 86% they claim, and much closer to the 50% reported by New York Magazine.

To be clear about timing, the New York Magazine article was published February 19th 2020, the interview was on March 3rd, and the Outcomes Report was published March 28th 2020. So these should all be about the same cohort, or at least a similar set of students.

So where does the 78% come from? Following article, Allred published a note on LinkedIn with the following chart:

So 78% is what you get if you ignore students who graduated late, ignore students who have yet to graduate, ignore the “graduated and disengaged” students, and pretend everyone is making over $50,000.

To his credit, Allred does provide another table which includes disengaged sudents and reports a 70% placement rate. Not only is this a far cry from the 86% originally reported, it also includes the 15% of placed students making under $50,000.

Although it may seem fair to count late graduates in the H2 report, note that this is a misrepresentive sample. Students who graduate late will likely have worse results later on, and underperform students who graduated on time. If Lambda wanted to account for this, they could have included 2018 H2 late graduates in the 2019 H1 report, but they neglected to do so.

In March 2019, Allred wrote:

Regulators love us - they’re sick of for profit schools promising a lot and delivering little except debt. [emphasis mine]

December the same year, it was revealed that Lambda was actually operating illegally, with no approval from regulators.

Okay, so this looks bad, but it doesn’t mean Allred is lying. It’s entirely possible that he thought regulators loved Lambda in March, and wasn’t corrected until a $75,000 fine hit the following month.

When Lambda finally did get approval to operate in California, Allred wrote

Over the past year, Lambda School worked to advance an ambitious goal: become the first online school approved by California regulators to offer ISAs. We achieved a major victory when state regulators licensed us as a school in August, representing a significant endorsement for our all-online, career-focused education model. However, in order to secure this approval, we made the difficult decision to stop offering our ISA option to students in California.

To be clear, none of this is an outright lie, but it is a dishonest representation of the regulatory environment. To recap:

Then Lambda applies for approval, leading to this saga in which:

  • July 2019: Regulators deny the application, ordering Lambda to cease operations
  • Lambda continues operations anyway
  • December 2019: Lambda re-applies for approval, is told to cease operations
  • Lambda continues operations anyway
  • June 2020: Lambda submits another application, regulators say they cannot use ISAs
  • Lambda finally agrees to stop using ISAs in California
  • October 2020: Allred calls regulator approval a “major victory” and “significant endorsement”

As best I can surmise, those are the facts of this case. You can decide for yourself if you think Allred is acting with integrity.

Bonus: Allred Lies about Experiencing Homelessness

This has nothing to do with Lambda School, and is admitadely somewhat gratuitous. But it does point to continued dishonesty on the part of Allred, and is too blatant to ignore.

In 2017, Allred tweeted:

I was homeless, had no skills, no degree, just $300 and a laptop, but that’s all you need.

In a 2018 Hacker News comment, he wrote:

I’m also formerly homeless… I know how hard it is to focus on getting a job when you’re just trying to survive

An article includes a screenshot of an Allred’s reply to a tweet which reads:

I have been homeless, slept on those same streets [1]

The arc from homeless to multi-millionaire founder would be inspiring, except that it isn’t actually true.

In a 2013 Hacker News comment. Allred writes:

I lived in a Honda Civic this summer as I was getting a startup off the ground… and I had a half dozen people offer to let me stay at their place or crash on their couch rent-free.

…As a result, I wasn’t living in a car for lack of other options, but rather out of belief that I could create something by sheer will-power, and that I was going to do that come hell or high water. My homelessness was a matter of seeking something greater than myself, not being lost to poverty. [emphasis mine]

Similarly, in a now-deleted blog post titled Voluntary Homeless in Silicon Valley Allred writes:

Candidly, living in a car in Silicon Valley had much more appeal to me than a single bed and a shared bathroom… Much more than money, what fuels me is obsession with minimalism, reading way too much Thoreau, and trying to continually see life from a different angles…  It’s about questioning society at its fundamentals, and seeing what a life not tied down by a foundation really feels like. And so far I love it.

So again, Allred’s claims are… not technically a lie? I mean he did sleep out of a car, but he did it voluntarily. You’re free to make your own judgements, but when I hear “I have been homeless, slept on those same streets”, I do not think of someone with half a dozen offers of free housing. That isn’t poverty, it’s cosplay.

Conclusion: Why does any of this matter?

Am I just piling onto an already heavily criticized company? Am I guilty of being a critic rather than a creator?

I initially wrote about this purely using Lambda School as an example to illustrate a broader point about incentive alignment. But the more I read, the clearer it became that they were just consistently dishonest. Not outright fraudulent, but just really misleading about everything from student outcomes, to regulatory pressure, to the founder’s own history.

Still, why does any of this matter if Lambda is genuinely educating and helping students?

First of all, despite the outcomes report, we still have no idea what the median outcome actually is! They’re reporting data for 178 out of 448 enrolled students, which is just 40%. According to the report, the other 60% of enrolled students either:

  • Remain unemployed (54 students, 12% of cohort)
  • Graduated late or not at all (164 students, 37%, of cohort)
  • Became mysteriously unreachable, or went mysteriously uncounted (52 students, 12% of cohort)

And none of them are included in the salary calculation. So what’s the actual median outcome? We have no idea.

Second, if their outcomes are actually good, why do they have to constantly lie? As Vitalik once said:

If you have a good way of proving something, and a noisy way of proving something and, you choose the noisy way, that means chances are, it’s because you couldn’t do the good way in the first place.

If Lambda actually cares about transparency, they should just report:

  • How many students have ever enrolled
  • How many students have been placed in jobs earning over $50,000
  • Average and median time to placement

And then I’ll happilly shut up and accept that exagerated marketing is sometimes required to make good things happen. Until then, pointing out that their transparency report is not actualy transparent and their anecdotes not actually representative is fair game.

[1] Allred’s tweet is deleted, but the original is still up. There is another tweet replying to Allred, suggesting that he did have a reply before deleting it.

Coda:

It’s me, I’m the asshole.

EDIT 11/19/2020:

A few more things to mention:

  • Lambda Fellows launched today, a seemingly illegal unpaid internship. Because nothing screems “valuable skills” like giving away labor for free.
  • Lambda’s Outcomes Report is descirbed as “biannual”, but over 6 months since publication, there has been no news of a subsequent report for the 2019 H2 cohort.
  • One more example of questionable honesty: Allred claims on Hacker News that he canceled a cohort’s ISAs because it was “the right thing to do”, but a student from the cohort claims it took “months of work” and an greement not to sue.

Transcendance and Material Gain in Queen's Gambit

(Spoilers for Queen’s Gambit, Pan’s Labyrinth, Monsier Lazhar and Seven Samurai.)

Queen’s Gambit errs in the same way as every other American show: Characters develop spiritually, but only for the sake of material gain.

It’s a narrative analogous to the real-world misappropriation of self-care-as-productivity-hack. You mediate not to find inner peace, but to leverage that peace into productive output.

Similarly, the final episode takes us through Harmon’s recovery from alcoholism and reconnection with old friends. But these successes are not considered intrinsically valuable. They exist only for the sake of delivering a final victory against the Soviets.

In the penultimate episode, she is largely sober, crushing every player before her. Then, in a moment of weakness, she goes out drinking, wakes up hungover, and loses horrendously to Borgov.

In the final episode, she recovers from her alcoholism, flushes her pills down the toilet, reconnects with friends, and crushes Borgov in the final match.

Stated otherwise: it is only through appeal to material gain that her sobriety is considered valuable.

To be clear, this narrative is not inevitable. As far as I know, no other national cinema insists on it so consistently.

Consider the French Monsieur Lazhar where a teacher forms a genuine bond with his students, but then gets fired anyway. Or the Japanese Seven Samurai where elite warriors learn to understand the lives of peasants, but then mostly die anyway.

Or consider the Mexican Pan’s Labyrinth which actually inverts the trope by having Ofelia die in the material world, but then rule as a Princess in the underworld. This is an actual spiritual narrative, one that places inner lives first, and the material world second.

Instead, we get the American version where not even friendship can be an end unto itself.

Appendix: Against the Umbrella of Substance Abuse

Nearly every review I’ve read conflates Harmon’s use of alcohol and Librium under the umbrella of “substance abuse” or “addiction”. That’s an easy reading, but it’s totally undermined by the text itself.

Alcohol, specifically, is responsible for her loss to Borgov in Paris, her subsequent slump, and her mother’s death from hepatitis.

In contrast, Librium never causes Harmon to lose a game. The only time it becomes problematic is when she’s forced off of it and goes into withdrawal.

Reviewers’ inability to distinguish between the two, points at a disturbing whitewashing of the show, and an unwillingness to accept the validity of drug-induced mysticism

Harmon’s childhood genius and accompanying hallucinations were the result of Librium, and was the vast majority of her career. When she quits drinking in the final episode, she doesn’t quit taking pills, instead bringing them with her to Moscow.

She does flush them down the toilet before the final match, but then acquires more. Whether or not she takes them is left ambiguous, but there are ample clues:

  • Harmon says she can’t visualize the board without Librium
  • Before the final match, she tells Townes that she’s asked the front desk for pills
  • The next morning, Townes wakes her late in the afternoon, saying “Lady, you were out!”

And then in the final match, Harmon goes on to hallucinate pieces on the ceiling, which only happens under the influence of Librium.

There is no evidence either, that Librium has a negative effect on her mother, or any of the other students at the orphanage.

I’m not making any kind of moral judgement on drug use, nor suggest that anyone in real life should take prescription drugs.

But within the show, the message is clear. Medication helps Beth win, and has no discernible negative side effects.