The Awesome Language Paradox

Language designers want to design the perfect language. They want to be able to say, “My language is perfect. It can do everything.” But it’s just plain impossible to design a perfect language, because there are two ways to look at a language. One way is by looking at what can be done with that language. The other is by looking at how we feel using that language—how we feel while programming. ¹

– Yukihiro Matsumoto

Say for the sake of argument you’re a programming language designer. Some time ago, you thought about the programming language landscape and how bad & ugly the big languages are. After years of hard work, your new programming language is finally done and ready for real-world use! It’s a well designed language: expressive, powerful, pragmatic, tooling you’re happy with, and it has a fast reference implementation, so you name it awesome-lang to reflect its awesomeness. Most importantly, you have a clear vision for where the project should go so you can prevent your language from being a bloated mess like C++. You’re certain that as soon as the world learns of your language, surely everyone will embrace awesome-lang for exactly what you designed it for (world conquest!).

You might be surprised to hear, then, that awesome-lang will almost certainly never go on to conquer the world, that it is almost guaranteed to fail to usurp whatever language you were clamoring to take down, and that it will likely die in relative obscurity.

Why is this? It’s because in order for a programming language to be successful, they first must defeat the Awesome Paradox.

Formally Defined Awesome Paradox

The Awesome Paradox is my attempt to define a set of properties that must be true about a new language in order to define it as “successful”. They are as follows:

• Requirement 1: You need people to learn your language:
  • 1A: It should be reasonably easy to learn without any weird syntax or complex new theoretical topics, or it should pass 1B
  • 1B: It should be forced upon people by a corporation/standards body, or it should overwhelmingly pass 2B so it’s the obvious choice for some problem
• Requirement 2: Your language needs a community healthy enough to justify use:
  • 2A: Build a critical mass of maintained, style-enforced, open-source libraries for others to use (meaning people coordinate together to build an ecosystem)
  • 2B: At least one library/framework/language feature should be a “killer feature” - something so good others will evangelize the feature and it will make people consider the language just to use it
  • 2C: The community should exist outside of the environment(s) of the people who made the language

There’s potential for endless bikeshedding about what success means, so let’s define success as “a language being well-used within the domain it’s targeted to be useful in”. Even this is relatively vague, but it’s a good enough working definition for this essay, and a more nuanced discussion is out of scope here.

Importantly, note that language quality is completely unrelated to its success. Otherwise, how do you explain PHP’s success?

Awesome Paradox Failures

In Paul Graham’s essay “Beating the Averages”², Graham discusses the idea that languages can be arranged on a hierarchy from the most powerful (the Lisp family, according to Graham) to the least. However, 20 years later, Lisp³ has not conquered the world, and the reason is simple: with great reductions in power comes great increases in coordination, and vice versa. Lisp is so powerful programmers have zero reason to coordinate on a similar style, which makes it difficult for a critical mass of libraries to emerge, and no top-down force ensuring coordination (unlike, say, Jane Street with OCaml: very flexible language with enforced style).

Even if the coordination problem were solved, there’s still the issue of actually teaching people to use these programming powers correctly. Consider Clojure, the answer to Lisp’s problems: with perfect Java FFI, it gets to piggyback on Java’s ecosystem, solving the coordination problem. However, many of Clojure’s features actually make it hard to pick up and write idiomatically for programmers unaccustomed to Lisps or functional programming (like macros, immutability, transactions, and especially multimethods), plus there is no compelling forcing factor, so it too fails the Awesome Paradox.

Perhaps the strongest case that power and clean design does not equal success is Haskell. It innovated features which appear in many places in the modern programming language space (typeclasses, monads, Maybe and typed errors, lazy eval, purely functional). It failed for a variety of reasons (bad tooling, no community consensus because it’s powerful, no forcing function) but its most instructive failure is that other languages began strip-mining it for parts. Every time another language bolts on Result<T> or lazy or more expressive type systems, the argument for switching to Haskell becomes a bit weaker. Smalltalk, ML, and F# are similar sad victims of the “bigger language steals all the good ideas and kills momentum in the originator” Awesome Paradox failure type.

Of course, failing the Paradox is not exclusive to languages which are powerful. Visual Basic (Classic and .NET), being blessed by Microsoft in the early 90s, was once one of the most popular languages of all time. As soon as Microsoft ended support Classic Visual Basic in 2008, it began a slow decline into obscurity; a similar fate was reached with Visual Basic .NET, which was put in maintenance mode in 2023.

There is an even longer list of languages which fail the Awesome Paradox more cleanly (like D, CoffeeScript, Vlang, Julia, etc), but those aren’t interesting to discuss. Determining if any given language fails is left as an exercise for the reader to do with a language of their choosing.

Awesome Paradox Successes

What types of languages manage to escape the Awesome Paradox, and what does that imply for them? When languages escape the Awesome Paradox, they usually go one of two ways: they can either focus in on dominating some niche where they become the de-facto choice, or they try to conquer the world, which can fundamentally never work out well.

Niche Domination

Consider Fortran as an unusual case: it’s widely mocked by those who don’t use it, it’s full of weird syntax, and it’s got a somewhat arcane ecosystem. However, its specialty is something it’s so overwhelmingly good at that it’s ensured Fortran will still be written in 50 years: fast linear algebra. No other language, not even C, rivals Fortran’s speed in this niche, and it’s been optimizing for it exclusively for many decades now. Every language where fast math is done links against BLAS and LAPACK, since nobody will seriously attempt a reimplementation. It beats the Awesome Paradox against all odds by choosing the niche path (although not intentionally, moreso a historical accident). Erlang/Elixir is a similar case, except for network programming instead.

There are some other types of “niche” success story worth mentioning before discussing the “conquer the world” types. Some successes come from the language being heavily embraced by a corporate benefactor, giving the language a clear use case (their platform) to solve the Paradox: examples include Java (historically), Kotlin (Android and the JVM), C# (.NET), and Swift (iOS/macOS). Sometimes a framework is the essential piece, like with Ruby (Rails) and Dart (Flutter). Possibly the rarest niche of all is when a language becomes significant because the language itself is the killer feature. This elusive niche includes languages like OCaml (for building compilers and extremely reliable systems), Lua (small, easy to embed, beloved by gamedevs), and arguably Fortran (its restrictions make it fast).

Conquer The World

Let’s consider Rust, the most recent world-conquering modern example. Rust checks many of the boxes and is genuinely new: it was forced by Mozilla onto its employees (early on), it quickly built numerous useful libraries, and its core feature, GC-less memory safety, is indeed very compelling. Rust has successfully defeated the Awesome Paradox and has become used across many domains, which is excellent news for it in the short term, but almost certainly bad news for its long-term future.

Why is that necessarily so? When communities grow, they inevitably bring in people who want to use the language for things it wasn’t being used for before. This results in an inevitable crisis, where these new people want something that the language doesn’t fundamentally have a good design answer to yet (possibly because the problem is unsolved broadly or is just difficult to add to the language). No standards body is perfect; Rust’s designers will make mistakes (they already have, in fact), even though they did avoid many historical ones by making good decisions (Editions, small std, standardizing the tooling and style with cargo).

C++ has been mocked for being huge and bloated, but this will likely also be true of Rust in the future. Rust is nowhere near complete, and there are several areas of the language which probably won’t become stable in a clean way. Async especially is a mess: executors are different, using it without a Pin<Box<dyn Future<T>>> is hard, and understanding how async desugars is weird (at least we finally have async closures and traits, but way too late, Tokio is now de facto std because of how long it took!). It’s frustrating that Rust suffers from function coloring all the way down to the types (although it is understandable given the tradeoffs they had to make). Error handling in Rust also isn’t aging well: there are so many different error solutions (thiserror, anyhow, snafu) and none of them really make the split between application-level and library-level errors clean.

Unlike C++, Rust will probably take longer to be bloated due to it learning from history, but it’s inevitably going to happen. When it does, new programming language research and historical wisdom will allow for some new, cleaner, simpler language to eat Rust’s breakfast. This pattern is not exclusive just to Rust: Go has had this growth creep too despite being initially very clean, most notably with the introduction of generics in a way somehow carefully crafted to be both weak and ugly at the same time, appeasing nobody. Everyone will be insulting these languages in 15-20 years like how people make fun of C++ now and made fun of COBOL back in the past.

Perhaps the only way to avoid such a fate is to allow a BDFL to run the language’s design, but denying changes for new use cases make it harder to grow. This is going well for Zig’s integrity but not for its success: Andrew consistently makes great choices for the language (which is why Bun dropped Zig) and it’s exactly the reason Zig will probably not conquer the world; he is the rare kind of person who would rather get it right the first time⁴. Guido was doing this for Python, before he left after many years of gradually burning out from constantly fighting this exact problem. So a BDFL is a lose-lose that doesn’t solve the core issue here: you get all the bad choices of having a committee make the choices but at a slower pace, or you get no changes and watch the language become niche or die out.

Implications For Newer Languages

My aim was for C++ to become the second best language for many things.⁵

– Bjarne Stroustrup

This is probably the worst time in the last few decades to be a new programming language, for a few reasons:

• The incumbent languages are all getting better (Python losing the GIL and getting better tooling, Ruby has a JIT, real improvements to C++, modern C and Fortran are good, even Java is much more pleasant these days)
• There are some missing areas of programming language research, particularly with things like side-effect tracking (Monads are cool but hard to use), lifetimes (need to be more ergonomic), gradual typing (Luau is working on this one), easy formal verification (Ada/SPARK went down this road and it’s somewhere Rust will need to go, too)
• As people become more dependent on AI tooling, new languages will be hard to use with AI tools due to a lack of high-quality training data, which entrenches existing languages
• Many niches are already filled well, or have a corporate favorite installed
• Rising expectations have made some features a requirement for new languages. Among them are some type of static typing, package management, and fast to no compile times.

So what should new language designers / young programming languages do?

• Prioritize having one niche where your language is simply better than all the incumbents, or offer something new nobody is doing in that space
• Make peace with the fact that your language will be a product of its time and surpassed (if you seek to conquer the world), or accept that it will be beautiful, elegant, and plundered for ideas by other languages (if you seek to conquer a niche)
• Ensure your language has good interop with C (or some other big language) so you can overcome the coordination problem with libraries; you can swap to native ones later
• Corporate benefactors are, despite first impressions, extremely important. Many languages that are successful today are not grassroots, but achieved their success thanks to a corporate sponsor/user, giving the language a real workload and stress-testing the language, allowing it to grow into something ready for real world use.

If I’d put my money on any upstart language right now to potentially escape the Awesome Paradox, it would probably be Gleam for a niche success. It overcomes the dynamic typing weakness of existing BEAM languages, it also compiles to {Java,Type}Script, has good tooling for both execution environments (plus FFI to both), and it’s minimal. In particular, Gleam’s real advantage over other languages is that it fundamentally can’t be “mined for parts” like other functional languages such as Haskell; BEAM and the browser being possible in the same language is a strength other languages can’t cleanly retrofit.

Conclusion

People should continue to develop new languages, but ideally should be focused on trying to build languages that are focused on niches rather than trying to resolve the much harder problem of big, general-purpose languages. As always with good language design, be careful to remember the Vasa!