Software I am grateful for

2021-11-25

Some of the things that come to mind, this Thanksgiving.

open/read/write/close

Most Unix-ish APIs, from files to sockets, are a bit of a mess these days. Endless poorly documented sockopts, unexpected changes in write semantics across FS and OS, good luck trying to figure out mtimes. But despite the mess, I can generally figure out the open/read/write/close. I can trace a binary and figure out the sequence and decipher what's going on. Sprinkle in some printfs and the state is quickly debuggable. Stack traces are useful!

There has been considerable effort on many projects to replace this style of I/O programming, for efficiency or aesthetic reasons, often with an asynchronous bent. I'm grateful for this old reliable synchronous open/read/write/close standby, and hope to see it revived and reinvented over the course of my career to be cleaner and simpler.

goroutines

Goroutines are coroutines with compiler/runtime optimized performance, to make them behave like threads. It breathes new life into the previous technology I'm grateful for: simple I/O blocking. With goroutines, it becomes inexpensive to write large-scale blocking servers without running out of OS resources (like heavy threads, on OSes where they are heavy, or FDs). It also allows the use of blocking interfaces between "threads" within a process without paying the ever-increasing price of context switching in the post-spectrum world.

Tailscale

This is the first year the team working on Tailscale has grown and eclipsed me to the point where I can be grateful for Tailscale without feeling like I'm thanking myself. Many of the wonderful new features that allow me to easily connect machines wherever they are, like Userspace Networking or MagicDNS, are not my doing. I am grateful for the product and the opportunity to work with the best team of engineers I have had the privilege of being part of.

SQLiteName

Like open/read/write/close, SQLite is an island of stability in an ever-changing technical landscape. The techniques I learned 10 or 15 years ago using SQLite work today. As a bonus, it does much more than that: WAL mode for highly concurrent servers, advanced SQL window functions, excellent ATTACH semantics. He did all of this while minimizing the number of "clunky design" decisions, in the project's own parlance, and staying true to his mission to be "lightweight." I aspire to write such wonderful software.

JSON

JSON is the worst form of encoding except for all the others that have been tried. It's complicated, but not too complicated. It is not easy for humans to read, but it can be read by humans. It is possible to extend it intuitively. When it's printed on your terminal, you can figure out what's going on without fetching the magic decoder ring of the week. This makes some extremely difficult things with XML or INI easier, without introducing accidental Turing completeness or turning country codes into Booleans. Software writing is better for this and shows the immense effect carefully describing something can do for programming. JSON was everywhere in our JavaScript before the term was defined, the definition allowed us to see it and use it elsewhere.

WireGuard

WireGuard is a great demonstration of why the sheer complexity of the implementation ends up affecting the UX of the product. In theory I could have tunneled between my devices for years with IPSec or TLS, in practice I had given up completely until something happened that made it easier. It didn't make it easy by putting an elegant user interface on top of complex technology, it simplified the underlying technology so that even I could (eventually) understand the setup. More importantly, by not eating up my entire complexity budget with its own internals, I could suddenly see it as a building block for larger projects. Complexity makes more things possible, and less things possible, simultaneously. WireGuard is a great example of simplicity and I'm grateful for that.

The speed of the Go compiler

Before Go became popular, the fast programming language compilers of the 90s had mostly been abandoned, to be replaced by a bimodal world of interpreters/JIT on one side and slow, squeaky compilers tempting to produce extremely optimal code on the other. The mainstream Go toolchain has found, or rediscovered, a new sweet spot in the trade-off plan for programming languages: compiled ahead of time, but with a less than optimal fast compiler. He's managed to maintain this interesting and unstable balance for a decade now, which is incredibly impressive. (For example, I would personally like to improve...