An Algorithmic Lucidity

a blog

Category: meta

Blogging Technology Interlude

When I started An Algorithmic Lucidity back in 2011 when I didn't know anything about computers, I used WordPress—briefly on wordpress.com, but then on my own site on Namecheap's true-to-its-brandname shared hosting service.

WordPress Media Library edit screen for a file named reciprocalx2.png, with arrows pointing to the File type and File URL fields to show the appended "2"

Even early on, the technical limitations of WordPress were chafing. In my first post, "The Derivative of the Natural Logarithm", I included images of graphs of \(y = \log x\) (lnx.png) and \(y = \frac{1}{x}\) (reciprocalx.png), but I must have tried to re-upload edited versions that (I can only infer) WordPress automatically renamed to avoid filename collisions, because I somehow ended up with images called lnx3.png and reciprocalx2.png in my "Media Library" whose filenames I couldn't edit (the "File URL" field being "grayed out" in the GUI). Maybe it was a well-intentioned technical limitation—you don't want your GUI to let people edit filenames that might break references elsewhere—but it still felt like something precious had been stolen from me.

I want to control my work! The human mind is too small to micromanage every byte, and we wouldn't want to, but we can at least stive to have clean interfaces to what is there whenever possible. The same impulse condemns WYSIWYG editors, which is why I soon took to writing my posts in Markdown and then pasting the converted HTML into WordPress's "Code Editor". How can people stand not knowing what lies beneath the friendly illusion of paper?

By the time I started my gender-politics blog in 2016, I knew a little more about computers and used the Pelican static-site generator, which suited my taste much better: my posts are canonically in Markdown and the entire site is versioned in Git. If I want to change how something works, it's all Python. I can control my work with tools that I know how to use.

The previous WordPress site, September 2012

Meanwhile, while this site stayed functional, it felt shabby to work with. As I grew as a writer and started writing longer and more serious (and non-gender-political) essays from 2019, most of them went up as "exclusives" on Less Wrong (and merely linkposted from my own site, if that), unless something seemed more personal or less rationality-flavored, in which case the canonical copy lived here and Less Wrong got a crosspost/linkpost if I thought it would be of interest there.

My tolerance for the old WordPress stack ran out in March of this year when my "Terrified Comments on Corrigibility in Claude's Constitution" failed to post here: the request just hung when I tried to save the post. I don't know what went wrong, but it wasn't even worth debugging: I put up "Terrified Comments on Corrigibility" as a Less Wrong exclusive and vowed to throw out WordPress and switch to Pelican soon.

Pelican already shipped with a tool for importing from other blogging systems (including, obviously, WordPress), but I didn't like that the Markdown it generated from WordPress's HTML used asterisks (*) rather than underscores (_) for emphasis, and it turned out that the emphasis character isn't configurable in Pandoc (which pelican-importer was using). If I had been doing the conversion even just a couple years ago, the lack of configurability would have presented me with an uncomfortable trade-off (either eat the asterisks, or accept a significantly more labor-intensive conversion process), but in our terrifying new era of agentic coding, I just had Claude Code do the conversion with markdownify (which does have a strong_em_symbol parameter) and trusted that sanding down the edge-cases I hit by forgoing the standard battle-hardened importer could also be substantially delegated to Claude Code.

The full conversion (including styling the new site to mostly look like the old one, deployment to a DigitalOcean VPS, &c.) still took a fair amount of work (the bulk of it during two full days), but much of it was supervisory in nature: the same work would have taken a lot longer (or would have been somewhat lower quality) if I'd had to personally master the intricacies of CSS and Nginx site configuration and unfortunate Markdown/MathJax interactions rather than telling Sonnet 5 what I wanted to happen, asking questions, and clarifying when I didn't like the result. There were several straightforward tasks that I wouldn't have needed to learn anything to do myself, like porting over most of my Less Wrong exclusives (which I mostly had Markdown source for, scattered across two Git repos) to live here, but delegating to Claude was easier.

I'm pleased with the result. I'll be happier writing this way, and it was an opportunity to add cool enhancements. (The site responds to HTTPS now. We're serving Markdown alternatives in case any agents or crawlers prefer that. I'm using /20XX/Jul/ rather than /20XX/07/ paths now, but that doesn't break any URLs because Nginx is issuing redirects.)

I didn't even bother with setting up comments for now (I'm using Isso on the other blog), but comments will probably be back soon. It's not like it's hard anymore.

An Algorithmic Lucidity Surprise Reader Meetup

I'm planning on going to BABSCon next week! I feel like I don't know how many readers this blog actually has: there's a CPanel counter, but so much software these days is just so buggy (remember, TheDailyWTF is not fiction), that I don't know if I can trust it. Maybe I'll see some of you there??

2016 Year in Reverse

(Previously, previously, previously.)

So, 2016 was an interesting year for me. (It was an interesting year for a lot of people.) I moved to Berkeley, landed 19 commits in the Rust compiler, and had my trust in the sanity of the people around me re-shattered into a thousand thousand bloody fragments. (The pain of this last is exactly what I deserve for allowing the trust to re-form after the last time.)

In 2016, this blog saw 77 posts and 65 comments. Among these—

It turns out that not everypony can be like Applejack. I have money. I went to a Star Trek convention. Bros will be bros. I took a break from blogging following a moment of liberating clarity. I went to RustConf and San Francisco Comic-Con. Some have concerns. You miss your beloved exactly because you can't model them well enough to know what it would be like if they were here. Destined best-friends-"forever" are still subject to the moral law. The map is usually not the territory. Living well truly is the best revenge. It turns out that thinking sanely about politics has a surprising number of parallels with writing a chess engine for fun (although the former activity is far less common). I'm kind of an asshole. There exist some less-common reasons to detest American football. If you wait too long to write something, you might lose your chance. Trade-offs and competitive forces continue to shape our lives even if we would prefer they somehow didn't.

What can readers of An Algorithmic Lucidity look forward to in 2017?

Well, it's looking to be a really exciting year for me, both intellectually and biochemically, for reasons that I can't talk about because I'm trying to minimize the sum of number of friends lost and bricks thrown through my window. (Only two so far!)

So, I don't know; maybe this'll become a math blog or something.

Hiatus I

An Algorithmic Lucidity is going on hiatus until December 1! There will be no new posts in November and the remainder of October. Thanks for reading, and hope to see you back in eight weeks!

Ideas Have Expirations

One often-overlooked aspect of the crime of not-writing is that the harm isn't just about the things that deserve to be said that you never get around to saying because you don't put in the time and effort. It's also about the things that you can't say anymore even if you suddenly had the will, because the opportunity to say it was bound to a particular time, and trying to recapitulate the thoughts months or years after the fact would be irrelevant, or impossible.

This phenomenon comes in degrees. Start with irrelevance. Often the inadmissibility of tardy words isn't absolute: you could say things late, but the product would be less valuable than if it were timely—especially in a medium like blogging, where the posts being dated and displayed reverse-chronologically creates an expectation that the entries are associated with a particular point in time—at least, that they were written not too long before their publication date, even if the actual content isn't about the ephemera of the day or season. This has contributed to An Algorithmic Lucidity not being as good of a blog as it could be.

BABSCon swag

Like—each of the last two Aprils, I attended (one day of) BABSCon, the San Francisco Bay Area's premier convention for fans of the animated series My Little Pony: Friendship Is Magic, both times—even the second!—with the thought that the experience would make good fodder for a blog post in the "autobiographical account of my day at this timely Special Event" genre. I was going to tell you about how the first time, I made a couple of social faux pas while meeting Tara Strong and Nicole Oliver, only one of which was intentional; I was going to tell you about how the second time, I hadn't been planning to buy anything in vendor hall, but couldn't help but say "Shut up and take my money" in response to the demonstration of the Twilight Sparkle's Secret Shipfic Folder card game, the game-opening card of which I later got autographed by both members of Sherclop Pones, whose Friendship Is Witchcraft fandub series had clearly inspired some of the cards and probably the game itself, as well as being the source of some of my favorite music. I was going to tell you about how the first time, I was considering buying a coffee cup depicting the Mane Six (because I frequently bought medicinals at the outpost of the American coffee hegemon, and felt guilty about the wastefulness of accepting the default disposable cup every time like every other American), but hesitated, explaining to the vendor that I wasn't sure whether I wanted to use that cup in public, whereupon she said she could throw in a free button, to which I replied, "Sold!" And I was going to do a careful sociological analysis of curious observations like how I hesitated to buy that cup and why H. liked it so much in terms of gender ideology and signaling contrarianism.

But I didn't, and those Aprils were nine and twenty-one months ago, respectively. Not timely. My being motivated to write now ("That my past does not define me, 'cause my past is not today") can push a 350-word counterfactual "postview" of what I thought about saying over the threshold into existence, but it would be a bit unseemly to try to construct (I almost wrote reconstruct, but the re- prefix should be reserved for things that ever existed in the first place) a 2000-word personal account of the timely Special Event that happened last year or the year before that.

screenshot of the Leafline web client mid-game, showing the chess board and a scrolling move log

Or like—I made a thing not too long ago. I didn't mean to. It was an accident, really. It's a sort of oppositional strategy game engine—like a board game you play against the computer. The core move-scoring application is written in Rust, but there's also a web-application GUI (the "web client") in Clojure and ECMAScript 6 that's much more ergonomic for playing a game against. The game is—okay, well, it's chess: the endeavor snowballed out of my desire to participate in my coworkers' friendly office chess games combined with my reluctance to spend effort learning to be good at a task whose essential nature is so obviously suitable for automation. But writing a chess engine is just so cliché, and I enjoy naming things, so I quickly settled upon the conceit that actually I was writing an engine for a game that just happened to be exactly like chess, except that everything has different names: for example, instead of the pieces being black and white pawn, knight, bishop, rook, queen, and king, the figurines, or agents (never "pieces"), in my game are blue and orange servant, pony, scholar, cop, princess, and figurehead. (I kept rank and file, though.) This "adorable idiosyncratic names for everything" convention permeates the codebase. Despite the fact that unnecessarily gendering things isn't normally my style, I decided that ponies, scholars, and (inevitably) princesses were female, and that servants, cops, and figureheads were male, specifically in order to have an excuse to leave // ♀ and // ♂ comments in the enum declaration, because I feel like too many of my code comments are restricted to the ASCII or maybe Latin-1 subsets of the Basic Multilingual Plane, such that I don't want to pass up an opportunity to legitimately ("legitimately") use \u2640 FEMALE SIGN and \u2642 MALE SIGN. There are comments with ludicrous rationalizations for using the standard Forsyth–Edwards notation abbreviations ('P' is for peon, 'N' is for neigh, 'B' is for book, 'R' is for the Rule of law; the blue team's runes are lowercase because lowercase characters have higher ASCII codepoints, just as blue light has a higher frequency than orange light). Instead of pawn promotion, it's servant ascension, and a different verb is used to describe the process depending on the target figurine type (the servant moving the to the final rank can transform into a pony, be brevetted to cop, or transition into a princess or scholar). Or, like, Rust is quite conservative in the Yeggean sense, as manifested in features like the compiler forcing you to explicitly handle all possible variants of an enum; in cases where you know an instance of the enum can only be one of a strict subset of the possible variants, but the type system doesn't know that, you have to fill in those branches or supply a default case anyway, probably with a panic, which is like an exception that you can't catch, to abort the program with a message indicating that something entirely unanticipated has happened and it makes no sense to carry on. So I had been ending all of my panic messages with a stock phrase about how the unexpected thing was contrary to the operation of the moral law ("non-princesslike agent passed to princesslike\_lookahead, which is contrary to the operation of the moral law", and so forth), because this isn't the kind of codebase where you just say something like "assertion failed," as if the program should die because of some human authority's mere assertion, rather than only as a matter of justice when its behavior is contrary to the operation of the moral law. And then when my coworker Alexander Corwin started contributing, he abstracted away the boilerplate in these panic messages into this moral_panic! macro, which I thought was brilliant.

The web client is configured to run on port 2882. It's easy to remember because 2882 is Magnus Carlsen's peak ELO score.

terminal output from a Leafline console session, listing move alternatives with scores and principal variations

Software, like poetry, is never really finished (only abandoned), but getting the project to the point that I felt it was a minimal viable "product" took about eight glorious weekends (plus a few weekday nights, and with some help from Alexander). Near the end, watching the program utterly trounce me in web client play and feeling for all the world like a lieutenant junior grade, I started looking ahead to what came next for me. Over the past two years, the first two years of my life (that I feel comfortable admitting to), I had spent many, many night and weekend hours hacking on various side projects and coding exercises out of genuine enthusiasm and curiosity and desire to improve my craft—and, honestly, a feeling of insecurity, sensing that I needed to prove my worth as a hacker ("I threw myself into my studies, to have the world in my control"). But decent chess AI in Rust as an impulsive throwaway project seemed like a sufficiently strong signal of my programming prowess that maybe it was time to tie off this project, write the obligatory exciting blog post about it, and start allocating night and weekend hours towards some of the non-programming (!) interests I remember having in the before-time (assuming those memories are not fake). I could do some math! Write some fiction! Maybe even meet new friends ("I will not be shy; I'm going to try; I bet I'll find the reason why so many people, like me—")?!—all while feeling secure in the knowledge that my technology skills are clearly adequate for my continued existence to be economically viable. (For now.)

So ... about that end-of-project blog post. I was going to title it "Project Review: Leafline version 0.0.14-MVP; or, Lessons From Writing an It's-Not-Chess Engine in Eight Weekends", and I was going to explain everything to you—not just the silly names for everything that inexplicably amuse me (and only me), but the actually interesting substantive technical details of the implementation. Not that the AI is anything special—it's just textbook minimax search (in the negamax style) with α–β pruning and a position-evaluator that mostly counts material but also has bonuses for things like having both scholars or ponies and servants being in the center sixteen squares, plus a transposition (hash)table to look up scores of game-states it's already seen before, and another hashtable for history-heuristic move ordering. It's multithreaded, to take advantage of multiple cores (although I confess that the specific threading strategy is kind of a questionable hack). It can search up to 7 plies if you have a decent machine and are willing to wait ten or twenty minutes for the answer.

But even if it's nothing special, I was going to tell you about all the deeply moving philosophical insights I got from doing it. It's one thing to know how to refute the classic anti-AI argument (or straw person) that "humans couldn't possibly build something smarter than themselves by definition," but it's another thing to have the personal experience of having written something smarter than yourself in some particular domain, to the extent of feeling an acute sensation of futility while playtesting it, thinking: I've already coded my understanding of what it means to be good at this game; obviously I'm not going to do any better thinking about it with my slow meat-brain. (Although this is partially explained by my knowing almost nothing about chess strategy that I didn't learn in the course of this project; I've seen one of my coworkers beat the program at 4-ply search very quickly.) I was going to explain to you how α–β pruning works, with hand-drawn diagrams, and how you can interpret it as disregarding possible worlds that are too good or too terrible to be true (given rational play on both sides). I was going to tell you about the exciting surprise where the program seems to behave as if it understands the concepts of pinning and forking, even though those ideas aren't represented anywhere in the code!—but that when you think about it, that shouldn't actually be surprising: to the extent that we think pinning an opposing piece is a good idea that will lead to us picking up material, then pins should naturally appear in the results of searching the game tree for moves that are predicted to pick up material. And the reason we search for game-states where we hold a material advantage is because we expect that we're more likely to mate from those positions (in future nodes beyond our current planning horizon). Things that are, in all philosophical strictness, mere instrumental values, might profitably be treated as if they were terminal values by some algorithm that can't see far enough ahead to the actual goal, and this is a quantitative phenomenon: the shorter your lookahead, the more you want to rely on near-term approximating rewards. That's why I threw in a bonus for servants advancing to far ranks; I suspected that the existing code searching at the not-greater-than-7 plies that it could get in a reasonable amount of time, wouldn't adequately appreciate the true value of servant ascension, even though I expect that value would be naturally emergent in a deeper search, just as the value of some forks and pins emerged from mere 4- or 5-ply searches. (This was just a suspicion, though; I didn't take the time to actually test ascension-seeking behavior.)

So—though I've managed to just now haphazardly summarize some of the things about this project, this isn't really a careful project-review post. And now the distance between today and 27 September's 0.0.14-MVP tag is longer than that between that tag and the start of the project. And I ended up making a few more commits in subsequent weeks. So, no longer timely, at least not the way I originally imagined publishing a grand just-finished-project review post as a symbolic milestone marking a transition in how I'm going to start spending my precious non-dayjob time.

Or like—in late October, I saw the recent Jem and the Holograms film, because the previews (accurately) made it look like it was going to be really bad, and I thought I'd write a post combining a negative review of the new film with praise for the original Jem cartoon that this awful film mendaciously desecrated the name of. And then ... I didn't get around to finishing the post. Maybe my notes and memory from that night at the theater are accurate enough such that it's not too late—but the potential impact departed with the timeliness; I can't warn you not to go see it, because it's not in theaters anymore. (And good riddance.)

But beyond these sad cases of ideas that didn't get written up properly while they were still maximally relevant, there's an even worse way to fail to communicate, which is when the would-have-been-author has changed so much since first having the idea, that there's no way they could plausibly do justice to the idea as it was first had. You can't write a Diary entry about the day five years ago that you don't remember, and—more poignantly—you can't write a grand ideologically-driven novel in favor of the ideas you don't believe anymore. Some would argue it's just as well—if it's something you wouldn't write and couldn't stand by today, aren't you relieved that it doesn't exist to sully the name that represents who you are today? Even so, I would still favor intertemporal solidarity among past and future selves against the common enemy of our illiteracy.

Yes, illiteracy! Some would call it writer's block, but I know better than to bother with the unobservable distinction—whether you choose to describe your hypothesis as "doesn't write because unmotivated" or "doesn't write because doesn't know how," the result is the same: death of the non-author's memetic lineage.

2015 Year In Reverse

(Previously, previously.)

Dear reader (that's reader in the singular because I doubt that there are two actual humans who read my blog; it's not because there are many and I'm addressing you individually), it's that time of the year again—the time of the year when it stops being the year. A time to think thoughtfully and ask, "What happened since the last time we did this?"

In the year 2015, this blog has seen (at press time) 43 posts and 30 comments. Among these—

We shared a recipe for a more-wholesome-than-some-of-the-likely-alternatives breakfast (although honestly, I worry about coffee having unknown-to-me degenerative effects on the brain—just because lots of people in your Society indulge in a drug doesn't mean it's safe; it just means that any long-term damage has to be subtle enough that people still feel safe using it—and is that really so high of a bar?). We heard some error message free verse. I wished for a crypto subplot in the yet-unnamed 2017 Star Trek series. A train was late. We heard a poem about sin and a pæan to one's employer. A tablet app for sports recordkeeping enthusiasts was proposed. I got mail. We considered the quiet holocaust of things left unsaid perpetrated by the forces of laziness and evil that lurk in people's hearts. I got a patch into OpenStack Swift (ninth draft landed as 06bede894 on 7 October, first draft proposed for review on 19 July). You heard about some of my favorite things in September. We went back to basics on the justification of a classic property of arithmetic. I commented on a certain matter of misperceived incentives, and another matter of determination in the absence of what others could discern as incentives. I went to the first Rust conference. My bold investigative reporting showed that the seemingly-straightforward $ directive treacherously means different things to different regular-expression engines. I confessed to favoring a nonstandard pronunciation of a file extension. I mentioned that I was trying to write a compiler, although honestly, I don't think it went so well (the lexer, parser, and obligatory Emacs mode weren't that bad, but I kind of intuited the context-handling and code-generation parts, and should really get farther in the textbook and give it another try later). I explained how erasure codes work. We heard some more error-message poetry. Some readers of this blog (!!) relieved me of some of my confusion about why the Rust borrow-checker rejects bad code that you might think is good if you grew up the cartoon fantasy world that calls itself Python. I explained why "Pi Day" is an unholy festival of sin that is corrupting our children. You heard from me during February, and we saw a macro that is just too cute.

Beyond these, there are many more things that remain to be said.

But they will have to wait until next year.

Compilers Week!

It's Compilers Week here at An Algorithmic Lucidity!

Which is to say that there will be no blogging this week because I'm busy writing a compiler.

But maybe you already guessed that there would be no blogging this week.

This obviously doesn't count.

In more ways than one.

2014 Year in Reverse

(Previously.)

If I had any readers who still believe in the A-theory of time, I might say: 2014 is dead! Gone! Over! But since I probably don't have any readers like that (since I probably don't have any readers, full stop?), it's better to face the truth: 2014 is an immutable part of our universe; just because we don't—get to?—have to?—experience it "now", doesn't mean it has "stopped" existing, any more than 2016 doesn't exist "yet" just because we don't remember it.

Anyway. In that two-thousand-and-fourteenth year of our Common Era, the first year of my life (that I feel comfortable admitting to), and (unfortunately) not actually the Year of the Em Dash, this blog saw 45 posts and 40 comments. Among these—

The weariness of being monolingual was confessed to. We saw how to convert Markdown to HTML within Emacs (a technique which is proving itself to be of some convenience to your author in preparing blog posts for publication). We considered one weird trick for what to write when you can't infer the correct spelling of someone's name from what you heard. It turned out that the word apology can mean different things, and that characters in popular 1990s science-fiction television programs aren't always completely honest in interpreting the moral law. We were prompted to prove why we will never write anything. We had a wild Halloween party, noted a baffling error message from Git (hint: commit hooks and virtualenv), and drowned our sorrows in tower defense. The American coffee hegemon started serving pumpkin spice again. There were feelings of inadequacy, at least one contrived distraction from writing that ineffectually pretended to not be a distraction, and the occasional obscure pun. We examined where I stand and were enlightened by some standard advice. There were more feelings of inadequacy. Even conditional on the hypothesis that all's well that ends well, I think it's important to consider the condition of people for which all is not looking to end well. We heard a poem for OpenStack object storage, and a lament against git push --force. I argued that Twilight Sparkle is a disaster waiting to happen and confessed that perhaps too many of my life decisions are determined by what things GitHub happens to provide graphs for. I ate too much ice-cream once and explained how consistent hashing works.

And as for that other nearby immutable span of reality, the one called 2015? Well, that would be telling (and I can't know that from here).

2013 Year in Reverse

Dear reader, as another year comes to a close, it is perhaps wise that we should take a few moments to reflect on what we learned here at An Algorithmic Lucidity in 2013—the year that was!

In the year 2013, this blog saw 103 posts and (at press time) 40 comments. Among these—

We heard a poem about why you should hire me. We surveyed a few numbers between 0 and 1. (Friend of the blog Grognor told me that the funniest part was the explicit disclaimer that the list was non-exhaustive. I replied that not everyone knows about the diagonalization argument—but considering this blog's, um, selective audience, maybe my readers do—all five of you!) We mentioned some hidden costs of talking about stuff. We noted that some words have substrings which are other words. I wrote a series about my experiences studying web development at App Academy (9 8 7 6 5 4 3 2 1). I pasted my first attempt at an OKCupid profile. We observed that I'm a moron. Books arrived in the post. We mused on the nature of personhood and implemented some classic algorithms: the Ford-Fulkerson technique somewhat clumsily in Ruby, quicksort in the form of a letter to a fictional horse, and Huffman coding in Python. We lamented the nature of existence, noticed the non-observance of an unusual tradition, and heard a poem about watching a motion picture by someone you met at university. Three problems with unsolicited advice were discussed, as was the role of bases in the theory of vector spaces, and the subjective indistinguishability of propaganda and other forms of instruction.

So that was 2013 in the year of our Common Era. But what, I hear you ask (both of you who clicked through after the break), what about twenty-fourteen? Having considered the year that was, what of the year that hasn't yet? What is your faithful correspondent thinking about writing ("thinking about writing")? Well, without giving too much away—and, come to think of it, without much of a track record of successfully predicting my future behavior—I'll say to expect more computer-scientific contrivances and more detailed fiction, but probably somewhat fewer overall posts (this last partially because I'm going to be busy contributing to world economic growth, and partially because I want to put more emphasis on the virtue of Quality rather than the virtue of Shipping). So don't touch that bookmark! The best is yet to come! I thank you for reading, and remain,

Your faithful correspondent,
Zack M. Davis

Genre

"I didn't intend for my blog to become a compendium of very short dialogues! It's just—it's just—"

"They're easier to write than anything else?"

"—they're easier to write than anything else."