ongoing by Tim Bray

ongoing fragmented essay by Tim Bray

Pulp Fiction 16 Jan 2022, 8:00 pm

A dozen years ago, our growing family created a problem: Too many books! So after I finished ripping our thousand CDs, we repurposed the CD rack into a bookcase, stuffed it with our genre paperbacks, and put it in the guest room. A few years later, the guest room became our son’s room and the pulp fiction went into boxes. But just now after another bout of domestic reorganization, they’re back out among us. I find their dingy colorful presence cheering. Check out the picture!

Bookcase full of pulp fiction

For those who want to pixel-peep authors and titles,
here’s a full-sized version.

There you have it; 7½ shelves of Sci-Fi, 6½ of mysteries and thrillers, 250 or so books. Likely very few of these will feature in Lit Crit courses any time soon, but they are full of imagination and intensity and color, and in aggregate contributed significantly to the comfy and cluttered interior decor of my mind.

Now a few statistics, which I suspect that connoisseurs of 20th-century popular fiction will find pleasing.

  1. Most popular authors: Rex Stout (32 volumes), Gene Wolfe (21), Elmore Leonard (16).

  2. Less-renowned authors whom we apparently enjoy and you might too: Michael Dibdin (9), Clifford D. Simak (8), Sara Paretsky (7).

  3. Special smiles: Delany, Disch, LeGuin, Leiber, Zelazny.

  4. We have moderns! Charlie Stross (9), by which we observe that he became productive before pulp became Kindlified.

  5. Most-loved, judged by the level of tatter: The Mote in God’s Eye and The Moon is a Harsh Mistress. I’m pretty sure I bought the latter when I was about 14.

  6. Retired due to not having aged well: Travis McGee.

  7. Still on the upstairs shelves due to being mostly hardcover: Gibson and Gaiman.

Their new location means we walk by them when we go upstairs to bed. After another shitty 21st-century day, the appeal of curling up with 20th-century storytelling will be strong.

New Year’s Day 12 Jan 2022, 8:00 pm

In Vancouver, we have a tradition of going for a walk on New Year’s Day, no matter how foul the weather, if only to prove to Other Canadians that the thing is possible. 2022 too, despite its being an extremely low-expectations year. And it wasn’t the worst New Year’s Day beachwalk ever, so maybe it won’t be the worst year ever. And it leaves me with advice for the world. But first…

The traditional Polar Bear Swim, in which hordes of howling Vancouverites, half hungover and the rest inebriated, charge pell-mell into the hostile waters of English Bay, was Covid-scotched like so much else good in life. But that didn’t deter everyone; here and there an espontáneo shed their garments and made made a singleton splash. There was a bored burly bearded lifeguard not getting in the way, happy to chill with passers-by.

Solo Vancouver polar-bear swimmer, 2022

Hmmm, do those clouds look… a little weird? Yes, and not just a little. I don’t think there was a temporal vortex behind that mountain waiting to vomit city-eating chthonic lifeforms over its peak, but if I were a production designer for a city-eating-chthonic-lifeforms movie, that’s how I’d fill in the clouds around the mountain they were lurking behind.

Weird clouds over Vancouver’s Mount Cypress

Let’s look West, where the Blessed Lands would be if there were any, but in fact it’s just Vancouver Island and then Japan. But yeah, those clouds.

Looking west across English Bay on New Year’s Day 2022

See the ships? Those are the “supply-chain issues” that are screwing things up. Every ship parking spot out there is occupied, every time I go and look.

Looking west across English Bay on New Year’s Day 2022

That snow wasn’t melting because it was pretty damn chilly. So our walk didn’t go on for that long. But of course we couldn’t pass up Vancouver’s major new talking point, Marge the Barge.

Cargo barge washed up on English Bay beach

This big sucker blew up on this very-downtown beach in a big windstorm last November that unfortunately combined with a super-high tide, and it’s not coming off any time soon. A little bird told me that its bottom is severely damaged by those rocks it’s hitched up on and if the tide got high enough to float it off, it’d sink again. Another little bird told me that it was sloppily and amateurishly tied up to the barge buoy out there when the windstorm came along and disposed of its moorings in about fifteen minutes.

The city’s kind of adopted it, as though we had any choice. I particularly admire that mooring line someone’s tied up to it.

Barge tied up to a rock on the Vancouver waterfront

Now, that’s a damn expensive piece of rope. And I don’t know (but admire) how the hell they got it around that rock. The effect, of course, is purely symbolic. If Marge decides to float away via some accident of tide and wind, without immediately sinking, the rope isn’t going to help. But it’s stylish.

The scene by the barge was pretty cheery; everyone had a story to tell. There were toddlers sledding on the unaccustomed snow. The rent-a-cop down by the barge totally ignored the people checking it out. Every photographer was looking for their own special angle. Strollers and longboards and spaced-out hippies shared the paths, reasonably amicably. Everyone who wasn’t talking to the person walking with them was talking to their phone.

Advice

Covid sucks. Politics sucks. The weather sucks (and is a hideous harbinger of worse to come). But, don’t stay home. Go outside anyhow. Take pictures. Tell stories. Laugh at the laughable. Tune out the jerks. Nobody promised there’d be a tomorrow so do today today.

Happy New Year!

Long Links 5 Jan 2022, 8:00 pm

For a while there I was posting a Long Links early in each month, recommending long-form pieces that I’d had the time to read, acknowledging that people who aren’t semi-retired wouldn’t have time for all of them but perhaps one or two would add value even for the busy. I fell out of that habit but now it’s 2022 and there are plenty of tabs I meant to write about, some dating back to early 2021. So… Welcome, everyone, to 2022, and let’s go long!

I bet a lot of people who read this blog play video games. I bet a lot of people also read The New Yorker. I bet they’re probably not the same people. So, for people on the left side of that Venn diagram, I recommend The Best Video Games of 2021. The list is idiosyncratic and light on AAA productions. I want to give a few of these a try!

Anyone who knows me well, or has read much of what I write, has probably encountered a rant about the irritating habit, among the rich and successful, of attributing their position solely to their own talent and energy. I’ve never believed that, simply because I know how many of my own successes have involved a lot of good luck, and because I know many people who are smarter and harder-working than I am but who struggle to get along in life. For those who agree, and especially for those who don’t, I offer Talent vs Luck: the role of randomness in success and failure by Pluchino et al, published in Advances in Complex Systems. I quote from the abstract: “In particular, we show that, if it is true that some degree of talent is necessary to be successful in life, almost never the most talented people reach the highest peaks of success, being overtaken by mediocre but sensibly luckier individuals. As to our knowledge, this counterintuitive result - although implicitly suggested between the lines in a vast literature - is quantified here for the first time.” Told ya so.

John Battelle asks, on his blog, Why Is The Streaming Experience So Terrible? Nobody really liked the era of cable TV, and streaming felt like a fresh new direction when it arrived, but today it’s starting to have that cable-TV smell, on top of becoming really expensive. I enjoyed reading this because I thought I was the only one generally grumpy about the streaming-video era. I guess I’m not.

Batteries are much in the news these days, as we embark on the first stage of the titanic shift away from a world that runs by burning fossilized micro-organisms to one that runs on renewables. There are a whole lot of issues around batteries: Cost, durability, and the environmental impact of building them. I can’t say how much I enjoyed Electric cars and batteries: how will the world produce enough? published in Nature. It addresses all the issues around batteries that I knew about and a bunch that I didn’t, is impeccably written, and, as you’d expect from this publication, armed with high-quality graphics and first-rate references.

In recent years I’ve been fascinated by the science of “Dark Matter” — I put the words in quotes because no matter how hard physicists look, they can’t find the stuff that needs to be there for galaxies to behave the way they do, according to ΛCDM, the “standard model” of cosmology. Some suggest that Dark Matter doesn’t actually exist, and offer an alternative explanation of the galactic weirdness that produces good results, but requires an adjustment to Newton’s famous gravitational formula G(m1m2)/r2, a thing that (rightly) makes physicists nervous. This alternative vision of cosmic reality is called MOND. Recently, there’s been lots of coverage of the JWST space telescope, and in his Triton Station blog, Stacy McGaugh considers What JWST will see. Because JWST will allow astronomers to establish the size of the furthest-away (and thus oldest) galaxies, and the predictions of ΛCDM and MOND on what they’ll see are very different. One waits with bated breath.

David Heinemeier Hansson (universally referred to as “DHH”) is the inventor of the popular “Rails” website-building software and a successful businessperson, with a loud voice and many interesting opinions. When I first ran across him in 2005, he was also kind of an asshole, spewing testosterone all over the blogosphere and Web-conference circuit. In recent years he’s become way more grown-up and likeable, and usually worth paying attention to, although his standing took a beating last April when a controversy about what sort of political discussion was legit on internal company channels led to a third or so of the staff bailing out. Whatever you may think of him, if you’re interested in software and in particular Open Source, I recommend I won't let you pay me for my open source. It’s a little longer than it needs to be but covers interesting territory, and lots of it, and although I certainly don’t agree with every word, I sure enjoyed reading it.

Long-form works aren’t all in words. Here’s 12 minutes of Vivaldi’s Concerto in E minor for Bassoon; super tasty stuff. The performance is from Poland and the performers almost all women. First-class music, beautifully recorded.

Let’s hope 2022 is better.

Locally Optimal 27 Dec 2021, 8:00 pm

It’s six months since my second Covid shot (team AZ/Moderna) and thus booster time. Which I found out this morning via SMS from my local (BC, Canada) Centre for Disease Control. The registration process was fast and painless. This is exactly the sort of thing we thought we were building the Internet for, and has lessons to teach.

SMS invitation for Covid booster

On my phone, I tapped the link in the SMS, which took me (more or less instantly) to a bone-simple browser form with two fields, my reference number (pre-filled-in) and my Personal Health Number (had to open my wallet to look at the card). Helpfully, that field switched to numeric-input mode.

One more tap took me (instantly, once again) to a list of vaccination facilities. I picked the one closest to my house and (without delay) saw that dates were available the day after tomorrow. Another tap popped up an appointment-time chooser, and the last one yielded “confirmation on the way by SMS and email”; they arrived within seconds.

Each achievement in the following list is modest but above the bar for typical public-facing systems both from government and the private sector:

  1. A single tap on my phone screen took me to the right place.

  2. I accomplished the task without failures or errors.

  3. The service requested minimal information.

  4. That information was made easy to enter.

  5. Interactions were fast and smooth, with no waiting.

Things that were not observed:

  1. Graphical decoration.

  2. Popups.

  3. Animations.

  4. Marketing.

What is shocking isn’t that a level of government managed to deliver an essential service with this level of graceful attention to detail. It’s that so many of them fail to do so.

This kind of thing couldn’t be accomplished without the Internet. It’s more important than any dozen Bay-Aryan unicorns and social-media MegaCorps put together.

[Of course, it also helps that in Canada, healthcare is provided for free as a consequence of citizenship.]

2021 Series Survey 12 Dec 2021, 8:00 pm

What with Covid and being semi-retired, I had plenty of reading time in 2021. In particular, I got to the final volumes of multiple long-running literary franchises. (Some of those final volumes had been published years before.) I think that every series listed here is a good investment of your time. Two warnings: It’s mostly pretty mainstream, and kind of sci-fi heavy. Quick table of contents: Broken Earth, The Expanse, Merchant Princes/Empire Games, Neapolitan Novels, The Sandman, Sandman Slim, Trickster Trilogy.

I offer links for each work: Amazon, because it’s convenient and if you follow these it might help sell panties and thus get my family free books, and Alibris, because they seem to be about helping independent booksellers (although their ownership is opaque).

Now, in alphabetical order…

Broken Earth

Premise: On a planet plagued by destructive seismic activity, some people can control earthquakes with their minds; they are feared and persecuted. We follow the adventures of three such women in a time of high crisis.

Each volume of this trilogy won N.K. Jemisin the Hugo award for the year it was published. What’s great about it is the complexity and believability of the characters, and the immersive intensity of the action; once you start reading you can’t stop.

What’s not so great (for me anyhow) was the slightly shaky world-building, my suspension of disbelief wavered every so often. Put any internal knowledge of physics on the back shelf and you’ll have fun reading these.

Best among these at: Taking you to really weird and intense places and leaving you with a strong impression of what it felt and smelled like to be there.

Amazon, Alibris.

The Expanse

Premise: Human civilization has spread across the Solar System and is beset by empire-building and colonialism. On an asteroid, an alien artifact is discovered, with momentous consequences; plenty enough for nine books and six TV seasons. We follow the adventures of a half-dozen or so lead characters and their doughty spacecraft, the Rocinante.

I mean, either you like Space Opera or you don’t. If you do, you’ll really like the work of James S.A. Corey, the pen name of Daniel Abraham and Ty Franck. Also, the TV series (its final season just shipping as I write this) is really very good.

The biggest criticism I can offer is that there’s a little too much talk about what the characters are thinking and feeling, could have been shown not told.

Best among these at: Clever plot construction, space battles.

Amazon, Alibris.

Merchant Princes/Empire Games

Premise: Our Earth has multiple timelines, similar to each other but different in interesting ways; certain people (and eventually, machines) are able to move between them.

There are a whole lot of books here! After writing the first few, author Charles Stross anthologized and reorganized a bit, so you have to do a bit of work figuring out what to buy to move from one end to the other. I’m a little surprised nobody’s taken this to TV yet, there are loads of juicy parts for good actors.

The thing about Stross is that he wants the audience to have a good time, and you get the sense that he’s having fun too. So at no point is there a feeling that anything’s taking itself too seriously. Loads of amusing characters and surprising situations and clever twists of fate.

Best among these at: Being convincingly set in the current real world, plus amusing variations. Also, nuclear explosions. Lots of nuclear explosions!

Merchant Princes Amazon, Alibris.

Empire Games Amazon, Alibris.

Neapolitan Novels

Premise: Two women grow up in a seedy part of Naples, then each makes a lot of bad choices as they move through their twentieth-century Italian/European lives.

Author Elena Ferrante is pseudonymous, although there has been a journalistic exposé which convincingly claims to have identified her.

Now, a confession: I haven’t actually finished the final book in the quartet, The Lost Child, because I got so mad at one of the characters (did I mention bad choices?) that I would have thrown the book across the room except for it was a Kindle. I actually think this is a symptom of it being a really good book. And I will go back and finish it.

The quality of the writing (and necessarily the translation) just can’t be praised too highly. This is beautiful, beautiful, stuff. The texture of life is examined and unwoven with a precision and intensity that I have never encountered anywhere else. Every page of the narrative is totally convincing: Yes, these people were in these situations and did these things and felt those things. The situations, growing as they do out of working-class bad-neighborhood society, are as exotic to most educated readers as any outer-space battle; except for being real.

There’s a minor geek-interest side-plot: One of the protagonists works her way out poverty via the IBM System/3, an important 1970’s piece of computer history.

Best among these at: Raw emotional intensity without loss of realism.

Amazon, Alibris.

The Sandman

Premise: Morpheus, or Dream, is one of the seven Endless, along with Destiny, Death, Desire, Despair, Delirium (formerly Delight), and Destruction (also known as ‘The Prodigal’). Dream is a complex character, not terribly admirable. Any attempt to say more is going either to be inadequate or require ten thousand words to get started up.

These graphic novels, written by Neil Gaiman and illustrated by a legion of wonderful artists, were published between 1989 and 1996. I’m really not big on graphic novels but I read Season of Mists on paper once many years ago; at the time, it didn’t change my mind about comic books. After buying a Samsung S7+ with a large and fabulous screen, on impulse I bought the whole 12-volume omnibus.

I just can’t say enough good things about the Kindle presentation of The Sandman on a big bright tablet; it was maybe the most intense reading experience I’ve had in years. It took me three months to get through this, and while it’s long, one reason was that sometimes I just needed a rest.

Among other things, the Kindle app helps deal with fanciful/explosive full-page compositions by gliding through the dialog bubbles one at a time in the right order. Not being a native comic-book citizen, I found this terrifically helpful.

Anyhow, the writing is fabulous, distinguished by its high velocity: You move from one frame to the next and realize that a whole lot of stuff must have happened and Gaiman hasn’t bothered to fill in the details because there’s enough there to deduce them. I admire this; and anyhow, the books are plenty long enough.

One small criticism of the big omnibus edition is the self-indulgent front and back matter for each volume, with fulsome praise from a panoply of famous names. No value is added and I just paged through them after the first couple.

Um, trigger warnings. I mean, all the trigger warnings, there is some extremely dark, brutal, awful stuff in here; I also took a reading break one week when I was having a tough time and just didn’t need the blackness.

I wonder if I now like graphic novels? Or perhaps this is entirely sui generis.

Best among these at: Making utterly strange and phantasmagorical scenarios believable. Delighting the eyes.

Amazon, Alibris.

Sandman Slim

Premise: James Stark returns to Earth after a decade in Hell. He is a monster who kills monsters. And, it turns out, something of a sensitive new-age guy when he’s not. Love, romance, and extreme violence.

This is author Richard Kadrey’s only real hit. The twelfth novel, which just came out, is advertised as the last, so we’ll see what he can do next. This series, to be honest, is perhaps less ambitious than some of the others here but it’s fun and you can’t spend all your time reading Ferrante, can you?

Stark is a really fine piece of character creation, and not the only one in this series, there are loads of interesting people and supernatural creatures, well-distributed across gender and other demographic divisions. Also, Kadrey has a major talent for dialogue; I’m not saying Elmore Leonard, but it’s not crazy to mention them in the same sentence.

And there’s action, lots of extremely violent action, in which many terribly undesirable creatures come to bad, messy ends, and then also other players of whom the reader will have grown fond. But in these books, you can after all come back from Hell and various other afterlife scenarios.

The world-building is necessarily somewhat theological and frankly ridiculous, but once you stop taking it seriously that’s not a problem.

For supernatural thrillers, these are nicely grounded, specifically in Los Angeles and its reflection in Hell. Just a taste: Stark is having a conversation with Lucifer (yes, that Lucifer) walking around in LA, and they stop for huge sloppy greasy burritos from a food truck. Lucifer is wearing a beautiful white linen suit and manages to eat the whole burrito with absolutely no food stains. Now that’s devilish.

Best among these at: Snappy dialogue.

Amazon, Alibris.

The Trickster

Jared is an indigenous high-school student in Kitimat, BC who’s got problems with family and addiction and school, despite his successful weed-cookie business. His mother is a real piece of his work and his father… well, there’s the story, his father is probably a Trickster, a powerful and malevolent supernatural creature with so many children they have to self-organize on Facebook. Jared eventually ends up in Vancouver at community college, making donuts for money and pursuing a struggle with unnatural enemies that builds to an apocalyptic climax.

Author Eden Robinson is, like Jared, from Kitimat and indigenous. She claims she was trying to write a “trashy reservation romance” but it got out of control and became three pretty big thick novels full of teen angst and magical violence.

There was a CBC-TV series that managed to deliver a lot of the Trickster flavor while wandering way away from the books’ plot-lines. I enjoyed it but it has apparently fallen apart after just one season partly because the director’s claims of indigenous heritage came into question.

This delivers you-are-there flavor for a lot of places that nobody will ever go and is bulging with lovable personalities. Most people are kind and decent, some are monstrous, and Ms Robinson has fun with both flavors. An entirely entertaining read end-to-end, often hysterically coffee-snorting funny. And also with plenty to teach about our indigenous people and the realities they face.

At this point, it may be a little surprising when I say that I enjoyed this series more than any of the others here. Granted, it’s possibly because it takes place partly in my home-town. But these are outstanding books.

Best among these at: Taking you to interesting real places that you’ve never been, and imaginary ones too. Making you laugh so hard the other people in the room will want to know if you’re OK.

Amazon, Alibris.

Filtering Lessons 3 Dec 2021, 8:00 pm

Last week, AWS announced (docs, blog) event filtering for Lambdas reading from SQS, DynamoDB, and Kinesis. Lambda’s new filtering is consistent with EventBridge’s. Also announced was faster, slicker, cheaper S3/EventBridge integration. These features rely on software I helped write. The problem turns out to be more interesting than you might expect, by a factor of 10¹² or so. I think it’d be really great if the code were open-sourced.

The core idea is that in modern applications, streams of data flow around between services, and very often you want to select just some of the data out of the stream. For example, if you’ve written a few Lambdas, quite likely some of your code was about looking at the input events and deciding whether to process or ignore them. Or deciding which other software to dispatch them to. If you can achieve the same effect by writing pattern-matching rules, you can subtract code and save money: a win-win.

Interesting why?

It’s not as though these filtering features are doing anything new or innovative or surprising; this sort of thing goes back as far as 1973, with grep. There are lots more landmarks along that road; for example a lot of smart people seem to like jq these days. So, why should anyone care about this software?

First of all, the stream it’s filtering might be carrying millions of events per second. Second, people will want to post a million rules against the stream. (Neither of those numbers are hypothetical.) So it’s a million times a million times trickier than you might think.

Bearing that in mind, we created a library which I’ll call the Matcher. Its query expressions are a little idiosyncratic, but people seem to like them; that blog I linked to earlier is probably the best introduction I’ve seen.

The performance envelope is, um, unusual. But pleasing. “Good enough for EventBridge” is decent testimony. Even stronger is “good enough for S3”. Check out this short thread from Usman Khalid, GM for EventBridge and some other stuff, a guy I really enjoyed working with at AWS. He explains that the event-matching logic has been pushed down into S3, which doesn’t even emit the events unless they match a rule. Now, “inside of S3” is a very special place. An almost unimaginable amount of customer data lives in there, subject to an almost unimaginable transaction rate, and everyone is very very latency-sensitive.

How it works

Well, that’d be a secret. At least I thought so until I sat down to write this, and discovered U.S. Patent 11,068,487: Event-stream searching using compiled rule patterns, filed in 2015 but just issued last July. Patent theory says that the disclosure should be sufficient to enable a competent practitioner to implement the technology. Heh, good luck with that. I know a better way than patents to share engineering progress.

Open-source why?

The Matcher would be useful for a lot of people, if the AWS experience is any guide. It’s easy to integrate. It’d be a significant and visible give-back from AWS to the software commons from which it’s profited so richly. I also think it’d be a useful teaching tool (see below).

Having said that, and to be brutally honest, my reasons are selfish. I enjoyed working on this so much, and would like to do it some more. So from that point of view, AWS would get some free time out of an engineer with highly relevant experience.

I might also like to make versions of it in other programming languages.

And courseware

I applied plenty of lessons hard-won in decades of programming to this software, and learned useful new ones. So there are deep software-tech blog pieces screaming to be written. In fact, I think I could probably put together a 400-level CompSci course on “Topics in high-performance software design” using the Matcher as a lab substrate.

I’ll support that claim with one interesting and counter-intuitive lesson. But first…

History

When I started at AWS on December 1, 2014, my hiring manager said “Our idea is we’re gonna have an event bus carrying notifications of everything that’s happening in everybody’s AWS account and you can subscribe to it, select the events you care about, and process them with Lambdas.” So, it wasn’t my idea; and that manager is now a VP who owns a huge swathe of AWS Networking.

But I helped write the PR/FAQ and was senior engineer on the dev team.

I couldn’t stop thinking about matching the rules that defined the subscriptions. My mind hadn’t stretched to millions times millions yet, but it seemed obvious that the biggest customers, who would have the most events, would also have the most rules. So if you coded up a simple-minded loop like “for each event, retrieve all the rules posted by the event’s account and check one by one to see if they match” that’d probably be bad.

So I coded up most of release 1.0 of the Matcher by January, with a couple of breakthroughs happening on my Mom’s couch when we visited her in Saskatchewan for Christmas.

EventBridge (launched as “CloudWatch Events”) worked out pretty well and soon other groups started using the Matcher. It grew plenty of features over the years, some only exposed internally. Naturally, like any other widely-used piece of software that started out lean and mean, it became fatter and more complex. While there’s a ton of my code there, I’m sure it’s a minority by now.

I met some wonderful people from other AWS and Amazon teams who said “Oh we need Matcher but it also needs to do X, mind if I code that up?” Working at AWS was a fun job, and Matcher was one of the most-fun parts.

Now, I promised a non-obvious finding that emerged during the Matcher work…

Lesson: JSON is faster than binary

Here’s a surprising statement: If you want to filter messages very, very quickly, you can do it much faster if they’re in JSON as opposed to a strongly-typed “binary” format like Protobufs or Avro.

No, really. Because for those formats, you have to parse the whole binary blob into a native data structure, which means looking at all the fields and allocating memory for them. You really want to minimize memory allocation for anything that happens millions of times per second.

JSON, however, allows stream parsing. Consider for example, the Jackson library’s nextToken() or Go’s json.Tokenizer.

Which means you can skip through each event, only keeping around the fields that you’re interested in filtering on, which in most event-processing scenarios is very few of them. So yeah, you do have to parse strings of digits into numbers, but only for the fields you care about.

And there are better structures for storing a small selection of data fields than JSON’s messy, weakly-typed tree. I’ve measured; the performance delta between parsing incoming events into an object and just skipping through their fields is freaking huge.

That’s just one take-away; there are plenty more. I’d sure like to share them.

[Update: I’ve seen feedback that there’s no reason in principle that you couldn’t build streaming parsers for the strongly-typed binary data formats and that, for Avro in particular, it ought to be really fast. As far as I can tell, there isn’t much in the way of such APIs out there at the moment; sounds like fun little programming project.]

Listening to re:Invent 30 Nov 2021, 8:00 pm

This is the week that AWS owns the tech news cycle because anyone who cares about it is watching the stories coming out of the re:Invent conference. I watched Adam Selipsky’s keynote this morning and it looks like AWS is moving in a very traditional direction for tech companies: Away from tech-first, towards business-first. Which is inevitable but also dangerous.

Messages

The “S” in AWS stands for “Services”. So presumably, the biggest imaginable announcement would be of new Web Services. There was only one in among the avalanche of biz-talk: managed 5G, nice but not terribly exciting. Powerful message!

What were the companies either heavily name-checked or invited on-stage? DISH, NASDAQ, United, Goldman Sachs, and 3M. None of them West-Coast. None seen as tech players. Powerful message!

The number-one problem I hear about from every single tech leader I talk to is the extreme difficulty in hiring and team-building. So I’d think that providing developer force multiplication by automating away server management would be a big deal. None of that; the focus was almost all (except for a bit from 3M) on hosting existing applications. Powerful message.

There’s also the fact that a high proportion of developers and technologists viscerally loathe at least two of those highlighted companies; so impressing those demographics was not on the priority list. Powerful message!

History

Let’s look way back, to the year I graduated from University: 1981 (no, really). My first job was with DEC. At that point it was the second-largest computer company in the world. The largest was IBM, which was ten times our size and, as our leadership said, “growing at one DEC per year”.

DEC at that time had the coolest computers you could buy and nobody working where tech mattered wanted anything else. Leadership noticed that despite that fact, IBM was kicking our asses. And IBM people never really talked about technology, it was all business all the time. So the word came down from on high that we would too.

I remember going to the announcement of some new computer which was a one-hour presentation, and the first 45 minutes were how DEC cared above all else about the Modern Enterprise and Leadership Agility and Profit-oriented Innovation. This wasn’t a computer, it was a Business Solution! Meanwhile the audience got angrier and angrier because they wanted to hear about memory and form factors and cycle time and benchmarking and so on. Those things were eventually crammed into the last ten minutes of the session, and the computer actually did pretty well.

But the company didn’t. It was skidding downhill by 1990 and didn’t make it to Y2K. My own geeky tribe was pretty convinced that it was because if you wanted to sell computers, you should find people who needed to buy computers and talk about computers to them.

I saw a similar story when I worked for Sun, which for quite a while sold the coolest computers but eventually fell on hard times and was snapped up by Oracle. Oracle talks about business and only about business. Once again, the tech-nerd demographic sneered at Oracle’s lack of technical acumen.

I’m pretty sure my tribe was wrong both times. Here’s why.

Growth space

People think the cloud business is pretty big, and it is. AWS is running at $60B/year and by my guess represents somewhere between a quarter and a half of public-cloud spend, so let’s say the whole Public-cloud biz is in the range of $200B or so. Wow, that sounds big! But estimates of global IT spending are north of $4T.

That math says that 95% of IT isn’t on the cloud yet. Will it all move there? No, but it feels inevitable that the cloud revenue potential is at least 10× today. And where is that 10×? I’ll tell you where it isn’t: In the kind of startup and cloud-native scenarios that led the charge onto the cloud over the last fifteen years. It’s in Establishment IT, where by “Establishment” I mean “big organization with significant systems in production”.

Establishment-IT organizations are cost centers, mostly. They report to people who neither understand nor care about technology, and are incented to find cheap decent solutions to their business problems. So it seems pretty obvious that whoever figures out how to talk to that leadership is in a position to win the 90% of the potential cloud market that’s still up for grabs. And that’s not by enthusing about leading-edge Serverless Event-driven Container Orchestration or whatever.

Horizontal Selling

Here’s another problem that the public Cloud faces as it tries to expand into the enterprise, one I’m bitterly familiar with because I’ve spent my whole life working on general-purpose infrastructural software. Which inevitably leads to the following conversation:

Tim: “Check out this slick new optimized event router we cooked up!”
Product Person: “Great! What can I do with it?”
T: “Well, anything! You know, make distributed apps faster and better!”
PP: “You don’t get it. What actual specific job, like manufacturing cars or processing insurance claims, will this help with? So I can go and talk to people who do those jobs.”
T: “But your apps will be more flexible and robust, too!”

At the moment, pretty well 100% of AWS revenue comes from totally horizontal technology, so you should pity the Product and Sales people who have to build a bridge between, for example, DynamoDB latency and Fast-Fashion inventory management. The fact that they’ve gotten this far is a tribute to them.

I couldn’t help but notice that in Adam Selipsky’s keynote this morning, there was a specifically-called-out emphasis on vertical offerings, for example the “Fleetwise” automotive-telemetry thing (which however launched without a charter customer). I think it’s pretty obvious why AWS is doing this. But the absence of a track record is worrying.

Which way forward?

I’m not gonna offer any recommendations about how AWS should position horizontal tech to sell to Establishment IT. If I did, they should be ignored. [You know who’s really good at it? Andy Jassy.]

But I do have three pieces of advice for AWS on their keynote presentations:

  1. The all-biz-all-the-time messaging implies a trade-off. You should do better selling into that huge 90%-of-trillions IT space that isn’t in the cloud yet. But you’re less likely to land the business of the next Netflix or Intuit or, well, Amazon.com. Maybe that’s OK, but also consider at least some messaging aimed at technology pioneers.

  2. I think that current technology leaders, even in the fustiest corners of Establishment IT, are noticing that they’re having a big recruiting and staffing problem. Modern Cloud-Native tech lets you deliver more business-app goodness with fewer developer-hours. I think that angle is becoming increasingly strategic, and deserves keynote time.

  3. Pick up the damn pace! That two-hour keynote was grindingly slow and contained maybe a half-hour of actionable information. Your audience mostly comprises really smart people and they appreciate high-intensity high-density discourse. (This is another space where you might want to learn from the Jassy style.)

More to come

I understand that we systems Morlocks aren’t the target audience for the big glamorous launch keynote from the CEO. Do I feel slightly disrespected and ignored? Yes. Am I going to go back and listen for actual technology announcements from one of the world’s most interesting technology companies? Sure.

CL XLII: Picture Hugs 29 Nov 2021, 8:00 pm

Out on a walk with Lauren, we stopped in a park and I noticed she was holding an evergreen branch with one hand, stroking it with the other. “Sometimes, you just got to hug a tree” she said. No lie. Especially when times are shitty. I can’t put you next to a tree but at least I can take you out in the woods photographically. It’s dark now and we can use all the help we can get.

Ferny forest floor and medium-old trees on Keats Island

These pictures were taken on the morning of November 13, 2021. Around noon, the first of the “atmospheric river” storms hit, dumping 200mm or more of rain and causing floods and landslides that killed at least five people and, for a period of several days, entirely cut off Vancouver and environs from the rest of Canada by road.

Dunno about you, but I’m hurting and so’s almost everyone I know. The viruses infecting the population, the torrent of lies and craziness infecting public discourse, the onrushing effects of the climate disaster infecting our lives and landscapes. We’ve got family issues too; but then so does everyone, one flavor or another.

Mossy tree trunk on Keats Island

This forest is on Keats Island in Howe Sound; when we bought our cabin there in 2008 we were told that the island had been logged “about a hundred years ago”. I can testify that the trees have grown noticeably in our years here. I won’t see them attain their full height, but if I have any grandchildren, they might.

Tree trunk with with fungi on Keats IslandTree trunk with orange fungi on Keats Island

You probably don’t want to hug most of these; they’re too big to get your arms around, and also the bark tends to be rough and have things living on it. Doesn’t matter, being among them is plenty good enough. And just looking at these pictures helps, for me at least.

Forest study on Keats Island

As we finished up the walk, raindrops were starting to lash down. That was the first rain tsunami. As I write, we’re now in the gap between the second and third. They used to call them “hundred-year” storms but the climate catastrophe is making that a cruel joke.

I’m not sure we’re gonna make it through the tough times to come but we should work hard on it, as much for the sake of the forests as for ourselves.

Overtorqued 27 Nov 2021, 8:00 pm

I don’t write much about really personal stuff, but in recent years there have been mild-to-serious mental-health issues affecting my immediate and extended family. As a consequence I have uneducated ideas about evolution and mental health to offer. And with them, for comfort, photographs of a beautiful garden.

A member of the extended family, let’s call them FC, stayed with us for a while in September because their immediate family needed to go overseas and it’s not OK for them to travel internationally or be alone in the house. Also we are happy to have FC visit, because they are cool and oddly erudite and charming.

FC has multiple diagnoses, both psychiatric and neurological, is on a formidable medications regime, and suffers more than anyone should have to. It’s so, so sad.

Arbutus in Vancouver’s Queen Elizabeth Park

Arbutus, I think. All the photos here are from Vancouver’s Queen Elizabeth Park.

Last winter a member of my immediate family received a mental-health diagnosis which included the word “mild”. While our experience is less bad than FC’s, I’m here to tell you that even a “mild” mental-health issue can inflict real quality-of-life damage on the patient and those near them. This is serious stuff and should be taken seriously.

And then, earlier in the year, I visited with my brother, who works for a healthcare provider that serves people in the worst-imaginable situations, typically with mental-health diagnoses and addiction problems and criminal records. They put them in apartments — not large or luxurious but not dingy. This is funded by a conservative (for Canada) government because otherwise these people are going to be spending time in the emergency room or prison, which is way more expensive than a basic flat.

My brother’s organization allowed me to go on rounds with him, after swearing me to absolute secrecy. It was inspiring and mind-expanding. I think I’m staying within the bounds of the agreement by sharing one impression: His clients are, you know, just people. I enjoyed talking to them, one at length because they grew up in a place I know pretty well.

The program seems to work, but these people are teetering on the edge, could be gone in a flash without the right level of support; or even with it. This is a profoundly humane activity and makes my life’s work feel pathetically inadequate.

Back in Vancouver, we walked around gardens with FC; they and I both took lots of pictures, but I have a fancier camera. I’m not gonna go all mystical about the healing powers of nature, but I can testify that I find in-garden time helps my own stability. And I think this experience isn’t uncommon.

These days, my own family spends quality time with family-support groups and psychologists and various flavors of therapist. It’s exhausting and sad, but there is progress. I gather there are many people who stumble quickly into super-helpful combination of meds and therapies, but I can testify that the road is longer and rougher for plenty of others.

Purple leaves in the sun

Coleus, I think?

Why?

You have to wonder. Homo sapiens is the result of millions of years of evolutionary pressure, and mental-health issues are not exactly rare in our species. So the question is obvious: How is this state of affairs consistent with that evolutionary filter?

I have no expertise in either evolutionary biology or mental health, but I do have a theory. The world is full of artifacts and activities that are what I’m going to call “overtorqued”; that is, exhibit high performance at the expense of reliability and flexibility.

Here’s one: Serving in tournament tennis. Anyone who’s watched elite play has seen first-serve misses, often in an extended series stretching across multiple points. That first serve, if the effort were dialed back say ten percent, might avoid many faults. But the win potential of a full-effort first serve is high enough that the athletes are taught to accept its high failure rate.

Red flowers and large succulent backed by a dark rock wall

QE Park’s gardens are built into former quarries; as you can see, the stone is impressive.

Another example: If you’re a serious computer nerd, say a video editor or gamer, you can “overclock” your computer; make it run faster than it was designed for, at the risk of it becoming more failure-prone or even (literally) burning out. Lots of people do, because fast is good when you’re doing something that matters. Yes, I’m saying that to the extent that the human brain is like a computer, it’s like one that’s been overclocked.

Finally, consider the cheetah: A fragile species, with little resilience against habitat degradation and whose young are highly vulnerable to predation. Evolution took that bargain, trading off for exemplary hunting efficiency in just the right set of environmental circumstances.

Possibly evolution selected a similar trade-off for humans: Boosting genes that raise brain function past the level that an engineer would consider “safe”, because that gets us the occasional Marie Curie or Ramanujan or Miyazaki, whose successes benefit the family or the tribe. The cost is that everyone knows someone who suffers some degree of mental dysfunction.

It’s not hard to imagine what other direction evolution might have taken: Suppose our species had a predisposition to loathe and be repulsed by anyone whose behavior fell even slightly outside the expected norm, to the extent that those people would never find mates and pass their genes along.

Obviously there are issues of recessive genes; there are several well-known genetic lethalities that remain endemic in Homo sapiens; the postulated difference wouldn’t entirely stamp out pathologies of the mind. But still, if we were as a species less tolerant of eccentricity, it might be a very different world.

And in fact, people are unsurprised when those who exhibit exceptional achievement are also, to use the vernacular, “a little crazy”.

Fallen blossom on leaf-covered water

I don’t know what the fallen blossom is, nor the nature of the green vegetation covering the surface of the water.

Thank goodness

But most people fighting through mental-health issues aren’t mad geniuses; they’re just our loved ones, ordinary people who got dealt a shitty hand and are doing their best, with whatever support and acceptance the rest of us can grant them.

Maybe you’re among the lucky few who haven’t (yet) been exposed to this kind of thing; if so, you probably will be, eventually. Thank goodness that we have therapies that sometimes work. (So sad that sometimes they don’t.)

Helping out a person going through these issues is often painfully difficult, simply because their perception of reality differs from yours. You can’t change that. But you can hang in and do your best. Sometimes things do get better. And when they don’t you still need to be there.

Blah, Blah, Blah, Boom 9 Nov 2021, 8:00 pm

COP26 is winding down, and I think nobody can seriously disagree with Greta Thunberg’s take: “Blah, blah, blah.” And then she continued “The real work continues outside these halls. And we will never give up, ever.” I wonder: What form does that “real work” take? Well, I just finished reading How to Blow Up a Pipeline by Andreas Malm, which doesn’t include instructions for blowing up pipelines but argues that that Real Work should include infrastructure destruction. It’s a good book, and if you (like me) are horrified at the onrushing climate catastrophe and enraged at our leaders’ incapacity and unwillingness to take actions that might seriously mitigate the devastation, you might want to read it.

How to Blow Up a Pipeline

To start with: I believe that the path the world’s leaders are on, post-COP26, is simply unacceptable. There are hundreds of millions of lives that might be saved, but won’t be. There are whole nations and irreplaceable ecosystems that might be preserved from destruction, but won’t be. To act ethically, we must find other paths than the ones our leaders are offering.

Before I jump into the whole blowing-up-pipelines thing, some takes on where we stand, to get you in the mood.

  1. Mohamed Adow of Power Shift Africa offers What's needed if COP26 is not going to end badly (it ended badly).

  2. Bill McKibben, one of the central voices in global climate activism, from his New Yorker platform writes Trust Is Hard to Find at the U.N. Climate Summit in Glasgow. It is a lucid and comprehensive tour through the many reasons why nobody should believe the messaging coming out of COP26. (More on McKibben later.)

  3. In Rolling Stone Jeff Goodell offers Obama Addresses Climate Activists in Glasgow, But Should They Listen? Obama’s speech was (of course) both soaring and punchy, urging the young activists to stay angry and frustrated, to gird themselves for a marathon not a sprint. Goodell continues: “But the big problem was Obama’s words were that they were just words. And there have been too many words before. Too much high-mindedness. Too much bending of the moral arc of the universe. Too much blah, blah, blah.”

  4. Finally, let’s go back to 2016, to Alex Steffen’s Predatory Delay and the Rights of Future Generations. Because that’s what the 500+ oil-company delegates at COP26 were there for: To, at all costs, delay the arrival of environmental sanity, so that they can keep the oil cash flowing. They know about the scale of the problems they’re creating, but they’re making too much money to care.

What next?

Let’s agree that climate-change activism, decades into the struggle, has mostly failed. The progress is not remotely proportionate to the scale of the problem. What are we doing wrong?

How to Blow Up a Pipeline offers an answer. Although short-ish, the book is rambling and discursive, finding its way indirectly but smoothly through the story it wants to tell. It spends time outlining the direness of the current climate picture, and quickly proceeds to marvel at the absence — up till now, anyhow — of anything that could be called “terrorism” or even sabotage, given that the problem is urgent and targets aplenty are easily accessible: SUV’s and gas stations, to start with.

Malm isn’t just a talker, he has been on the front lines, talking about how “we brought 100,000 people to the streets on a daylong march” to COP15. The numbers of COP demonstrators have escalated from there, year over year (until COVID) but the velocity of fossil-fuel extraction and burning keeps increasing remorselessly.

He goes deep on two organizations: Extinction Rebellion (XR) and, in Germany, Ende Gelände, which has led a very hands-on program of direct action against that country’s insanely filthy and destructive brown-coal industry. Both have a core principle, not only of nonviolence, but of no destruction of property or infrastructure. Both of them, by any reasonable metric, have failed miserably.

Beyond Nonviolence

This leads to Malm’s core argument, which is an articulate and stimulating read, from which I quote::

“So here is what this movement of millions should do, for a start: announce and enforce the prohibition. Damage and destroy new CO2-emitting devices. Put them out of commission, pick them apart, demolish them, burn them, blow them up. Let the capitalists who keep on investing in the fire know that their properties will be trashed. ‘We are the investment risk’, runs a slogan from Ende Gelände, but the risk clearly needs to be higher than one or two days of interrupted production per year.”

Climate activists, he says, should abandon the policy of absolute nonviolence. They should strictly refrain from actions against humans, but proceed with sabotage and destruction, specifically of energy-industry infrastructure and apparatus.

There are plenty of strong moral arguments for forceful and even violent action in certain circumstances, and it’s not hard to construct one when the lives of hundreds of millions of children and poor people are at stake.

Existing organizations preach — Bill McKibben has been very urgent on this subject — that nonviolence is the only way forward, and that anything else would instantly and entirely discredit the movment. Anyhow, they continue, violence is unnecessary; the suffragettes, the anti-slavery Abolitionists, and the movements of Mandela and MLK showed that nonviolence wins.

Malm pushes back at length, with a whole lot of historical evidence. First, the end of slavery cannot be divorced from the blood and pain of the Haitian revolution and the US Civil War. The suffragettes regularly used anti-property violence, breaking windows, starting fires, successfully managing to damage property without causing human casualties (although one suffragette apparently horsewhipped Winston Churchill). As for Mandela and MLK, they were personally nonviolent, but in each case there were scarier factions waiting in the wings, for example Malcolm X in the USA and uMkhonto we Sizwe (MK) in South Africa.

I’m not going to reproduce Malm’s subsequent discussion of the moral and political appropriateness of this course of action, nor his specific recommendations on what classes of asset should be subject to attack, and when. Once again, it’s worthwhile and readable.

His final chapter (of four) is a harsh rebuttal of the councils of despair, represented by by Roy Scranton’s Learning to Die in the Anthropocene and Jonathan Franzen’s cries of pain in The New Yorker. I think this is wasted; anyone who’s read three-quarters of a book entitled How to Blow Up a Pipeline is not the kind who intends to give up.

What then?

I mean, what then for me and people like me? The forces of the state will react to the infrastructure sabotage with brutal overkill. I have some personal experience here: When I got arrested for violating an injunction against interfering with the construction of the pathologically stupid and misguided TMX pipeline, I was fined a big $500, but when people weren’t deterred, kept on doing it day after day, they started being sent to prison.

And it gets way worse very quickly. Last August, Jessica Reznicek self-reported to a U.S. federal prison to begin her eight-year sentence for sabotaging the Dakota Access pipeline. Ominously, and I quote: “Judge Rebecca Goodgame Ebinger also concluded that a terrorism sentencing enhancement could apply because Reznicek tried to stop the flow of oil, retaliated for decisions by state and federal governments to approve the project and wanted to prevent the government from approving future projects like the Dakota Access Pipeline.”

A few brave individuals here and there going to jail is absolutely not going to slow the fossil-fuel juggernaut that is wrecking our grandchildren’s world. So although I would very much like to see that TMX pipeline reduced to a pile of rubble, I have no plans to be a member of another tiny group getting arrested and possibly charged with terrorism.

But, I noticed something that day. The process of law is extremely bureaucratic, inefficient, and time-consuming. Some thirty people were processed, and it took hours; the court cases stretched on for many months. If it had been a thousand, they would have had to put every available cop within a hundred kilometers on overtime for days to handle the load, and the court workload would have been insane. If there’d been ten thousand, there’d have been no point even trying.

So I’d like to point out to anyone who is exercised about a particular piece of petroleum-driven catastrophe unfolding near them: You can shut it down. All you need to do is pull together a big enough crowd. Which is one of the hardest things in the world, of course, but on this issue it’s getting less hard all the time.

Because, you know, it is going to happen. When the crops fail across the US Prairies, when a hundred million people start walking north from the Bay of Bengal with empty stomachs and empty pockets, when Miami and Kolkata and Ningbo are wrecked by a nasty combination of a bad tide and a bad storm, people will run out of patience and start engaging in spontaneous unprincipled violence.

The sort of thing that Malm is proposing is a much better alternative for all concerned.

Footnotes

In the rare pushback I get on this subject, I often hear something along the lines of “But how about the poor in the world’s South, who need the cheap energy offered by oil to lift themselves out of poverty?” This is ridiculous. The onrushing climate catastrophe is going to hit those people way, way harder than it hits us rich white folk in the Northern Hemisphere who’ve mostly caused the problem. Also renewables are less capital-intensive and can be built faster.

Another pushback: “But a whole lot of the carbon is coming from dictatorships like Russia and China, which just don’t allow any flavor of activism, let alone sabotage.” Which is true but irrelevant. First of all, we need to prove that you can decarbonize an economy. That’s not actually in doubt and while it’ll be a huge effort, it looks like a whole lot of money is going to be made along the way and the society on the other side will be considerably more pleasant. But we still have to prove it. And just because my neighbor leaves garbage all over their front yard doesn’t mean it’s OK for me to.

In any case, it’s our moral responsibility to act.

Diablo Immortal 30 Oct 2021, 7:00 pm

I tried out Diablo Immortal on my Galaxy Tab 7+ and, well, it’s pretty convincing off the top. But I wonder if it’s ethical to play anything from Blizzard, these days? Protip: At the moment you can’t get into the closed beta unless you’re on Android in Canada.

History

I’ve never been a serious gamer — often going months or years between playing anything at all. But Diablo is kind of an exception; back in D2 days I was in a pretty serious Amazon-focused guild and put in a lot of hours. Then a mere nine years ago I took a D3 Demon Hunter to level 60 and then at some more recent point I worked a Barbarian pretty deep on the PS4. I was having trouble getting past Malthael when boredom set in.

So I’m pretty well steeped in the culture.

Ethics

When I turned on my Pixel and Google Play popped up the Diablo invitation, I was about to flick it away because who wants to spend hours dungeon crawling on a handset? Then I remembered the big tablet, which has a beefy CPU and 120Hz screen, and decided to give it a try.

As I was getting spun up, there was this little voice in the back of my head saying “Wasn’t there some nasty news about Blizzard recently?” Well, yeah, it turns out to have been a shitty company with shitty leadership and shitty culture. They got hit with state-government #metoo litigation, which they characterized as “irresponsible behavior from unaccountable state bureaucrats.”

Ewww. I mean, seriously. But wait… maybe there’s hope? The CEO got dumped and then, just last Thursday, Blizzard apparently hit “Reset”, making apologetic noises and announcing a range of measures with serious money behind them, and going along with a lot of the disgruntled employees’ demands.

Normally I’d be pretty cynical about this sort of corporate arm-waving, even though they’re doing a pretty convincing job of sounding like an organization that recognizes it fucked up and is sincere about getting its act together. But, the changes were welcomed by the ABK Workers Alliance, which says they are the direct result of widespread grass-roots employee pressure. Which is something I’m very strongly in favor of. If I hadn’t seen that I would have uninstalled the game already. Now I’m feeling uneasy and watching closely.

Diablo Immortal on a Galaxy Tab 7+

Heading out after Leoric

What it’s like

There’s only one question, really: Is it possible to have a convincing Diablo-like experience on a mobile device? My initial answer: Apparently, on a big expensive tablet. And on a handset? Dunno, and I’ve no interest in trying.

Anyhow, check out that screenshot, which I’ve enhanced a little in Lightroom, partly because Diablo on a tablet, just like every other Diablo, is very very dark. The way it works, you move around using your left thumb; you can see the move-around doohickey in the lower left corner, dimly; normally it’s hardly visible till you drop your thumb on it, then it lights up.

Then, with your right thumb, you bash away at your attacks and spells and health orbs and so on. The big circle down at the very bottom right is your primary attack and, just like regular Diablo, you spend most of your time bang-bang-banging away at it. The other attack buttons have mechanics like long-press to charge up an attack and so on. So I guess the biggest delta against “real” Diablo is that you can’t use the mouse to direct your attack.

In practice, it feels… a lot like playing on a PC or Playstation.

More random notes, in no particular order:

  • Aesthetically? Yep, Diablo, nailed it.

  • “Leoric?” So this is the same story as D3? Um… so far, yeah. In fact, I played last night until, at L18, I got ganked by the Skeleton King. And yep, Deckard Cain is there.

  • The little blue blobs at the top right are mobs entering the scene.

  • The combat is pretty crisp but, even on the big Sammy screen, it’s really hard to see what’s going on when you’ve twenty or so skeletons and rats and grave robbers clustering around you. Hey, just like Diablo!

  • Performance was perfectly OK, except for moving my toon around with the left thumb, which felt kind of sluggish. Maybe that’s because the Crusader character is just slow?

  • It looks pretty wonderful on that 2800 x 1752 OLED screen.

  • Dunno why I picked a Crusader. I might just go back and re-roll because it’s not fitting with my style, so far at least.

  • It’s multiplayer without any extra work or effort. I was a little surprised and disoriented when other characters were darting around my game. When you enter a dungeon, you get a chance to assemble a raiding party first. I haven’t tried this yet, but will before I go take another run at the SkellyKing. My crusader can tank him for a decent amount of time, probably long enough if there were effective ranged attacks happening.

I’ll keep an eye on this because I’m wondering if you could possibly have a good time playing this on a handset. But as of now, I’m convinced that there’s fun to be had on decent modern tablets, which I assume includes recent iPads. In 2001, if you’d told me I’d still be playing Diablo twenty years later, I’d have laughed at you.

And maybe I won’t be if it turns out Blizzard remains a shitty company.

URI-based Integration 26 Oct 2021, 7:00 pm

At the end of September, AWS announced a big new feature for its Step Functions product, and my tweet noting the announcement got a shocking number of impressions for something way out at the geeky end of Cloud tech. In retrospect, a design choice we made back in 2016 turns out to be working very well, and there’s a lesson to be learned here: If you need to integrate an arbitrarily large and and diverse set of software capabilities, URIs are the best integration glue.

Background

Step Functions launched in December 2016, and I did a whole lot of work on it. In particular, my fingerprints are all over the Amazon States Language, a JSON DSL that describes all the workflow stuff: What software to run, branches and loops, error handling, retrying, parallelism, and so on. In the States Language, each of the steps in the workflow is represented by a little blob of JSON called a “state”, and each has a Type field saying what it does. (I wanted to call the product “Amazon States” or “AWS State Machines” but Andy and Charlie puked at that and we ended up with Step Functions, which isn’t terrible.)

The argument

When we first cooked up the product, the only real target we had was Lambda functions, and so the suggestion was that we have a "Type": "Lambda" state, with another field that would give the name of the Lambda function.

But I said “Long-term, we want to be able to orchestrate lots of other things, not just Lambdas, right?” Everyone agreed. So I said “OK then, let’s just have a Task state which identifies the worker with a URI. That way, everything we orchestrate has the same contract, you send it some JSON and you get some JSON back.”

People looked a bit puzzled and said “But Lambdas don’t have URIs.” I said “Sure they do, they have ARNs and ARNs are URIs.” (Well, they would be if Amazon registered the “arn:” URI scheme, which I should have while I was there and they should now. But close enough.) There was a little push-back on making people use the long klunky-looking ARN as opposed to the nice user-friendly function name, but I was pretty convinced and eventually won the argument.

I was remembering the dawn of the Web, quoting from someone (I think TimBL?) who said “On the Web, a resource is a unit of information or service.” Which I thought was a good fit here.

Flashing forward five years

Let’s just have a look at what Step Functions has been integrated with. Start here, and scroll down (there are duplicate anchors, grr) to the “Service Integrations” header, and look at the table. As I write this, there are 17 “Optimized” integrations, and then 200+ SDK-based integrations. And they all use the same Task state and address the target worker by URI (which at the moment is always an ARN).

The ARN for an “Optimized integration” looks like (taking EMR for an example): arn:aws:states:::emr-containers:createVirtualCluster. “Optimized” means it’s smart about the way it calls the service and can operate in either fire-and-forget or wait-for-completion mode. Also, it can autogenerate IAM policies to make your life easier.

The recent announcement that kicked this discussion off made it possible to call more or less any API in the AWS SDK, addressing it with an ARN like:

arn:aws:states:::aws-sdk:emrcontainers:createVirtualCluster. There’s a really excellent blog that walks you through the process.

I’m happy

I’m feeling just the tiniest bit smug that they were able to add all these integrations, and in particular this latest huge one, without needing to make any major changes to the States Language.

But to be honest, all of that comes more or less for free once you decide that everything you might want to integrate is a resource and thus should be identified by a Uniform Resource Identifier. I recommend this design pattern.

The future

I’ve always thought that once you agree to address things by URI, well that includes HTTP URLs, so why shouldn’t a Step Functions Task state be able to include an arbitrary external Web endpoint? SNS can already do this. Now… it’s kind of scary making an AWS service take a runtime dependency on an uncontrolled external anything, so this would be tricky to implement.

But it’s another thing you could do with no language changes, just because you decided to do things the Web-native way.

Worst Case 8 Oct 2021, 7:00 pm

Suppose you’re running your organization’s crucial apps in the cloud. Specifically, suppose you’re running them them on AWS, and in particular in the “us-east-1” region? Could us-east-1 go away? What might you do about it? Let’s catastrophize!

Acks & disclaimers

First, thanks to Corey Quinn for this Twitter thread, which got me thinking.

Corey Quinn tweet on us-east-1

Second, while I worked for AWS for 5½ years, I’ve never been near a data center, nor do I have any inside information about the buildings, servers, or networking. On the other hand, I do have a decent understanding of AWS culture and capabilities in software engineering and operations. Bear those facts in mind as you read this.

Finally, since this blog fragment concerns itself entirely with catastrophic scenarios, I’ll try to be cheerful about it.

[Those of you who know what us-east-1 is can skip over the next section to the first entertaining disaster.]

“us-east-1”?

AWS means “Amazon Web Services”, Amazon’s insanely huge ($60B/year revenue) and profitable (~30% margin) collection of cloud-computing services. Basically, AWS will rent you computers and databases and the use of many other software services. So more or less everything your IT department owns can be rented by the hour (or second) rather than installed in your own data center.

If you’re using AWS, you have to pick one (or more) of its (24, as I write) “regions” to host your systems. They have boring names like “us-west-2” (Portland) and “ap-northeast-1” (Tokyo).

“us-east-1” (N. Virginia) is generally thought to be the biggest region, by a huge margin. There have been estimates that 30% of all Internet traffic flows through it. Here’s AWS’s official write-up and here’s a nice Atlantic story by a person who drove around Northern Virginia looking for the actual buildings.

Before we leave the subject, I should say that each AWS region is divided into multiple “availability zones” (AZ’s), data centers that are independently operated and geographically separated, so to really lose a whole region, you”d have to take all of them out.

If us-east-1 went off the air, it would be Really Bad. How could that happen?

Threatening clouds

Threatening clouds.

Terrestrial disaster

This is the first one anybody thinks of.

Suppose a big late-summer hurricane somehow misses Florida and Texas, cruises north offshore picking up energy from an anomalously-warm western Atlantic, turns left just south of DC, and savages anywhere that’s easy driving distance from Dulles airport. We’re talking about inches of rain in a few hours so every waterway floods; also, high winds and lightning are playing hell with the electrical and network infrastructure.

The other obvious candidate would be an earthquake, which can ravage infrastructure to a degree unequaled by any other flavor of natural catastrophe. Among other thing, the Potomac bridges and lots of freeway overpasses would be rubble, so your ability to bring help in would be severely reduced.

If you’re the unlucky proprietor of systems hosted at us-east-1, they’d be off the air, and while AWS would probably arrange to answer your distress call, there’s really not much that could be done. How would your business do if it were off the air for, uh, nobody really knows how long?

How much should you worry?

This one worries me less than a lot of the other scenarios here. First off, the hurricane scenario is so utterly predictable that I bet anyone with a significant data-center presence in the region has been planning and wargaming around this one for at least a decade.

Modern data centers all come with self-contained backup generators and some sort of power-bridging gear, so assuming the water doesn’t actually get in and flood the equipment rooms, things should be fine. You’d expect Internet-provider outages as well, but once again, modern data centers strive for redundant connections and are built in places where there are multiple providers, so they’d all have to go down to go completely off the air.

Having said that, the climate is changing and possibly, everything we know about that storm system will turn out to have been wrong.

The earthquake scenario is tougher, but fortunately that’s not a seismically active zone.

Also bear in mind that the availability-zone architecture is going to help you. You can imagine one data centre’s backup power failing to operate, but it’d be really unlikely for that to happen in all the AZ’s.

I’m not sure this is much consolation, but: If an event of this scale occurs, you’re not going to be the only operation who’s off the air. Probably, quite a lot of the United States government would be in the same boat. So while your customers and employees are going to be mad, they’re also going to be distracted from worrying about your downtime.

Extraterrestrial disaster

What about devastation raining down from space?

Sophie Schmieg on solar storms

Sophie Schmieg is a high-level cryptography/security Googler, and Knows What She’s Talking About. She refers to the Carrington Event, a major solar storm (“Coronal Mass Ejection” they say) that happened in 1859, and severely disrupted the world’s telegraph system for about eight hours. This is an example of a Solar proton event. If/when one happens, it’s going to seriously suck for astronauts and for anyone who depends on aerial radio-frequency communications. How hard will it hit modern data-center and Internet infrastructure? The deepest dive on the subject seems to be Solar Superstorms: Planning for an Internet Apocalypse (PDF) by Sangeetha Abdu Jyothi.

Physicists I’ve talked to say “Yeah, that’s gonna happen someday.” Bear in mind that since the duration is measured in hours, we might get lucky and find us-east-1 facing away from the sun.

How much should you worry?

I figure that this is actually a more likely disaster scenario for us-east-1 then either the hurricane or the earthquake. But I’ve got no special insights into how much it will hurt. In Abdu Jyothi’s paper, she offers lots of specific recommendations about how to solar-storm-proof the infrastructure. How much have the operators of us-east-1 tried, and how well will their efforts work? We don’t know.

However, as with the terrestrial disasters, your personal pain may not matter that much. After all, as Abdu Jyothi points out, “A recent study … which analyzed the risks posed by a Carrington-scale event to the US power grid today found that 20 - 40 million people could be without power for up to 2 years, and the total economic cost will be 0.6 - 2.6 trillion USD.” So… there’s not going to be much leftover attention for your little outage.

Labor unrest

It’s increasing around the globe as multiple decades of increasing inequality in wealth and power bite down harder and harder. Also, it may turn out that Covid has disturbed the balance of power between the working and owning classes. A wave of Big Tech unionization would be surprising, but not that surprising.

So here’s the scenario: Some group of employees whose services are essential for the operation of us-east-1 wins a unionization vote and starts trying to negotiate a contract with AWS, because they’re looking at that 30% margin on the tens of billions in revenue.

Unsurprisingly, Amazon goes all hard-ass, explains that unionization is incompatible with Day One thinking and Amazon Leadership Principles, and refuses to talk. So they take a strike vote, and on one fine spring day, don’t come to work. Nobody’s watching the graphs, whether those are graphs of electrical-supply stability, fiber-repeater failures, or data-storage latencies. How long does us-east-1 stay operative? I have no idea. But it’s a terrifying scenario.

It’s going to be difficult to explain to your customers that you can’t service them because of a labor dispute between a company they’re not dealing with and a union that doesn’t contain any of your employees.

How much should you worry?

Not at all. This will never happen.

Let’s ignore the passion and fury with which Amazon will resist unionization, and suppose hypothetically that things proceed as described, the strike vote passes, and it’s becoming apparent that several thousand essential workers are absolutely not going to show up on a near-future morning. What happens? Amazon caves instantly and does whatever it takes to come to a settlement with the workers.

The company is always talking customer obsession and that’s no BS, they really mean it. Failing to provide services that customers pay for and rely on because of internal management failure (and this is one of those) is violently antipathetic to Amazon culture. So they just won’t let it happen.

AWS software or operational failure

I’m talking about something like what happened to Facebook this month: For reasons that nobody who’s not a serious software geek can understand, us-east-1 suddenly vanishes from the network. Or is still on the network but is refusing all requests. Or is accepting requests but timing them out. Or is accepting requests but returning empty answers.

Once again, you’re in a bad spot when you have to explain to your customers that you’re off the air because you made a bet on a provider who couldn’t deliver the goods.

How much should you worry?

I’m not going to say this could never happen. But I’d be shocked. AWS has been doing cloud at scale for longer than anyone, they have the most experience, and they’ve seen everything imaginable that could go wrong, most things multiple times, and are really good at learning from errors.

Also, AWS has a powerful and consciously-constructed culture of operational excellence based on extreme paranoia. To be honest, I’m just the tiniest bit concerned over the recent departure of Charlie Bell, because he, more than anyone else, deserves credit for building and maintaining that culture. But it runs very deep.

War

It doesn’t seem likely that foreign attackers are going to swarm ashore on the Virginia beaches and send tank battalions through the industrial parks to blow up us-east-1. So maybe you don’t need to worry?

But wait; how about civil war? Let’s see; suppose Trump wins the Republican nomination for 2024, and runs on a rabble-rousing campaign of Revenge For The Steal, and explicitly rallies the Proud Boys, Oath Keepers, Sovereign Citizens, Three Percenters, Groypers, and police unions, telling them, “We can’t lose in a fair election, so if we do, let’s not let them steal it again.”

His election rallies are stuffed with Second-Amendment fanatics brandishing assault weapons. Every debate and campaign interview features questions along the lines of “If you lose, will there be an insurrection?” The majority of voters are out of patience with Trump and vote in Kamala Harris by a decent popular margin, but once again it’s a squeaker in the Electoral College.

The Trump supporters scream “Steal!” and launch a march on Washington; it turns out they have support from significant factions in the police forces and the US armed forces. Northern Virginia becomes a key strategic battleground, and both sides deploy heavy artillery…

OK, that’s a little far-fetched (I hope). Here’s another scenario: Beijing launches an invasion of Taiwan and the US comes to its defence. China’s cyberwar apparatus turns out to have discovered multiple zero-day attacks against Internet exchanges, poison pills that knock BGP off the air and keep it from coming back up. In this scenario, us-east-1 may be up and running, but nobody can reach it.

How much should you worry?

Probably not very much. Like the hurricane or solar storm, your problems are going to vanish in the static.

Enemy action

In this scenario, the Bad Guys (who knows, maybe those Chinese cyberwarfighters I just mentioned) figure out some combination of poison pills and DDOS and Linux kernel zero-days to knock over us-east-1 and keep it that way.

Once again, there you are explaining to your customers why AWS’s incompetence is screwing up their lives.

How much should you worry?

Not at all; I just can’t see this happening. I remember an AWS meeting with a customer looking at moving to the cloud, who asked “What about DDOS attacks?” The Amazon executive in the room said “Yeah, there’s probably three or four of those going on right now, they’re a cost of doing business for us.” There’s nobody in the world with more experience than AWS in dealing with this kind of crap.

But there’s a bigger reason. The vast majority of hackers are in it for the money, and they know perfectly well that AWS has one of the best-defended attack surfaces on the planet. So it’s in their interest to go after softer targets; big companies with juicy customer lists and password files and so on who aren’t minding their perimeters.

Note: You might be one of those big companies; while AWS is generally secure, it’s possible to run insecurely on it. So while the Bad Guys might come after you, they’re almost certainly not going to go after us-east-1 as a whole.

Public legal risk

It seems quite unlikely that any force of nature or criminal action could wipe out us-east-1. How about the US Government? Bear in mind that Republicans hate Amazon because of Bezos’s Washington Post and because the whole tech industry is (somewhat correctly) perceived as progressive.

Suppose Trump or some guttersnipe like Cruz or DeSantis wins the Presidency in 2024, and the Republicans control congress. Could AWS survive a US Federal legal move that forced a us-east-1 shutdown? Could it even survive a continuous credible threat of such a thing happening? The temptation might be too much for the GOP goons.

How much should you worry?

I would. But in a more general way; the existential peril to the USA following on the exercise of power by the Trumpist faction seems to me very severe, not something that can be ignored. So I would be watching which PACs I donated money to, and encouraging grassroots political activism to stave off the wreckage before it happens.

But then, I’m on the respectable left of the Canadian political spectrum, which makes me a raving Commie by US standards.

Surviving

Let’s assume you’re not going to wait for us-east-1 to come back, you want to resume operations elsewhere. So, you need to pick another region. Depending on which scenario worries you the most, you might want to be (as Sophie Schmieg suggested) in a different hemisphere, or if you’re worried about political/legal risks, at least a different jurisdiction.

The best thing you could possibly do is, don’t wait: Run “active-active”, which is to say have your application live in both regions all the time. Netflix kind of wrote the book on this, for example consider this 2013 write-up. I’ll be honest: I don’t know if Netflix has ever actually failed over in the face of an actual region outage. But their thinking is correct: The only way you can be sure that your backup region will run in production is by running it in production.

But let’s suppose you’re less ambitious; you’re not going to try to keep operations running continuously in the case of a failed region, you just need to be able to get back on the air in a reasonable amount of time, probably accepting that some transactions happening just as disaster struck might get lost.

Your app inventory, if it’s typical, probably includes virts running your code, along with load-balancing and fire-walling gear, and your code accesses a variety of services such as messaging systems and databases and serverless stuff. Let’s assume you’ve got your configurations all stored as code with Terraform or CloudFormation or whatever, so that if you needed to rebuild the system from scratch, you could. You do, right? Seriously, given that, if us-east-1 got blown to hell and you have a copy of the config code, revivifying your app is plausible.

Then there’s your data, which lives in some combination of databases, filesystems, and S3.

S3 has had region-to-region replication built in for a long time, and clearly people at AWS have been thinking about this; consider Introducing Multi-Region Asynchronous Object Replication Solution.

If it were me in my ideal world, I’d have copies of everything stored in S3 because of its exceptional durability; I sincerely believe there is no safer place on the planet to save data. Then I’d have a series of scripts that would rehydrate all my databases and config from S3, reconfigure all my code, and fire up my applications. I’d test this script regularly; any more than a few weeks untested and I’d lose confidence that it’d work.

Anyhow…

We probably won’t lose us-east-1. I’m not absolutely 100% sure that these scenarios are even worth thinking about, in a strictly economic sense. But if I were running a big important app, I wouldn’t be able to not think about it.

Catharsis 2 Oct 2021, 7:00 pm

It was just a minor story in the business section, nothing front-page. But when I heard (earlier in the day, from an insider), the news was cathartic; I had to stand up and take a walk. Summary: Amazon was about to find itself in court with the US National Labor Relations Board over the firing of Emily Cunningham and Maren Costa; the event that caused me to walk away from the best job I ever had and a whole lot of money. On September 29th, Amazon avoided that trial by signing a humiliating out-of-court settlement. Which is to say, that particular firing spree was not only unethical and stupid, it was probably illegal.

If it’d gone to trial, I’d have been a witness. I wasn’t looking forward to that.

Which is really about all I have to say. No, wait; some of you might not follow a link to Emily and Maren’s statement, so here it is:

Statement from Maren Costa and Emily Cunningham

We are thrilled to announce that we have reached an agreement to settle the charge against Amazon at the National Labor Relations Board (NLRB) alleging that the company illegally fired us for speaking up about warehouse workers' conditions during COVID. This is a win for protecting workers rights, and shows that we were right to stand up for each other, for justice, and for our world. Amazon will be required to pay us our lost wages and post a notice to all of its tech and warehouse workers nationwide that Amazon can't fire workers for organizing and exercising their rights.

It’s also not lost on us that we are two women who were targeted for firing. Inequality, racism, and sexism are at the heart of both the climate crisis and the pandemic.

Tech workers standing up together have immense power to move the biggest corporations in the world. Everything we love is threatened by climate chaos. Workers at every company need to be standing up for each other and the world, together. Now is the time to be our best, bravest selves. We can only do this together. We hope you’ll join us.

Twitter: @AMZNforClimate
Email: info@amazonclimatejustice.org

Strong work, Emily and Maren!

I wonder what else might be going on involving the NLRB and the other activists they fired in the early days of Covid, for example Courtney Bowden, Gerald Bryson, Bashir Mohammed, and Chris Smalls?

Rock Decades 26 Sep 2021, 7:00 pm

On Saturday September 25th, 2021, we went to a concert by Cousin Harley featuring Paul Pigat. The last live indoor music we’d seen was March 8, 2020, which is to say a gap of 566 days. Speaking of gaps, 2021 marks, more or less, both seventy years of Rock ’n’ Roll and 50 years of my loving it.

Weird coincidence: That last pre-Covid concert was by Coco Montoya “and Paul Pigat”, my calendar says, but I’ll be honest, I remember almost nothing of that evening.

Cousin Harley, live at the Rickshaw

Live at The Rickshaw.

Cousin Harley

This was the first event at the lovable but ratty old Rickshaw Theatre run under the latest Covid protocols: Max audience 250, vaccine passport required, stay in your seat, mask on unless you’re actually drinking, no dancing. The organizers apologized but also rejoiced in the return of live music. And yeah, when the guitar struck up, I found myself a little emo.

The band is one of two Paul Pigat projects; its musical center is rockabilly, but they sprawl pretty far across the guitar-trio landscape. Pigat himself is a fine axeman in a whole lot of styles, wasn’t boring for a moment. He got an exceptionally beautiful tone out of some little amp I didn’t recognize, particularly in a slightly-surfy instrumental.

This music is played fast, which means you need a strong drummer, and he was, but the web site doesn’t mention who he is, which seems kind of churlish.

Pigat, in between-song banter, mentioned that he not only writes songs, he builds guitars, although I can’t turn up any evidence online. On the strength of last night’s show, I’d say they’re good.

One last footnote: I notice, on Pigat’s website, a tip o’ the hat to the Canada Council, the Canadian government’s arts-funding organization. Which shocked me a bit; I’d always seen the Council as focused on big-budget grownup stuff like opera and ballet. It’s cool that some of my tax dollars are going to keep Rock music alive during these hard times.

Anyhow: We had a wonderful time. It felt exotic and grown-up to go out to an evening concert. Live music, yeah! And if you care about music and musicians, and if your local Covid conditions allow, go out and buy some damn concert tickets already. In this era of music streaming (which means, basically, musicians starving) the best way to route a few bucks to the artists is going to shows.

And, it beats the fuck outta doomscrolling.

History

The conventional wisdom is that the first Rock ’n’ Roll record was Rocket 88 in 1951, featuring Jackie Brenson on vocals in front of Ike Turner’s band. The argument isn’t crazy, but the song lacks a backbeat, which troubles me. How about Wynonie Harris’ take on Good Rockin’ Tonight, from 1949? Anyhow, close enough for Rock.

I’m sixty-six, which is pretty old. But the music I love is older. I discovered Rock in 1971 when I got the Beatles’ White Album for my birthday. Oh wait, the evidence on this blog says that it was ’68, when my Dad turned me on to Cream. (No, really!) Whatever, once again close enough.

Anyhow, by the time I started listening, the Beatles were breaking up, Joplin and Hendrix were about to die; I pretty well missed the glory days. That’s how old the genre is.

My parents (despite my Dad’s contribution) hated Rock and hated me listening to it. In my room I had a record player, the kind you could stack the vinyl up on and it’d play a side then drop the next record and play that. In only had one speaker, but it was pretty big and pretty loud.

Thus started the trend, which continues to this day, of the women in my life telling me to turn down the damn music. One time, Mom showed me an article she was reading saying how some Eminent Musicologists had done A Thorough Analysis of Moderm Pop Music and Determined Authoritatively that it was Pretty Well All Trash. I objected that I really liked it and she said “You’re discounting the value of education!” To which I had no come-back. But I kept listening.

I feel like a Proust character, because in my mind Rock still feels fresh and new and enjoyably offensive. But in fact it’s the stomping grounds of the much-reviled Boomer generation who, to quote Mose Allison’s Young Man Blues, have all the power and all the money. But I dunno, I keep seeing young folk wearing T-shirts with Aerosmith and Zeppelin and Tom Petty on them.

Anyhow, I’ll be going to lots more concerts (Covid allowing) and so should you.

Blueskid Demo 19 Sep 2021, 7:00 pm

I’ve been in the conversation around Twitter’s @bluesky project, and last December I posted @bluesky Identity, a proposal for mapping between social-media identities based on public keys and signatures. Recently @bluesky announced the Satellite contest, whose goal is to take identities on three or more online properties and “Link them in a way that anyone can verify you are the author/owner of all.” Which is more or less what @bluesky Identity is all about. So I pulled together a working demo called “Blueskid” (GitHub). This is a quick walk-through of Blueskid.

Blueskid?

Well, I needed a project name and “@bluesky identity” has six syllables. “Blueskid” is euphonious and only has two, and blueskid.net was available. And I get this mental image of a kid playing blues.

Contest?

Blueskid is not an entry in the Satellite contest. First of all, I’m sort of a @bluesky insider and the idea is to bring in ideas from the community. Second, Satellite is looking for something with a focus on decentralization and radical innovation. Blueskid uses public-key and ledger technologies that, in the software-technology context, are as old as dirt.

I offer Blueskid as a low bar that the Satellite offerings really ought to raise.

Let’s watch it at work.

The assertions

Here’s a recent twitter post:

twitter.com@timbray claims the Bluesky Identity 55555

https://twitter.com/timbray/status/1438391330879590400

This tweet contains an “assertion” representing a claim by the Provider Identity (PID) twitter.com@timbray to the Bluesky Identity (BID) 0000000000055555. Let’s unpack that.

  1. We know that the claim is from twitter.com@timbray because it’s posted to the @timbray Twitter account.

  2. Since we’re going to be posting assertions to social media, they need syntax to delimit them from any other text that might be in the post, and to separate the fields. Since I’m a fun-loving guy, the beginning and end of an assertion are marked by "🥁" (U+1F941 DRUM) and the fields are separated by "🎸" (U+1F3B8 GUITAR). I’m not going to claim this is optimal but it worked OK in the demo.

  3. This assertion has two fields. The first is “C”, saying that this is a Claim assertion. The second gives the BID that’s being claimed.

  4. In Blueskid, Bluesky Identities are represented by unsigned 64-bit integers. There’s a lot to be said about how they might be structured and minted, but for the purposes the demo we just need something that can be represented in a string, in this case upper-case hex characters.

Another Twitter post:

twitter.com@timbray grants BID 55555 to tumblr.com@t-runic

https://twitter.com/timbray/status/1439270526598332424

It was followed shortly by a Tumblr post:

tumblr.com@t-runic accepts BID 55555 from twitter.com@timbray

https://t-runic.tumblr.com/post/662682878549852160/mpe

These two assertions are designed to work together. Each has six fields:

  1. In the first, “G” says this assertion Grants a BID, “A” that it Accepts one.

  2. The second field gives the BID being granted.

  3. The third is a nonce (in base64). This is currently 64 bits, which is kind of short by nonce standards, and I need to find someone with real cryptographic/security skills for advice. I’m having trouble thinking through attack models. At the moment I think 64 bits is plenty. But it’d be unsurprising if I were wrong.

  4. The fourth is an ed25519 public key, once again in base64. The encoding uses the horrible old ASN.1/PEM/PKIX machinery, which would be silly if the whole world used Go, but many other popular libraries in popular languages assume this is the one and only way to interchange public keys. Thus it’s the right thing to do in an Internet Protocol.

  5. The fifth is the signature (base64 again) produced by applying the corresponding private key to the nonce.

  6. The fifth is the counterparty, in the Grant assertion the receiving PID and in the Accept assertion the granting PID.

There is a another pair of posts to grant that same 55555 Bluesky Identity from twitter.com@timbray to mastodon.cloud@timbray, here and here. Also, note that:

  1. The nonces are different and so are the signatures.

  2. The keys are identical.

  3. The BIDs are identical.

  4. The Grant post is known to be published by twitter.com@timbray and names tumblr.com@t-runic, while the Accept post is known to be published by tumblr.com@t-runic and names twitter.com@timbray.

You might ask where the private key corresponding to the public key is stored. The answer is “nowhere”; it existed in the Blueskid server just long enough to produce the two signatures, then it was overwritten. It doesn’t exist any more.

It is my belief that these social-media posts, taken together, establish that at some point the owner of twitter.com@timbray and of tumblr.com@t-runic had access to the same private key, and published commitments respectively to grant and accept the “55555” BID. (The same exercise was performed for mastodon.cloud@timbray.)

Blueskid also knows about an “Unclaim” assertion, not illustrated here, whose effect is what you’d expect.

Q.E.D.

My claim is that these assertions in social-media posts constitute a verifiable proof that the same entity controlled both PIDs and expressed an intent to share a BID.

But, even if you agree with me, the social-media posts by themselves aren’t very useful. If you wanted to know what BIDs exist and which PIDs they’re shared between, you’d need to read all the posts from everyone in the universe and look for Blueskid assertions. So…

The Ledger

As the @bluesky Identity post outlines, you need a Ledger to make this work. For each of the BID Claim, Grant, and Unclaim assertions, there needs to be a Ledger entry noting what has been done and pointing to the social-media posts that prove it. The Ledger needs to be publicly readable and reliably immutable. Clearly, by processing the Ledger, you can build a little database of what BIDs exist and which PIDs are mapped to them.

The Ledger could be constructed with blockchain technology. That’s not how I’d build it if you asked me to, but it’d work OK. The write rate is probably low enough to survive blockchain’s pathetic update performance.

There’s an important issue the Ledger needs to address, based on the fact that social-media posts are not immutable; even Tweets can be deleted. Simply because I publish an assertion pair like the one illustrated above doesn’t mean that everyone can be confident that they can go and verify it years hence.

Therefore, the Ledger implementation needs to make a believable claim that it won’t append anything to the ledger that it hasn’t verified by fetching the social-media posts and validating all the constraints listed above. I’m not sure what the best way to achieve this is, but I have one idea: There could be multiple implementations, each reading new assertions as they are added to the ledger, repeating the verification, and rejecting assertions that can’t be validated. Hey, this is starting to sound like a blockchain.

What Blueskid does

First of all, it helps generate assertions. For example, you can ask it to make that Twitter/Tumblr BID grant assertion pair for you. Send this to the /grant-assertions endpoint:

{
  "BID": "55555",
  "Granter": "twitter.com@timbray",
  "Accepter": "tumblr.com@t-runic"
}

Then it’ll come back with:

{
 "GrantAssertion": "🥁G🎸55555🎸0E8hIvntXJc=🎸MCowBQYDK2VwAyEAzHaDqVdyhle4wVY/leNyZrtBKJKMVqVWZFfVJ3S8v60=🎸U1vPM6cQ+c5rdTKwa/2l/wjr2Z0Zu33t/qE59+94Ni/0TjEjDqcAZ/LfaFcJ6i+v+uLNhiN5LeiekFYByPWVAQ==🎸tumblr.com@t-runic🥁",
 "AcceptAssertion": "🥁A🎸55555🎸V5+dt5Me0kw=🎸MCowBQYDK2VwAyEAzHaDqVdyhle4wVY/leNyZrtBKJKMVqVWZFfVJ3S8v60=🎸1fLK2wHtRA24c/wu9uiiB42WOFur3TI9VozsYKImY0Vq3HgwDJU6xCX8GiW8rM+KIjOUTem6sQt5vTybK+dbCw==🎸twitter.com@timbray🥁"
}

Then, once the assertions are posted to social media, you can update the ledger. Here’s an example of a post that records the Twitter/Tumblr assertion pair, which you’d post to the /grant-bid endpoint:

{
  "GrantPost": "https://twitter.com/timbray/status/1439270526598332424",
  "AcceptPost": "https://t-runic.tumblr.com/post/662682878549852160/mpe"
}   

After you’d posted that, sending a GET to the /ledger endpoint would yield this:

{
 "Records": [
  {
   "RecType": 0,
   "BID": "0000000000055555",
   "PIDs": [
    "twitter.com@timbray"
   ],
   "PostURLs": [
    "https://twitter.com/timbray/status/1438391330879590400"
   ],
   "Key": ""
  },
  {
   "RecType": 1,
   "BID": "0000000000055555",
   "PIDs": [
    "twitter.com@timbray",
    "tumblr.com@t-runic"
   ],
   "PostURLs": [
    "https://twitter.com/timbray/status/1439270526598332424",
    "https://t-runic.tumblr.com/post/662682878549852160/mpe"
   ],
   "Key": "MCowBQYDK2VwAyEA7bk+ldmZEGCSAdR1RQek1nQ4Lp058QpcaNGnDlfsS/A="
  },
  {
   "RecType": 1,
   "BID": "0000000000055555",
   "PIDs": [
    "twitter.com@timbray",
    "mastodon.cloud@timbray"
   ],
   "PostURLs": [
    "https://twitter.com/timbray/status/1439271202699157511",
    "https://mastodon.cloud/@timbray/106953703798946745"
   ],
   "Key": "MCowBQYDK2VwAyEA+BBQLd4ks4vdJZzX1F4j51gtyfJpLBFpeqkT7t5GJ/0="
  }
 ]

I’m not going to spelunk through the JSON, but it says that the BID was claimed then granted twice, and links to the social-media posts which contain the assertions that prove it.

The code tries to be careful. It blocks BIDs from being claimed more than once and, when it processes assertion pairs, takes care that all the conditions listed above apply: The BIDs and keys are the same, the nonces and signatures are different, the signatures validate, and so on. Also it enforces the @bluesky Identity constraint that no public key can be used in more than one BID-grant transaction.

It also provides endpoints that let you query all the BIDs associated with a PID (/pids-for-bid), the reverse query (/bids-for-pid), and given any PID, list the group of PIDs of which it’s a member and which are mapped together via at least one BID. Here’s a little terminal session:

Interacting with Blueskid on the command line

What Blueskid doesn’t do

It doesn’t actually post the assertions to the social-media sites; I did that by hand. This will require a lot of API wrangling and the APIs are frankly not that lovable. It does actually use the Twitter V2 API to retrieve tweets. But Tumblr and Mastodon are just HTTP GETs followed by code that roots through their horrible HTML to find the assertion.

Blueskid’s ledger is a fake. It’s in memory, not persisted at all, and it doesn’t do signature chaining to ensure that it’s immutable. Databases and Merkle trees are hard, but implementing them to do this kind of thing is a fully solved problem.

Acknowledgments

The idea of establishing key ownership by publishing signed assertions in social-media posts is originally due to Keybase.IO, quite some number of years ago.

This work has benefited from several interventions by Paul Hoffman.

Saving þ 9 Sep 2021, 7:00 pm

Herewith a lost-pet story with (spoiler) a happy ending, starring a real bloodhound. Soon to be a major motion picture, I bet.

Hunter

Here’s the hound, who’s called Hunter.

What happened was, we’ve been introducing our 9-month old cat þ (pronounced “Thorn”, because reasons) to the great outdoors. This is hard to avoid because we have a back porch we eat most meals on when it’s nice, and it’s a pain in the butt to not have the door to it open.

It’s been going great because þ is a cautious kind of cat. He skedaddles back inside or up a tree if a car goes down the road outside or someone drops a spoon on the floor or even sneezes. We kept him on a leash/harness but then after a while stopped, because he seemed to have no interest in going outside our yard. He’d flit briefly under the fence to the neighbor’s yard to investigate a chittering squirrel, but stays away from the street in front and the alley in back. And he never wanted to stay outside long.

Then, last Saturday, we needed to make an overnight trip to our cabin in connection with the renovations; our 22-year old son who lives in our basement was happy to mind the fort. When we left, þ was on a chair on the back porch, fascinated with all the to-ing and fro-ing with totes and duffel bags.

þ the cat

The picture we used in the lost-cat posters.
[Photo: Lauren Wood.]

When we came home Sunday, we discovered our son hadn’t seen him since we left. Suddenly we were The People With The Lost Pet. We put up posters. We walked the neighborhood, whistling and calling. We advertised on the SPCA lost-animals page and one on Facebook. Since þ is microchipped, we updated that registry too. Lauren, who is more assiduous than I, talked to more neighbors in 48 hours than we had in the last year or two. Everyone was sympathetic.

Now, we live in a dense urban neighborhood with lots of cars and the occasional coyote, not to mention, from time to time, damaged and possibly predatory humans. So, there are risks for cats. But still, it didn’t make sense, we just couldn’t figure out the scenario in which he’d managed to get far enough away to be really lost. We’d had other cats that roamed far and wide and casually walked into neighbors’ houses, about whom we worried terribly, but they all had long peaceful lives.

Eventually we ran across Pet Searchers Canada, whose service is comprehensive and bloodhound-assisted. Also it’s not cheap, but we were feeling pretty emotionally beat-up. After we’d signed the contract and paid, Savannah the handler showed up with Hunter the bloodhound.

Hunter the bloodhand

We provided þ’s harness as a scent sample, then Savannah and Hunter vanished for the best part of two hours. Savannah explained that she’d picked up his scent on the next street south and the intervening alley, and the western continuation of our alley. She told us she’d send a marked-up map showing where Hunter got the strongest signal, and advised us to go out after dark to check those areas out. Also to take along well-worn clothes, heavy with our scent, and drag them along behind to lay a scent trail us as we came home from our expedition, on the theory that the poor little unadventurous guy had got a little too far away and just didn’t know the way home.

She emailed us the marked-up map and Wednesday evening that’s what we did. And as we walked along, I occasionally offered the special “come here for a treat” whistle, and Lauren made “come for dinner” sounds. After dark is definitely the time to do this, because it’s quieter and also people aren’t going to be giving you the side-eye for walking along trailing a pair of jeans in the grass behind you.

While we were walking along the alley between our street and the next one south, territory we’d tried before albeit not in the quiet and the dark, suddenly there were plaintive cat cries answering our whistles and calls. þ has a high-pitched and penetrating voice when he has something he thinks it’s important that we hear.

We converged on someone’s tall back fence that faced the alley with an apparently-locked gate, but there was a gap under it and almost immediately, a pointy furry little black face looking through it. The gap was pretty low and þ had to put in some real squirming, but he made it out.

Now he’s at home. He won’t be going out for a while. He had a minor injury on one front leg, but already healing, no call for a vet visit.

What’s shocking is that he was in the back yard of a house across the street and maybe two houses east of us. I guess if you stay in your own yard you’re not going to learn how to find your way home. There’s a lesson in that.

Now, our neighborhood has plenty of cats and they do not co-exist very peaceably, so I suspect that poor þ was cornered and chased by one of the local feline bullies and that’s how he misplaced himself. Our daughter has vowed to walk him around the ’hood on his harness whether he wants to or not, so that he’ll at least know the nearby territory.

Judging by the number of posters on utility poles I see, pets often go missing. But you can do more than put up posters. I recommend bloodhounds.

LG C1 6 Sep 2021, 7:00 pm

What happened was, our TV is ten years old and (following on some renovations) we could use one at the cabin for rainy winter evenings. So I bought a 48" LG C1 4K OLED screen (48" is the smallest in the C1 line), which is kind of this year’s hot TV. It’s not a life-changer, but the world of TV has shifted some in a decade, so here’s a dispatch from the front. Includes a pointer to a truly great TV stand.

By the way, this thing seems to be on sale at all the big boxes, which is a little weird given the global supply-chain crunch. We got ours at Costco, but in the US it seems Amazon has ’em cheaper. [Caution: Affiliate link.]

OLED

The Wirecutter and several other review sites I visited seemed pretty unanimous that LG and Sony OLED are ahead of the pack, and LG is quite a bit cheaper. Also I liked the look of the sets. OLED, compared to other display technologies, is said to offer more dynamic range, better color gamut, faster pixels, and other goodies.

What they keep coming back to is “blacker blacks”, so bear that in mind.

LG C1 TV

Blacker blacks!

4K

It means, basically, 3840×2160, which is to say 8,294,400 pixels. Does anyone really need that many? I got interested in the subject back in 2013 when 4K was new, and wrote code to figure out if there’s any value-add compared to regular HD (1920x1080, a mere 2,073,600 pixels).

In Is 4K BS? I concluded that the pixel count probably didn’t matter, but boy did that blog piece ever go viral, so I ended up writing More Things About TV, which noted (among other things) that the 4K spec also bumps the frame rate and color depth. And here too, blacker blacks come up.

So, does it make a difference?

Yeah, but… well, first of all, modern TVs have way smaller bezels, so even though our video cave is pretty small, we were able to replace a 42" model with a 48". And that does add impact. Also, these days they’re thinner and sleeker and generally easier on the eye — among other things, the backs are smooth black surfaces without vents and other uglification. So in fact I could have squeezed in the 55" but it’s OK, what we got is big enough.

As for the rest, well — and here’s the important part — it really depends on the source. Like a lot of people, our family watches streaming services, mostly episodic TV but some movies, and also live sports via a cable provider. Except for baseball, for which we subscribe to MLB.tv. The cable is still on 720p, no 4K there.

So, if you have a well-photographed, well-produced show, and if its visual palette is sort of noir, then yeah, a big 4K OLED is going to make you smile and say “Wow!” A couple of examples would be Lupin on Netflix and especially The Expanse on Prime. In particular the later seasons. Space, baby!

As for live sports, the story isn’t good. The picture comes nowhere near pushing the edge of what the screen can do. Note that MLB.tv is 4K as opposed to cable’s 720p, but still. You get a close-up on a batter’s face and OK, it’s dramatic, but then the crowd shots and wide whole-field views are super-disappointing. I know that they can do better. I wonder what the bottleneck is, and suspect it’s just stingy management that’s unwilling to pay up to pump more bits through the wires.

I’ve been thinking about dumping cable and subscribing to one or two sports streamers (in Canada, Sportsnet and TSN). It might save money; if it got me a better picture it’d be a no-brainer. But I suspect the problem isn’t with the streamers, it’s at the source, with the leagues. Anyhow, interesting territory.

Software

These days you have to worry about your TV’s operating system. The LGs come wth “WebOS”; when I saw that name I thought “Didn’t that used to be the nice Palm thing that was killed by iOS and Android?” It turns out that this is that; a distant successor, anyhow, that’s weaved back and forth between owners and in and out of Open-Source respectability. See WebOS on Wikipedia for details. Anyhow, it’s cool that the TV runs Linux.

And it’s pleasant enough to interact with. But mostly we don’t, because the little Roku box that drove the previous TV still works fine, and it spits out 4K and has Netflix and Prime and (unlike WebOS) MLB and plenty of other nice stuff. It turns out Roku is Linux too, so there.

I generally like Roku, it seems to pretty well Just Work and get out of the way. But I suppose they’ll turn out to be evil, just like every other big player in the entertainment ecosystem.

Privacy

Modern TVs spy on you. They are part of the global adTech ecosystem, a dismal, dark, diseased, and dysfunctional landscape that, generally speaking, contains nothing good.

“I’ll get a dumb TV,” you exclaim, “Then they can’t track me!” Well no, but your cable box still is, and if you have a Roku or a Chromecast or really any other widget that routes entertainment bytes from the Internet to your eyeballs, it’s probably tracking the hell out of you.

My neighbor has erected a small but exotic-looking “digital antenna” on his roof and tells me he gets lots of channels in rock-solid first-class high-resolution high def. And yep, nobody’s tracking him. But, no internet goodies for him either.

The situation isn’t hopeless. In my case, I care about the TV itself, the Roku, and the cable box. A bit of Web search reveals, for anything reasonably modern (in my case, the Roku & LG), how to minimize tracking. I’ve done that of course, but in my heart I think they’re probably lying liars who are watching my unhealthy affection for big soccer tournaments and anything with good space battles or that has Idris Elba.

And bear in mind that your mobile phone is tracking you all the time too, as are bushels and bushels of JavaScript embedded in pretty well every Web page you visit. So I’m going to suggest that the TV may not be the most intrusive internet-connected device decorating your lifestyle.

But I still hate being watched in the TV cave, and think someone should pass draconian legislation to end this travesty.

Imperfections

We are happy customers of Logitech Harmony remotes, which are now being discontinued because Logitech is evil and hates customers. I joyfully discovered that the C1 TVs are in the Harmony database; maybe one of the last things to be added? So I reconfigured our remote’s setup but now it won’t sync with any of our computers. There are a bunch of workarounds and hacks we haven’t tried yet, and if all else fails you can buy a new Harmony on eBay, still pretty cheap.

My heart sinks at the prospect of operating a system with a Roku and a cable box and disk player and a PlayStation and a Chromecast, all plugged into an A/V receiver, without some sort of universal remote. Wish us luck. And it seems totally batshit crazy that there isn’t a good business to be built around solving this problem.

One other thing. We watch Netflix & Prime via the Roku, but now the TV has them too. Maybe the picture is better that way? But I haven’t figured out how to make the Marantz AV receiver route the HDMI Audio Return Channel (ARC) from the TV to the speakers. I’ll probably wrestle it to the ground eventually.

Hardware elevation and VESA joy

There’s a problem with the LG TVs: They ship with this ugly low-riding plasticky-silver stand that positions the screen just barely above whatever surface it’s sitting on. Which raises the question: Where do the speakers go? Assuming that you don’t want to use the shitty ones built into the TV.

Whether you’ve got a (*sigh*) sound-bar or (as in our case) a pair of very decent little PSB Alpha speakers with an outboard subwoofer, these are things that want to sit under the TV. But with that base they can’t.

I’m here to help. I spent an absurd amount of time searching for “tv stands” and “monitor risers” and other permutations, thus routing money from Amazon to Google because all the unsatisfactory answers had Amazon at the top of the list, and that doesn’t come for free.

It took forever, but I hit pay dirt, in the form of the STAND-TV00Y (great product name there) from an outfit called VIVO. Based on this product, I will definitely have a close look at VIVO next time I need to configure a desk/monitor combo.

VIVO Universal Tabletop TV Stand for 22 to 65 inch LCD Flat Screens

Our TV, like this one, is backed by a brick wall. I couldn’t resist this picture from the VIVO website, even though I’m deeply concerned about what’s in that ominous wooden ladle. I’m pretty sure the Worried Jungle People shouldn’t let the Stoned Jungle Person drink it.

Pardon me for going all fanboy, but this is brilliant. It comes with (and I sob that this should be such a rare thing) crystal-clear unambiguous directions for putting it together that Just Work.

I mentioned VESA (a.k.a. FDMI), which is the standard that describes how to fasten TVs to stands and booms and walls and so on. The VIVO uses that and my first experience with it is good. Except for, the stand comes with a little plastic multipouch containing a remarkable number of different-sized fastening bolts, because I guess VESA didn’t standardize that.

Should you upgrade?

Most people with TVs that work OK probably shouldn’t. But ten years’ progress does make a difference.

How Much Range? 1 Sep 2021, 7:00 pm

When someone wants to talk to me about my car, invariably the first question is some variation on “How far can you go on a charge?” The next is “How long does it take to charge?” Ladies, gentlemen, and other flavors, please take note. These questions are wrong. I’m here today to explain why, and suggest what the right ones are.

[This piece provoked by my recent Trans-Canada driving experiment.]

“How far can you go on a charge?”

For almost everyone, 95% of their driving is commuting and shopping and going to the gym or whatever. Every contemporary electric car you can buy has more than enough range. Most EV drivers I know charge less than once a week.

Therefore, the question is really only relevant if you need to drive long-haul. I’m going to define “long haul” as “more than 250km” (about 150 miles). That number 250 may be controversial but I think it’s reasonable, because as of mid-2021, it’s becoming easy to buy an EV with that kind of range, with the price creeping further into mass affordability every quarter.

Now, when you’re long-hauling, you’re never going to use all of your range. To start with, when you’re using one of the fast-chargers on the highway, the process slows down when your battery hits 80% full, by a factor of as much as three. So if you arrived at 20% full, it’d take you the same time to get from 20% to 80% as from 80% to 100%. Since you want to get back on the road, and you don’t want to hog the charger unduly, you usually take off when you hit 80%. So to answer the long-haul range query, start by subtracting 20%.

Not only do long-haulers not start out full, they don’t run the battery down to zero. These days, there’s always the danger that when you get to the charger, it’s broken or busy or you just can’t find it. So you need to leave some reserve. People who plan ahead generally look for a charger where there’s another nearby to serve as a Plan B. What do we mean by “nearby”? Well, if there’s a Plan B charger a couple of blocks from your target, you’ll be willing to run pretty far down. If the chargers are say 50km apart, you’re going to want more reserve.

So the correct arithmetic isn’t “max-range - 20%”, it’s “(max-range - 20%) - Plan-B-safety-margin”.

Of course, there’s a special case when you’re starting from home, or ending up there. Where by “home” I mean somewhere that there’s a reliable low-tech level-2 charger where you leave your car plugged in all night and it’s back at 100% in the morning. So when you’re starting from home you don’t need to compensate for that 20%, and when you’re ending up at home you don’t need the safety margin.

But it’s more complicated than that. Because your range depends on how fast you’re going, how often you’re stopping, whether you’re going up and down hills, how hot or cold it is, and how hard it’s raining. For example, the worst-case scenario I can think of is the eastbound BC Highway 5 (“The Coquihalla”) which is 500+ km long, mostly uphill, and has approximately 0km of flat sections. Also, it has a speed limit of 120km/h. Also, it’s in Canada, which means that the local climate includes rain, snow, and extreme temperatures.

Among all these variables, there’s one you can partly control: your speed. I’ve been told that the formula for air resistance includes at least one quantity containing the square of your speed. So when a long-hauler is calculating the next leg of their journey, they’ll need to take that into account.

So the correct question is something like: On the rare occasions I’m driving cross-country, how far can I go in one hop, after you take off 20% for charging efficiency (unless you’re starting at home), allow for Plan B at the destination (unless you’re ending at home), and compensate for speed, weather, temperature, and hills?”

So someone who asks me that question is apt to get a long answer. Or in the (unlikely) event that I don’t want to explain, or the (common) event that I don’t think they have the patience, I say “Max 400km best-case, but I can always get 300.”

The right question

I suggest “Can you go 250km between chargers on a cross-country trip?” I confess that I’m influenced by the design of Petro-Canada’s Electric Highway project, which aims to have chargers no more than 250km apart. I think that’s about right.

Depending on how the ecosystem of EVs grows, we might end up using either a larger or smaller number. Of course, the more charging networks are out there, the easier Plan B gets, so the minimum viable long-haul-leg range gets smaller.

“How long does it take to recharge?”

If you own an EV, your life will be much easier if you have reliable access to a “Level 2” charger. This can cost less than a thousand bucks if you’re lucky enough to have a garage that already has decent electrical service. But it’ll be more for most people. For those who park on the street or in their apartment’s basement, it can be a real problem.

With that Level 2, then for basically every electrical car on the market, if you adopt a discipline of “Plug it in overnight whenever it gets down below half charged”, you’ll never have to think about it.

So once again, this only matters when you’re long-hauling. But then it matters a lot, because it’ll have a major influence on how fast you get there.

Once again, the answer is complicated. This time I’ll cook the factors down into a list:

  1. How far do you have to go? If the next leg is much less than your range (after all the corrections and adjustments listed above) then just charge up that much, plus enough for Plan B.

  2. How fast can your car charge? Some of the older and cheaper electrics can barely soak up 50kW. Mainstream high-quality cars these days can use 100kW (up until 80% full, that is). The Porsche Taycan and Hyundai Ioniq 5, however, can both use more than 200kW and this is what I’d expect from the whole next generation of electrics.

  3. How fast can the charger pump electrons? In my recent Trans-Canada trip, I encountered “fast” chargers at 50, 100, 200, and 350kW.

The take-away

In areas of the world with a decent charging network, pretty well any reasonably recent EV will long-haul. Probably the most important quality-of-life factor is your charging speed.

The areas of the world without a decent network are shrinking and will shrink lots more, quickly.

CL XLI: Forest Stories 24 Aug 2021, 7:00 pm

Recently I’ve had the joy and privilege of time spent walking in the Pacific Northwest forest, on a small island where we engage in Cottage Life.. Walking in the forest provides a fine opportunity to think, although the raw beauty of the forest pouring in through your eyes and ears will regularly interrupt. While forest-walking, I thought about pictures, modern mapping technology, strangers’ identities, and The Green Knight movie.

Forest on Keats Island

Snapping

I have a problem: It’s hard to photograph the forest. Out of sheer embarrassment I won’t share the number of times a combination of light and space and color has brought the camera to my eye. Because almost every such effort, on later consideration, ends up looking like a snapshot of some trees. I occasionally get the light and color but the special space eludes almost always.

Challenge accepted, OK? If that rainforest thinks it can hide its beauty from my camera, it’s got another think coming. With any luck I should have a couple more decades of life to work on the problem.

Forest on Keats Island

Mapping

There’s a problem walking in these woods. The trail network is a bit complicated and generally speaking, the trail forks look like the other trail forks. This makes it hard to re-create an excellent walk with a length known in advance, for example when you’re showing off the island to a first-time visitor who might not be up for a challenging two-hour scramble.

So I decided to map them. I surveyed the (many!) Android apps designed for this purpose. It seems that AllTrails is the most popular, but I found its learning curve onerous. So I installed Gaia GPS and Lauren installed Wikiloc, and we set out. They both worked pretty well. I think that if you’re signed into Gaia, this map should show my recently-marked trails. But I’m not sure I actually understand the publishing process yet.

Having created a Gaia GPS account and used the app/site briefly, I was charmed to get an email from them advertising that they were hiring and anyone interested in some combination of cartography, mobile apps, and server-side tech should get in touch. If I were younger I might.

Forest on Keats Island

Green Knight

I have a special relationship with the poem behind the recent movie.

The movie was our first such outing since Covid started. We even took a public-transit train to get there. I masked on the train but, since the film’s been out for a while and isn’t a big hit, there were only six people in the theater, widely separated, so I went bare-faced. It was frankly a thrill to go out and do adult things.

As for the movie, meh. The middle section, with Gawain wandering the wilderness seeking the Green Chapel, was very good. But I thought the ending, completely different from the one in the poem, was not an improvement.

And while the location shooting was very beautiful, the sound design was awful, with obtrusive heavy-handed Foley; for example, Gawain‘s horse plods slowly down a muddy forest path, and with each pace a huge “thud!” explodes from the theater speakers.

I think the problem is that the movie didn’t take itself seriously enough, as witness the hokey episode titles and the really dorky final line of dialogue assigned to the Green Knight.

I hope someone tries again and does it better, because the underlying poem is a fine piece of work.

Forest on Keats Island

Identity

We were walking one of those trails and my eye was captured by a flash of rectangular white in the undergrowth. It turned out to be a BC Services Card, which combines the functions of driver’s license and healthcare access. I’d sure be upset if I dropped mine on a forest trail — I’ve never had to replace one but I imagine the bureaucratic snarl is pretty awful.

Fortunately, the card displays, along with the holder’s full, name, gender, and birth-date, their mailing address. So it was easy enough to put it in an envelope and drop it in a post-box.

But I was unsatisfied, because if it was my card I’d want to know right away that it’d been found. So I went to look up the holder, a woman who had a dirt-common surname but moderately unusual first and middle names; I thought given an email address or social-media handle, I could set her mind at rest.

Google: No luck. Facebook: No luck. LinkedIn: No luck. The phone company’s “white pages” site (if you don’t know what white pages are, that’s perfectly OK): Yes, correct first-name/last-name combo in the right suburb. I called it and got a fax machine. Uh…

Anyhow, she got the card and I got an online thank-you via LinkedIn. But, first of all, I was surprised that with this much information, I was still unable to find any online evidence of this person’s existence. Weird, right?

No, maybe I’m weird. Given a random slice of a thousand or so people across the population, how many of them should one expect to be able to turn up online? How far has the Internet penetrated, really, into the fabric of society?

I don’t know, but I’d like to. I’m the last person to ask because I live online and the space of people who don’t is pretty well closed to me.

Thanks for listening

And if there’s a forest anywhere near you, count yourself fortunate and go take a walk in it. You won’t regret it.

Page processed in 2.489 seconds.

Powered by SimplePie 1.3.1, Build 20121030095402. Run the SimplePie Compatibility Test. SimplePie is © 2004–2022, Ryan Parman and Geoffrey Sneddon, and licensed under the BSD License.