OpenBSD IPv6 on TinyKVM

I had a lot of trouble getting IPv6 to work under OpenBSD 6.7 on a TinyKVM instance. So here, for reference, is how I finally made it work:

1. In your control panel, note the IP addresses for your instance and its gateways; for the IPv6 address of your instance, you get to pick the last 16 bits yourself, but obviously you can't pick the IPv6 of your gateway.
2. Edit /etc/hostname.em0 and put in the IPv4 (keyword inet) and IPv6 (keyword inet6) addresses (and netmasks) for your instance, nothing else.
3. Edit /etc/mygate and put in the IPv4 and IPv6 addresses for your gateways, nothing else.
4. Stop and disable the slaacd daemon using rcctl; I guess the OpenBSD folks are hopeful that most networks support SLAAC but I had no luck here.
   
5. Try restarting the network using sh /etc/netstart but if that doesn't fix things for you, reboot the instance and hope for the best.

Note that I had to add a "mystical" invocation of ifconfig to /etc/rc.local as well. Without that, the instance wouldn't pick up either of its IPs immediately on boot but only after a roughly 5 minute delay. No idea what the TinyKVM folks are doing there, but hey, with this "all static" configuration and the weird ifconfig trick things now work roughly as they should. If you figure out a better way of doing this, please let me know!

Hosting Options

For various reasons I've been looking around for "new" VPS hosting options again. This is a brief summary of the "neat" things I found. It's not a "review" of any kind, it's literally just a bunch of links and commentary. (If I ever use any of these options for long enough to have an opinion, I'll add a review or three.)

Historically I've mostly used Linode, especially "back in the day" when I was running lots of game servers across the US (and even in the UK for a while); my longest-running (shared) server is also on Linode. They have some great tech, nice support, and it's generally very smooth sailing indeed.

But there are "issues" as well, especially these days. First they don't have a "cheap" storage option. Everything is NVME SSDs these days and so with a small plan (what I usually run because I am cheap) you get 25 GB. Not too shabby if you grew up in the 1970s, but not great either.

After a bit of research, I noticed that RamNode actually offers plans with good-old "spinning rust" disks. Yes, the kind some of you don't even remember. As of today, you can get 160 GB for $3/month and 325 GB for $5/month. Pretty sweet if all you need is a bunch of space, for automated backups for example. Of course you need to keep an eye on bandwidth, but hey, here's the storage if you need it.

Another "issue" is that US companies are, well, US companies. Just like German companies are German and Australian companies are Australian. Doh! What I am getting at is that by picking a hosting company, you're also picking (to a degree) the laws that protect (or not!) your data, privacy, freedom, or life. So depending on what you plan on doing with your VPS, you may want to pick an "exotic" jurisdiction instead of the obvious one.

I ended up looking at Switzerland, Austria, and Iceland in more detail. From a "Five (Or More) Eyes" perspective, these are places that at least have no "overt" history of large-scale "illegal" spying. They also have "reasonable" Press Freedom Rankings which can be helpful if you plan to do some investigative reporting online. But don't forget the "small print" of your VPS provider either, different companies have vastly different "Terms of Service" regardless of the level of "freedom" that exists in their respective countries. Also these things are never as simple as shoving your data or services elsewhere, so if in doubt get yourself a lawyer!

For me, 1984 Hosting in Iceland checks most of the boxes I care about. Also their pricing is a lot better than many other VPS vendors in Iceland. For example they have a reasonable $5/month plan that's actually comparable (except for bandwidth) with a cheap Linode plan. On top of that I was impressed by their recent report on a 2017 incident that brought down much of their operation. Yes, maybe it's a little late, but it's certainly detailed and I very much like the idea of them trying to be this transparent about what happened, why it happened, and what they've done to avoid it in the future.

A final "issue" is operating systems: It's kind of a pain (or at least it used to be, maybe it improved recently?) to install OpenBSD on Linode. Yes, I've been mostly a Linux person for the past 15 years or so, but I've recently "tuned back into" the BSD world a bit more and I like what's happened there. I currently run OpenBSD on a small Vultr instance and that works reasonably well. But I also found a small outfit in Amsterdam that not only offers OpenBSD VMs but also uses OpenBSD to run the virtualization infrastructure itself! On top of that, they donate to the OpenBSD developers. Their pricing is yearly and works out to $5/month for a small VM (half the RAM of Linode, but twice the disk space).

That's it for me and hosting for now. Feel free to share any hosting tips you have in the comments, I am "all ears" as it were.

Cobalt Qube 2 / RaQ 2 Notes

I've had an old Cobalt Qube 2 in my possession since at least 2003 and I got my hands on an old Cobalt RaQ 2 sometime in 2007. These are cute ("qute"?) MIPS machines that (used to) work great as targets for my Compilers and Interpreters course. Sadly both boxes have been "out of commission" for a few years, but I recently decided that I want to run them again, the RaQ in particular. In the process I noticed that a few important pieces of Cobalt information are hard to find online these days, so this post is mostly a collection of notes for myself.

First here are two basic shots of the RaQ 2 in pretty much the state I received it in over 10 years ago. First note the lack of a cover for the power supply! You'll need to be somewhat careful working in here if you're worried about a capacitor zapping you.

Cobalt RaQ 2 Front

There is also a spot for a fan but apparently the power supply never gets hot enough because (as you can see in the next shot) that grate is actually covered by the Cobalt logo.

Cobalt RaQ 2 Back

The installed fan is incredibly tiny and in my RaQ 2 it doesn't sound particularly healthy anymore. I have not double-checked yet, but supposedly Cobalt used this SUNON fan for the RaQ 2. Sourcing an exact replacement fan has turned out to be difficult, but I've also found some ancient posts that claim that these machines don't really get hot enough to require a fan in the first place. I am leaving mine in for now, hoping that even if it fails it won't bring down the box with it.

Next topic: Failed battery. There's a CR2032 located on the main board next to the power supply. Sadly the thing has one of those dreaded sockets with an excessively tight retaining clip on top. You're supposed to "lift the clip" and then slide out the battery, but of course in the process of doing that the entire thing just "ripped off" the mainboard. To add insult to injury, a solder pad came off as well, complete with the attached copper track of course.

Socket and copper track, yikes!

Witness the complete destruction on the mainboard:

Battery connector destroyed.

The lesson here is either that I am incapable of treating a piece of history with the required tenderness or that one should just leave good enough alone. Good enough? Well it turns out that the machine works perfectly fine with an empty battery. Better yet, it works perfectly fine even with a ripped out battery. So luckily I did not actually destroy the entire machine with this horrible mistake. If I ever decide to give my Qube 2 the "same" treatment, I'll desolder the entire battery socket instead and replace it with a less insane one.

To be continued... :-)

Unlock LUKS Remotely on Ubuntu 16.04 LTS

I reinstalled my little server at work. It had been running the same Gentoo setup since 2008 (!) but in December 2017 the Gentoo folks finally managed to produce a convoluted enough update to completely hose the darn thing. Well, not "hose" as in "disaster" of course: I got all the data off safely and I could even leave it running over the holidays while I was away in Germany. But in preparation for the Spring 2018 semester it was necessary to "start from scratch" as it were.

Upset enough with Gentoo I decided to go completely the other way and do a minimal Ubuntu 16.04 LTS install. I also used this opportunity to finally add full-disk LUKS encryption to my setup. Not that it makes much sense (the server runs 24/7 so it's basically always decrypted) but what the heck, this is how you're supposed to set up your server, right? And instead of going for ZFS or btrfs (which is what I used on my new home machine), I went "retro" one last time:

1. Partition the disks into one "small" boot partition and one "large" data partition.
2. Use mdadm to create a RAID-1 boot and a RAID-10 data array.
3. Use cryptsetup to LUKS-encrypt the "huge" data RAID array.
4. Use LVM to create swap, root, and various home volumes in the encrypted RAID.

Great! But now I had a problem. You see, every now and then we actually have a power outage. (I know it's weird, but since we're not the hospital we apparently don't get to have a redundant power grid.) I might not be around to reboot the machine manually, but students want to submit their homework assignments. And I don't like giving extensions. So what to do?

Enter the recent dropbear-initramfs package! This thing is supposed to integrate a tiny dropbear SSH server into the early root filesystem, enabling "remote unlocking" of the encrypted RAID array. And it almost works. First the easy parts:

1. apt install dropbear-initramfs (and ignore the error message)
2. mkdir -p /etc/initramfs-tools/root/.ssh/
3. Put your SSH keys into /etc/initramfs-tools/root/.ssh/authorized_keys
4. update-initramfs -u

Now when the machine boots up, it'll still ask for a passphrase on the console, but it will also start the dropbear SSH server. In theory you can ssh in as root at that point and run the unlock script, but that didn't work for me. What did work, however, is to echo the passphrase directly to the FIFO created by cryptsetup:

1. ssh root@server
2. echo -n passphrase >/lib/cryptsetup/passfifo
3. exit

At this point the encrypted RAID will come up and the boot process will continue, just as if you had typed the passphrase at the console. I am positive that they'll eventually fix the scripts, but for now I am just happy that I was able to make it work.

Note that it's convenient to run dropbear SSH and the regular OpenSSH on different ports to avoid getting loads of useless warnings when connecting. It's also just good sane practice if you want to keep the number of script kiddies trying to get into your server down to a minimum. Yes, security through obscurity is bad, but it's only really bad if it's your only approach. Happy remote booting!

Buying an Old Thinkpad

I've recently been bitten by the theoretical Thinkpad bug. That is, I can see the attraction of these machines but I have no practical experience with them, especially not with the weird red dot on their keyboards. I also hate my current Lenovo laptop with a passion, so spending a lot of money without knowing if I'll like the machine seemed ill-advised indeed.

The good news is that the entire Thinkpad line seems to thrive on a myth of "eternal stability" meaning they simply don't change much (on the outside anyway) from year to year. So I figured I should just buy an older model for less money and see how it handles. If I end up liking it, I'll eventually shell out the dough for a new one.

I am obviously not the first person with that idea. Here's a reddit page that explains why older Thinkpads are a great way to go and there's even a buyer's guide specifically targetting older machines. So this post is less about pitching the idea of older Thinkpads and more about my own deliberations on which old model to get.

I decided to play the role of a "serious" user so the only Thinkpads I even considered were the T series and the X series. The strange sliver of my personality that made me an Apple fanboy back in the 1998-2004 time frame is immensely attracted to the X series. Thin, light, compact, sexy, wow! However, the newly discovered grown-up in me tends to prioritize things differently. I need a machine that I can work on comfortably for hours at a time which means I need a big display, good hand rests, a spacious keyboard, etc. If I really fall in love with Thinkpads during this experiment, no doubt I'll eventually own an X series product as well. But for now I'll stick to the T series.

Within the modern T series we have the bigger T5xx models (15.6" display, numeric pad) as well as the smaller T4xx models (14" display, no numeric pad). This is sort of a toss-up for me: A larger display is nice, but a numeric pad is completely useless. In terms of weight we're looking at 4.5 to 6 pounds for T5xx machines versus 3 to 5 pounds for T4xx machines (across the 2008-2017 time frame that is).

Just looking at the numbers above I couldn't really make up my mind until I remembered that my current Lenovo laptop is in the T5xx class when it comes to size and weight. And I have to admit that something a little lighter would be a lot nicer to carry around. So the entire T4xx line qualifies, but only the relatively recent T550, T560, or T570 models are light enough in the T5xx line. The crappy current laptop also helped me determine my "historical" cut-off point: I have a 2012 laptop now and I don't want to go earlier than that in terms of technology.

So now I am left with the T430-T470 as well as the T550-T570 in terms of my various criteria. But wait, the whole point was to buy an older, cheaper Thinkpad! That narrows the field further to just three realistic categories:

• 2012: T430, T430s, T430u
• 2013: T431s, T440, T440s, T440p
• 2015: T450, T450s, T550

So I started looking into all these models and the first thing I noticed was that the T431s and all the T440 models have a very different trackpad setup from the rest (and indeed from all Thinkpads before and since). That takes all of these machines out of the race as well, after all I want to have a "true Thinkpad experience" and not some strange aberration that only existed for a brief instant.

For the remaining models, I decided to read all the in-depth reviews I could find. But after doing that for a while, I realized that there's really only one review site worth reading, so here are the links to all those reviews, followed by my own comments:

• T430: The i7 tends to run hot at 55C. Granted, only under load, but I don't like the idea of a runaway computation setting my couch on fire. So it's out.
• T430s: The i5/620M tends to run warm at 43C, the i7/4000 tends to run warm at 46C, neither of which is terrible. Despite being smaller and lighter than the T430 this thing still has an optical drive bay, something I like. Depending on condition and features, these ran between $170 (crappy) and $400 (like new).
• T430u: The i5 tends to run warm at 45C. It's the only older model with HDMI, a big plus in retrospect. However, it's also the one model with only a fixed battery and that's a big minus. But worst of all the screen is terrible, so it's out.
• T450: The i5 tends to run warm at 47C and I'd be okay with that. What I wasn't okay with is that these started at $520 which I deemed too expensive.
• T450s: The i7/5500 is a dream and runs cool at 36C even under load! The i5/940M on the other hand runs hot at 52C which is too much for me. Alas it didn't matter, also too expensive at $600+.
• T550: The i7 tends to run cool at 39C. Sadly most of these were also very expensive ($750+), the only affordable one I could find (at $379) had a locked BIOS.

I am obviously a little obsessed with temperatures. Everybody has their thing and I've had plenty of "laptop too hot for my lap" experiences that I don't care to repeat.

You can probably guess that I finally settled on the T430s for my experiment. The one I ordered was a little more expensive than I had hoped for ($265 instead of $250) but that's close enough. What convinced me in the end was that the seller actually wrote a long reply, covering all the details that were important to me.

Once it gets in, I'll try to do another post (maybe with pictures) about the machine and then it's on to actually testing it by replacing my current laptop for a week or three of lectures. Oh, and I need to order an HDMI adapter because we mostly have HDMI at Johns Hopkins these days.

Playing id Classics on Linux

Just for reference since I've been going through a somewhat extreme retro-gaming phase lately. Of course you'll need the actual game data for all of these. Luckily I have old CD-ROMs that still work fine (even though I of course have backups as well). The details for extracting the data vary, maybe that's a topic for a future post. But if you can extract it, here's what to run the games with:

• DOOM, DOOM 2, ...: For the best "retro-feeling" I recommend Chocolate Doom. If your distro doesn't package it, just grab the source and build it yourself, it's not hard. Youngsters may not like it because it's designed to give you an "accurate" 1993 experience (pixels!), but that's how it's meant to be played so deal with it.
• Quake, ...: The only thing I can recommend is the DarkPlaces engine. Don't get me wrong, it's an awesome effort and it does indeed look a lot better than the original. But my vintage 2009 graphics card actually has trouble keeping up which is a little sad for what is ostensibly a 1996 game. Alas, every other source port I tried didn't actually work, so... Also surprisingly easy to build, just grab the source and hit make. Just be prepared to wait a little longer than for the Chocolate Doom build.
• Quake 2, ...: Even fewer choices for Quake 2, but luckily Yamagi Quake 2 is excellent and they didn't overdo it with the graphical polish. I can play this at full speed with all the options turned up and it looks more or less like it should. Once again a pretty easy build, just grab the source and boom there it is. One minor quibble: They don't have a command line flag to configure the base path, so if you don't want to throw the executable into your data partition you'll have to change two lines in the Makefile.

There you have it, the stuff I've been playing with over the last two weeks or so. Good clean retro fragging, who can resist? I may update this post with more details if there's interest, so leave a comment if you prefer another engine that I missed or if you can't get the above to work for you.

Protecting Weaker Party Members

I am currently running a B/X game that started with the sentence "We've all decided to play magic-users!" from one of the players. Luckily my adopted variant of "3d6 in order" prevented that outcome and they're a pretty typical "murderhobo" party now: two magic-users, two fighters, one thief, and one blood-thirsty halfling with suspenders but no shirt. (There was a cleric, but that's another story.)

In any case, one of the players was "completely new" and of course I recommended that she play a fighter. I sold this mostly with "it's the easiest class to run" but also with "all those robes-and-silly-hats people will need protection" which is what me got thinking:

There is no "rule" in B/X that would allow a fighter to protect a magic-user or anyone else for that matter.

Seemed like something important to offer at the time, but looking around the many B/X house rule documents out there I couldn't find the problem addressed by anyone. Even Delta who has probably forgotten more D&D than I will ever know left me out in the cold on this topic (aside from mentioning the need to have lots of fighters to protect magic-users in mass battles).

I wanted a simple mechanic that would make the protected character "safer" in some way (benefit) while also "exposing" the protecting fighter more (cost). I quickly settled on two "ground rules" for this:

• The fighter must be "close" (within 5' say) to the character they protect.
• The fighter must be "aware" of an attack in order to protect against it.

I guess you could say that I am treating the fighter as an "intelligent shield" of sorts? That approach seems fair to me because I don't want a fighter to simply throw themselves over the character they are trying to protect. I very much want "protector" and "protectee" to still be "in combat" instead of changing the focus to a Secret-Service-style "Get him out of here!" and nothing else.

The simplest thing I could come up with was to simply let the fighter take the damage. So as a rule, I'd phrase it something like this:

Fighters can choose to protect one character within 5' of them. If the protected character gets hit by an attack the fighter is aware of, the fighter can choose to take the damage instead.

Not too shabby in terms of simplicity, is it? But as always the "devil" is in the details. First of all this conveys almost complete immunity to the protected character. But the goal was to make them "safer" only, not "safe". In other words, I still want the magic-user to worry about maybe having their spell interrupted. Also we have reduced the fighter to an actual meat shield ("bag of hit points") and nothing else: Regardless of how good their AC is, they now get skewered because of the (presumably) much worse magic-user AC. That's not fun at all for whoever is playing the fighter. Finally, if we have multiple fighters protecting one character, the "immunity" could actually be complete and it could go on for a long time indeed. This may work great for Delta's mass combat scenario, but on the "party in a dungeon" scale I wouldn't want to deal with it.


So instead of letting the fighter take all the damage, let's try to do something about getting hit in the first place. Here's what I came up with next:

Fighters can choose to protect one character within 5' of them. Attacks against the protected character that the fighter is aware of suffer a -8 penalty. Attacks against the fighter, however, gain a +2 bonus.

The numbers are subject to debate of course, but this seemed fair to me. Presumably the kobold archers trying to hit the magic-user would notice that the fighter brushed away all their arrows, so next round they'd redirect their fire. But as so often when you design a rule with fixed penalties/bonuses, there's an exploit: Two or more fighters protecting each other would impose a -6 penalty against most attacks, clearly not the application that was intended. Except for that terrible flaw, however, we would get the desired effects: The protected character still has to worry about getting hit, and the fighter now makes for an easier target because they are "out there" trying to distract attackers.

What finally made me think of a better solution is this: With the fixed penalty, a naked fighter could protect a character in plate and shield. That makes no sense! What should happen instead is something like this:

Fighters can choose to protect one character within 5' of them. Attacks against the protected character that the fighter is aware of must hit the fighter's armor class instead (provided that armor class is better). Any damage is still suffered by the protected character. Attacks against the fighter also gain a +2 bonus.

I am reasonably happy with this rule. It's a little more complex, but not overly so. It does allow protecting weaker party members if they have a worse armor class, but it doesn't make them (almost) completely safe. It also has no obvious exploit that I can think of, mostly by virtue of letting the better armor class prevail. I'd also say that a spell like Protection from Normal Missiles cannot be used with this rule as it doesn't affect armor class, but your mileage may vary.

I've added the last version as a house rule to said B/X game, but after two sessions so far it has not seen any use yet. Admittedly the characters were not really in situations where it made much sense to try yet, but here's to hoping someone will attempt it sometime soon. In the meantime, I'd love to hear what everybody out there thinks of what I did here.

BTW, I am aware of the 5th edition Protection Fighting Style thing but it's just too horribly complicated and too far from B/X to try to convert. It's interesting however that the 5e folks also thought that the "big fighter protecting small magic-user" trope needed "a bit more rule" behind it.