You can't possibly enjoy that

Why would anybody actually enjoy maintenance programming?

It's a long story. Bear with me.

I grew up in the 70's and 80's when all my heroes worked at Bell Labs, but they were just these weird guys who did computer stuff that nobody paid attention to at the time. They wrote code because they needed to do shit, not because it was sexy.

I cut my teeth writing BASIC on a VIC-20. Not very sophisticated, but I felt I was doing more or less the same thing those guys were doing: I wanted the computer to do something it didn't do out of the box, so I taught myself how.

In the beginning, I was a terrible, unsophisticated, uneducated programmer, and I spent a lot of time rewriting my own code.

So what?

I got used to the notion that if I didn't get everything just right, the program would crash if you looked at it wrong. Refactoring wasn't a side quest, it was what I did after writing a long function to figure something out and then needed to break it down into manageable parts.

Fast-forward twenty years, and I decided this was something I'd like to do for a living, so I got the education I so desperately needed. I learned about data structures and software engineering and project management.

But I never forgot those early years rewriting my own code over and over, making it better, making it right. It was satisfying to me, and it still is.

I've spent a good deal of my career either porting or maintaining other people's code, mostly because nobody else wants to do it. They want to start something new, work on a feature, do the sexy things. That's cool, I get that, but that code might hang around for years, and it's unlikely to be perfect or completely future-proof.

Then it's time to call in the maintenance programmer. I've got you, dude.

Software Maintenance

Evidently Blogger is still a thing, and I feel like writing again, so here we are.

Once again I find myself in the position of maintaining a large body of somebody else's code. A lot of people find this task distasteful, but I rather enjoy it.

I've decided to write a series about this pursuit: why I enjoy it, why others do not, and how those who write "original code" might help those of us who have to maintain it.

See you soon.

Campeonas

Just watched the WWC final. If you weren't following this year, you should have. Top notch football all the way through. For a change it didn't come down to penalties, and everybody will agree that the score line matched the run of play.

The final was bittersweet for me this year: I'm overjoyed to have seen Lori's beloved Roja Furiosa win, but it's so very painful that she wasn't here to see it with me.

Somos los campeones, mi luchita.

Indie Game Dev: Automation Servers

Modern programming projects -- commercial ones, anyway -- should use build automation (aka CI/CD). Otherwise there is potential for nasty surprises come QA/release time:

• The person who normally builds a given piece of the final package is away that day and their workstation is locked
• Licenses for certain tools have to be given to everybody in the company who might need to build some artifact, just to avoid the previous problem
• Licenses for certain tools have expired, because nobody's run it in a while and you didn't notice the date had passed
• Which file is the right build? Is it qa-test-build.apk, or release-candidate-June.apk, or 2021-06-20.apk, or one of the myriad copies of each of those in the folder "recent builds", "test builds", "rc builds"? Oh, or did Frank do it this this week?

The right automation software alleviates all those problems and more. It doesn't need a person present in order to complete the correct task, it doesn't forget and it doesn't make mistakes because it's bored or distracted or pressed for time.

If you can only afford one license for a particular build tool, put that license on a single "agent" machine, and everybody can get the finished product from there.

If builds run regularly (you run them regularly, don't you?) then an expired license will make itself known the day it happens, hopefully before the eleventh hour when you're trying to submit a build to the platform.

Build artifacts are organized in a systematic way, so as long as everybody's communicating (Hey Aaron, build #314159 is the release candidate, please test that one), then everybody knows which build is the right build.

Before we get to what I chose, let's talk about how I chose it. I don't have very many deal breakers, other than the ones I've mentioned in the past. Actually, I guess I have two.

First, complicated and/or delicate installation. If the system needs <specific OS version x.y.z.123.b-2>, or <equally specific language runtime>, or has to be installed on a Tuesday evening during a waxing gibbous moon with a chicken in your left hand and a knife in your right, I'm going to pass.

Second, hard to use interface, and especially, hard to find build results/artifacts. If it's hard to set up a job, or hard to find the result of that job and get the build to QA, I may as well build the thing at my desk and email it around to people like an executive would. HARD pass.

Now the must haves. Has to interface with my desired source control systems. Absolutely must understand Subversion and Perforce, and if it doesn't understand Git I probably won't look at it either. "But Ian, I thought you didn't like Git". I don't, but I have friends who provide libraries and some of them keep their source in a Git repo, so I'm better served by a system that understands it.

Has to have a master-worker arrangement. As a mobile guy, I need at least 2 platforms -- OSX for iOS, and Windows or Linux for Android -- just for the application binaries themselves. There might be realtime pvp servers (Linux) or GUI tools (probably Windows) and any number of other things. So I need workers on all of those systems controlled by a master.

Gotta have a web interface. Welcome to the 21st Century.

I've gotten used to being able to "mark a build"; while builds normally get culled after some number of days, or bumped out by the most recent, I should be able to lock one in as a Release Candidate. I should also be able to attach some description to it, so we'll know what that was 2 years from now (rather than just being able to rebuild from a tagged changelist).

I had 3 candiates:

Buildbot - I liked this one because everything is on PyPI so it's pretty much create a virtualenv and pip install to freedom. I had some minor issues at the beginning getting the TLS libraries installed, but I think that's more a function of my inexperience with Python than anything to do with the code.

The master and worker setup are pretty much the same, so once your master is running, you can then install workers wherever you like and point them at the master. It is nearly as easy to do as it is for me to say.

Since it's all Python, I have to assume any place you have a Python install you can have a Buildbot worker. And since it's all self-contained, you can run it manually from your home directory if that's the cut of your jib.

GoCD - a little more professional looking, a little more complicated and a little more fiddly. You can install it via your operating system's package manager, if such a thing exists. Windows has an installer .exe, it's all good.

It's Java, which is mildly upsetting to me, but the installers come with the correct version of it, so I can overlook that just so long as I don't have to upgrade it myself.

My winner was Jenkins, and I'm not gonna lie, it was mostly because I have experience with it. Buildbot seems cool, and if you have to teach yourself one of these systems, it looks like the simplest, i.e. the shortest path. GoCD seems a lot more feature-rich, though the UI seems a little less intuitive, and I find myself having to drill down through the various parts of a job in order to get to the artifacts.

Jenkins is also Java-based, but once again the installation is easy and low-maintenance. I was able to apt install the entire system, including Java, and it wasn't some weird alternate, it was the stock Java that comes with Ubuntu. Sorted.

I've been using Jenkins at work full time for maybe 3 years, and it's pretty hard to find fault with it. It does all the things I've mentioned and a lot more. The UI is pretty good: if you have a bazillion jobs on a single master, and you weren't aware of folders you caveman, you can set up views to organize it all nicely, and that's what we do at work. Some of our leads have created pretty elaborate "landing pages".

Equally as important is that it's very easy to point QA to a specific view if they're testing for you that week. I go as far as Slacking them links to an install page that our build script creates, which is crazy handy. Then they can navigate to said page on their device and click the install link and start breaking my build (and my heart 💔).

I haven't created any of the more complicated job types yet, but I've seen them in action, so I'll be learning how to set them up once the applicable use cases present themselves.

Indie Game Dev: Languages

I wanted to talk about CI/CD systems next, but I'm still evaluating candidates. Maybe next week.

I haven't left very much to the imagination when it comes to me and languages: I've already stated that I'm using Android, and that I view Java as a necessary evil to that end. I like C and Lua. I have plenty to say about plenty of languages -- I've learned somewhere between 30 and 40 of them, depending on what counts (XML? HTML?) and whether variants count (QBasic, Commodore Basic, Visual Basic).

Rather than bore you further with the odyssey of my language scholarship, I'll talk about the few I will be using in this project.

Java

I dislike Java, which sounds a little rich coming from a mobile guy, and believe me, I ain't ever happy about having to use it. That said, most of the Android stuff works with old JDKs (5+ years), and all the tools know how to acquire their own dependencies. So I do the initial JDK install, work through the PTSD, and put it out of my mind.

Don't misunderstand: I don't mind Java as a language, I mind it as an ecosystem. It was created on SPARC workstations running SunOS in the mid 90's, and it behaves as though that's still the only place it runs.

You need something in the ballpark of 10 or 11 environment variables before the compiler will look at your code. One of those is CLASSPATH, and woe be to you if all your Jar files aren't somewhere in there. Seriously, in 25+ years, nobody could update that compiler code to accept some sort of per-project configuration? Everything's defined at the OS level?

So if you're in the middle of a Java project and you buy a new workstation, you spend a few minutes installing Visual Studio, and then the rest of the day preparing your Java environment.

Speaking of defining things at the OS level, why does the compiler care where my files are? If I create a package called org.thenibble.dante and there's a class in it called Worker, then it has to exist in ./org/thenibble/dante/Worker.java. What kind of Elizabethan control-freak bullshit is that?

Python

I have some recent experience with Python, and I'm mostly ok with it, provided we're talking about Python 3. It's been more than a decade, migrate already. I like that Python's ubiquitous, and there's libraries for everything. There's lots of documentation and examples around, and Stack Overflow fills any gaps.

There's a bunch of reasonably good, relatively lightweight REST frameworks, and most of them will work with whatever web server you choose. I'm used to Apache + Flask, but my backend guys at work prefer NginX + flup.

I don't like Python's scoping rules. I don't like that people feel compelled to use classes for everything when it's rarely necessary (though it's hard to tell with the scoping rules). I really don't like the import system, it feels like it's trying to accommodate every guess that every novice programmer ever made, and it's anticipating a few others.

I mean, whitespace?

It's too heavy and slow. Like, unwieldy if you're really churning through a lot of data, which maybe you sometimes do with a game backend. I get that it's scripting and that it's "batteries included", and I'm sure all the correct tradeoffs were made, but sometimes it's just, too, slow.

Lua

I like Lua. It's small and quirky and Really Fast. Don't take my word for it, go hunt down some benchmarks. That's mostly what I like about it, I'm as easy to please as I am to annoy.

There's lots of things to dislike if you want to call me out. It's niche, there isn't a large community or body of libraries. Outside of the obvious things like networking and database and JSON parsing, there's a good chance you'll have to bolt on that third party library yourself.

But that's easy to do because embedding is one of the things it was created for. It's very easy to call Lua functions from C and C functions from Lua. So bolting on that third party library, while a pain in the ass, isn't going to require a systems programmer. (Not gonna lie, having one is better)

Another thing people love to hate: 1-based arrays. Fair enough, it's a bit weird. That said, aren't arrays just for stuffing things into and then pulling out sequentially? Who gives a shit about their offset? If you want to store things and retrieve them from a known location, use tables with string keys. They're still faster than anything outside of C, and the 80's are over, we can spare the cycles.

I wish more people would move to Lua 5.3, it's just better. The previous versions encouraged the use of globals for library tables, and it's just not The Right Thing. I haven't looked into 5.4 yet, maybe it's better still.

C++

I don't much like C++. Sorry, but I don't. We're still writing templates as header-only code because we're still pretending that it's forward-compatible with C, so compilers can't do generics in a sane way. We use the pre-processor instead, like animals, even though the loosest compiler can't compile C code without alterations, because it doesn't allow a boatload of things that C does.

char *ptr = malloc(8);  // 100% legit C, but C++ requires a cast

On top of all that, template syntax looks like line noise; it is truly, spectacularly awful. So you'd better get it right the first time because there's no debugging that shit six months down the road.

Many people are still writing header-only classes for non-template code because evidently we're also still pretending that we can out-optimize compilers by inlining everything. This causes a downward spiral of dependencies, and if I touch any of those headers I have to rebuild my entire codebase because everything includes everything else, and we may as well not have linkers any more. I don't know why, but this is a thing peculiar to C++ people.

I hate that there's no compatibility between C++ compilers. You can't really distribute a pure C++ library, because everybody has their own name-mangling scheme (this might not be true, but it feels true) and their own ideas about linkage and ABI. If you want C++ inside a library, you wrap it in a C interface and you hide the filthy internals from the world.

Finally, any language that plans major changes every three years by decree seems like more work than it's worth. Spending years learning the syntax and nuances of a moving target is for people younger than me. As I mentioned, I already know 30+ languages, I'm just kinda over it.

C

I like C for the same reason I like Lua. It's small and quirky and fast. It took me very little time to learn the entire syntax, and anything I need to do that the base language doesn't supply is in a library somewhere. If it isn't, I don't want it, and I really mean that.

I like that C compilers are actually portable. If I write C code using GCC as my reference compiler, it will build just fine with Visual Studio or Clang, or any other weird compiler you want to dig up. I can also compile these things into a library, shared or static, and I know exactly what the linkage rules are and so will the system that I use that library with five or ten years from now.

What I use in practice is generally a mix of C and C++. I don't mind classes, though I rarely need polymorphism. C's biggest downfall day-to-day is string handling, and there's lots of support in C++, so if I have to do it, I have help.

Those languages really tied the room together

So how is the project using these languages?

• Java for system services, because Android, but only where I have to
• C/C++ via the NDK
• Lua for as much as I can get away with
• Possibly some Python for the backend, I haven't explored all my options with this

Lua is definitely not as fast as C, so once all the logic is in place, I profile any slow parts and convert them to C. This is kind of my normal workflow; it's my slightly-more-modern analog of profiling C code and writing the crucial loops in assembler.

Indie Game Dev: Source control

This probably should have come before Platforms, but I always knew it was going to be an Android game. Source control is usually the first thing I think about, because regardless of anything else you might do, you need to store files somewhere.

If you want to create all your design documents and spreadsheets with cloud tools, I get that. They're generally easy to use, and easy to collaborate with. But if any of those things produce artifacts that go into the build, you need to export them and check em into source control, because that online tool might change or go away.

I need my source control to handle large binary files in a sane way, because who knows what kind of bloated garbage those things are going to generate? While we're on the subject, artists also produce a lot of large binary files, and they too, have to go somewhere.

Everything is distributed/remote these days, so your source control system either needs to be fully network aware, or at least have a feature set that works well over a network. Don't forget, large binary files.

Automated tools will need to check things out, so there needs to be a CLI. Normies -- designers, artists, producers -- need to use it, so there needs to be GUI too, and it should be simple to use.

You need branches, and you probably need to be able to cherry pick changes from those branches from time to time. Maybe you can get away without it, but you'll be a lot less productive, I've seen the before and after.

You need logging and history. When your product actually ships, you'll be fixing bugs and it's easier if you can go through a file's history to see when that bug crept in.

Event hooks are nice. If your system can reject a submission based on a schema or other kind of validation, a lot of errors never make it through the door. You can have notifications, or do backups or any number of interesting things if you know when a specific file changes.

Enough about feature sets. It's decision tree time. What don't I like?

Discontinued/obscure: RCS, CVS, TFS. I especially don't like CVS or anything of that vintage, because it has to contact the server for everything, even just to do a diff. It's inefficient and frustrating, and nobody with any sense is still using any of these things.

Horror of horrors, I don't much like Git, either. The workflow is needlessly complicated, and it uses terms similar to other source control systems, but for different concepts. Sit down in the back, better people than you have tried to talk me into it.

Source control systems that I do like are Subversion and Perforce. Subversion is extremely easy to teach to non-technical folk, and while Perforce is definitely not as easy, its power makes the slightly steeper learning curve worth the investment.

I use Perforce at the office, and it ticks all my boxes. I'm choosing it over Subversion because:

• It's change-centric instead of file-centric
• Cherry picking individual changelists is easy
• Shelving changes is easy
• The GUI, while occasionally... imperfect, has things like revision graphs and "time-lapse view" and I've forgotten what life was like without them.

One thing that irritates me and always will is there seems to be no way to undo changes to a file and keep it checked out. You have to revert the entire file, which removes it from your changelist, and then you have to add it back. If you know of a quick way to do this, please let me know.

It's free to use for up to 5 users, but obtaining the P4V client requires that you hand your email address over to Perforce. Their sales emails are easily ignored.

So there you are. It's Perforce for me.

Indie Game Dev: Platforms

If I were publishing this game, what platform would it be on? Android. Fade to black.

I'm completely serious, but let's see how I arrived there.

I'm just doing this for fun and to build out my pipeline, at least for the first project. So it's not going to be a huge, serious piece. No 3D augmented reality, no social media presence, no infinitely scalable backend hosted out of an old missile silo.

Console is out, they don't give those SDKs to just anybody, and I don't own any consoles regardless. Ouya is dead, and sadly I never bothered getting one. I'm not going to cobble something together with a Raspberry Pi, that's not the project I'm looking for right now.

PC is of course do-able, but it's harder and less fun than you might think. If you're just making a game for yourself that's one thing, but if you have any designs on publishing down the road, you have to face the nightmare that is Other People's Weird Hardware.

I really just want to bang out a quick little game using a simple system. I want something that could be published via Somebody Else's Platform, in case I change my mind and go that route. I'm a mobile guy and I (mostly) understand that space and those systems. So it's gonna be iOS or Android.

Either platform is theoretically fine, I'm equally familiar with both. Let's do pros and cons.

iOS pros:

• C-like native code -- I don't mind Objective C, it's quirky, and I like quirky
• Lots of framework support

I had trouble thinking of any others. Cons:

• Objective C is probably going away in favor of Swift, and I'm too tired to learn another language today, so very tired
• It's a closed system
• You have to have a Mac, and what's more, you have to build and debug with XCode
• You can't build and run an app on your own device without a developer's license
• The license is $100 USD / annum
• The code signing procedure is bonkers complicated, and you have to renew 3 or 4 cryptographic keys/certificates/profiles every year
• IDFA and various other monopolistic practices
• The approval process can be a crapshoot

Android pros:

• Open system
• You can develop and build your apps on any platform -- including OSX if that's your jam
• Once you've built that APK, you can install it on any Android in the world, nobody cares
• A developer's license costs a one-time fee of $25 USD, and you don't need it until you want to distribute the app on Google Play
• You create a single production key once, and signing is all automatic after that. Google will even hold the key for you.
• Advertise with whomever you want. Collect whatever data you want, provided you adhere to the laws of the countries you're publishing in
• Is there an approval process? There is, but I've never seen a rejection. Seriously, never have I ever
• Crash reports that you can symbolicate yourself

Cons:

• Java. *Sigh*. This is mostly mitigated by the NDK, but if you want to interface with any OS services, you're probably doing it with Java
• App permissions are a bit of a moving target

As you can see, I'm no fan of iOS, and the "privacy" shenanigans over the last year have not helped that one little bit. I love Woz, but everything else Apple can be launched directly into the sun.

So... Android. Fade to black.

Indie Game Dev: How I choose software and systems

Most game projects start with an idea, a design, and that seems sensible. But there's nothing better for me than having the whole pipeline worked out so I can start executing on those ideas right away. A big part of this project is having everything exactly the way I want it, so that means I'm building that pipeline.

Where to begin? There are a lot of choices, and they can be paralyzing. I've always started with the deal breakers: the things I absolutely will not do, not use, not put up with. Having some part of your workflow irritate you day after day is not sustainable, so discard those things up front, and keep them out of the ecosystem if you can.

I have lots of deal breakers, so it's easy to whittle the list down quickly. I'll discuss some of them in greater depth in later posts about specific topics.

• Java - it's a necessary evil in places, but if the stock install doesn't satisfy a given system or tool, I'm not going to use it.
• Requiring an IDE to build a resource. Make CLI tools for that, my CI/CD server doesn't have a mouse.
• Not generally a fan of obscure/discontinued tools, languages, or resource formats. The documentation is usually thin, and good luck finding answers on Stack Overflow.
• Also don't much care for Git. You heard me.

Once you've called out all your deal breakers, a swathe of additional tools will be eliminated because they involve those deal breakers. Embrace the constraints, they'll set you free.

With the distasteful stuff out of the way, you can turn your attention to things you really like, and let those things drive your next set of decisions. If you can't have a little fun, why do any of this in the first place?

• I love C and Lua. I like small languages in general.
• I like vscode quite a lot.
• I like JSON.
• I like make.
• I like Jenkins.

If it sounds to you like I've already chosen a lot of things, there may be some, but not necessarily these ones. These are just my preferences, and any real choices have to find their way through this decision tree.

Sometimes I choose things based on what I know best, even if I think something else might be better. Sometimes your hand is forced.

Next time, I'll talk about what I chose first.

Indie Game Dev: How I'd Do It Today

This is my 20th year as a professional programmer. In the early days it was a side hustle -- cobbling shell scripts together to keep the lab ticking over (I got paid, it counts), creating a web page for the company web site, that kinda thing. Eventually I'd "paid my dues" and was able to get the programming jobs I wanted.

(Total bullshit. I did do those things and many more, but I got my first programming job via a referral from a friend. That's how you get the first gig, don't let anybody tell you different.)

I'm a generalist. I've done QA automation, machine vision, network security, web apps, games. Mobile games, probably nothing you've heard of, but lots of them. I'm interested in low level bit twiddling, high level scripting, backend, build engineering, QA, debugging. I've done more time in gaming than anything else, and I continue to surrender to its siren song, but I never really specialized.

A few years back I did indie game dev for real: worked at my own studio, created and released an original title. Our systems guy called it "the best little phone game ...that nowhere near enough folks played", and that feels about right.

I find myself missing those days, or maybe just missing the freedom to do things however the hell I wanted to do them. Even then I wasn't completely free, the studio predated me by a couple of months, so there were a few preexisting systems.

What if I could start out again, do the whole thing exactly the way I wanted? What would I choose? I don't really know either, but I have a plan to figure it out.

I'm going to quietly recreate some portion of the game my studio produced back in 2014. There was a lot of content, so I'm not going to do everything, but what I do make will be top quality, fully functioning, no-sharp-edges software. It will be shippable, but for reasons I won't share here, I'll stop short of shipping it.

This is meant to be the first in a series of posts, and hopefully I can stick to it long enough to convey everything I intended to when I started. What will we be discussing here? Off the top of my head:

• Platforms - what hardware/os will this run on?
• Languages - system, scripting, tools, and more
• Source control - the source has to live somewhere
• Build tools - why is there anything other than make?
• Third party libraries - sometimes you can't roll your own
• CI/CD - because building the shipped product at your desk is The Wrong Thing
• IDEs and editors - vim vs emacs is over
• Content tools - what artifacts do designers create, and how do they do it?
• Live Ops - shipping is now the first step, not the last
• Sunsetting - when the returns diminish, what then?

I'll probably talk a lot about software engineering, because after 20 years I still don't see very much of it, and that makes me sad.

I'm writing this for myself, because we all do, but I do hope others find it useful or interesting. Hopefully I can provoke an "ORLY" once in a while.

VS Code

 The other day I mentioned to somebody that I started writing code A Long Time Ago (my guess is 1982 or 83); among other things, this means I've witnessed the evolution of many programing languages and tools over the years.

It's unclear -- to me, anyhow -- whether languages have progressed much in that time. There's always something going on that "feels" new, but many up-and-comers languish in obscurity, and anything good eventually gets frozen in time because nobody wants to obsolete millions of lines of code. OOP isn't new, closures aren't new, strong/weak/dynamic typing isn't new, and whatever Javascript framework you're about to mention isn't interesting or useful. Or new; it's just Javascript.

I'm happy to defend this viewpoint for hours, but that's not my intended focus. I really wanted to talk about tools. Tools have absolutely progressed, mainly because (A) we've had lots of time to make them better, and (B) we have so much more speed and memory with which to drive them. Plus, you can update a tool without breaking people's code (are you listening Clang?). Since I'm a programmer, debugging tools is where I'm headed specifically here.

I would assert that Visual Studio has been setting the standard for debuggers for the last 20 years or so. Some would say, "On Windows, sure, but not on <platform XXX> or <language that visual studio doesn't do>". Maybe, but do the tools for that platform or language provide the kind of usability and programmer productivity that Visual Studio does? No they don't, put your hand down.

XCode has come pretty close as of late, though with all of the context-specific toolbars and menus that spring into place every time you select a different item in one of those menus, it's sometimes hard to find what it was you were looking at 10 seconds and 3 clicks ago. Not to mention updating the thing is a 10-12GB download, which is a little irritating.

Enter Visual Studio Code; hereafter referred to as "vscode". Don't confuse this with Visual Studio; for one thing, vscode is open source. You can check it out from GitHub, and as I understand it, they do accept external contributions. I don't care quite as much about that as I once did, but what I do care about is that it runs on Windows, Linux, and macOS. It has a plugin interface that allows anybody to add support for their language of choice. And it's fast.

All the languages I work with day-to-day -- probably 4 or 5 -- are debuggable in vscode with extensions freely available in the Visual Studio Marketplace. The configs are all json and extendable enough themselves that you can probably create a build/run setup that will enable you to do it all with a single keystroke. For bonus points, you can do "compound" launches, so if you needed to, say, run 2 game clients simultaneously in order to test PVP, it can run them both. In their own debugger. In the same vscode window. True story.

There are extensions to edit and run applications via SSH; I regularly sit at my Windows machine and write Python backend code. With the Flask extension, it will run that application on my Linux box, with my debugger attached.

You can't necessarily throw away all your existing tools. I still need Visual Studio to create a base Windows project, or Android Studio to get gradle set up, but once that's done, I can push them to the side. Most importantly, I never have to use a Java program to debug my Java Android code ever again. All other IDE's, don't let the door hit you in the ass on your way out.

It's better in person

All checked in, waiting for our flight. Things at YVR couldn't be more different (better) than my online check in experience.

We hit the airline counter when they opened, everything went smooth as silk. Huge line at passport control, still got through in 10 minutes. I think we walked a couple miles to the gate, but I guess somebody has to be furthest.

It's pretty clear that the airlines' internal systems are a hell of a lot more squared away than their external systems.

A Spanish Christmas (nearly)

We're off to Spain until Dec 24th. Herself and I have spoken often about going away for Christmas -- though to be honest, the talk was usually about Whistler or Harrison Hot Springs. Still, Christmas in a different culture seems like a thing I'd like to try at least once.

Airline check in went poorly as it always does for us: We're not actually checked in yet, because apparently online check in is not a solved problem. I did manage to reserve seats, so I suppose it'll probably be fine, but when the support line's tech support doesn't know what went wrong, it doesn't inspire a high degree of confidence.

Remember when "checking in" meant just showing up, and you were assigned seats when you booked the flight months and months in advance? Maybe I'm getting too old for this.

I think once we're wheels down in Madrid, everything's gonna be fine.

Hasta próxima

Whether I need to or not

Just having a whip round my bookmarks, and I realize that it's been exactly a year since my last post. Seems like something a guy should do at least annually, so let's do a review of 2013.

• Went to Spain
• Quit smoking. Again.
• Left my job at a game studio, invested in a new game studio. What the hell was I thinking?

There's probably some other major stuff, but it's a bit of a blur now. Other minor (but perhaps fun) things:

• Taught my sister's kids how to say "What's that do?" just like Gir
• Joined the 21st century and got a smart phone (albeit a crappy one)
• Started losing weight instead of gaining it for a change
• Finally set up an LDAP server successfully.
• Read a surprisingly large amount of surprisingly good sci-fi, both old and new

Is that enough? Can I go?

A connection by any other name...

A few years ago I mentioned James Burke's "Connections" here. A friend saw the link and was as pleased to revisit the show as I was. Yesterday he repaid the favour by pointing out a talk that Burke had given last year. Same old Burke - that is to say, completely new and innovative and chock full of connections.

Admiral Shovel and the Toilet Roll

Give it a listen - you won't be sorry.

CPU emulator - Part II

OK, posting version 1.0 - hopefully somebody finds it interesting. Everything I mentioned in the previous post is done. No hardware interrupts at this point, but there's no peripherals that would need one yet.

Get it here.

The docs need some work, but if you're the sort of person who's interested in this kind of thing, the architecture and brief instruction set are probably all you need. Maybe the peripherals. I've included a test assembly file that I think exercises the entire instruction set.

BTW, the files are hosted at cx.com, as an ex-coworker of mine works for them now, and it seems pretty cool.

CPU emulator - Part I

I messaged the mailing list a few days ago regarding a CPU emulator that I found last week. I played around with it for a day or two, and then decided that,

1. It was missing at least one instruction I wanted, and
2. the source is Delphi, and I don't want to go down that road.

So of course now I have to make one myself, so that I can alter the instruction set and add peripherals. And perhaps make it a little more configurable - put the peripherals on ports of my own choosing, select interrupts, etc.

All this is in aid of getting closer to a machine, any machine. Sure, I could just learn/write x86 assembly, but that just doesn't seem like enough "fun" to me. Plus, this gives me a little more freedom to be imaginative.

I'm currently writing it for Win32, simply because it's easier to hook up a GUI. If it makes you feel any better, the core is pretty much standard C++98 (I don't use any exotic C++ features), and I'm building it with a cross-compiler on my Ubuntu VM. This is mainly because modern MS compilers seem to require a fairly advanced version of Windows, and I want some backwards compatibility (Luddites rejoice!). You don't need Windows 8, or any .NET bullshit, or much of anything except a bare-bones Win32 installation; in theory it could run on Win98, but Win2K and up for sure.

At some point I might explore some sort of Linux GUI, but regardless of that, I'm fully committed to making sure the core CPU classes are portable enough that one could easily connect it to a simple console program. Load machine code, step, examine registers/memory, step, etc. We'll see. FYI, bitching always sounds to me like volunteering...

Current Feature Set
It's an 8-bit microprocessor, with (predictably) 256 bytes of RAM. It has 4 general purpose registers - AL, BL, CL, DL - and 4 special purpose registers - IP, SP, BP, SR (aka FLAGS). I've implemented all the move, arithmetic, binary logic, and comparison instructions. The GUI can load a machine code file, and you can step through it; RAM and registers are on display full-time.

Coming soon...
Obviously need to expand the instruction set - branching, procedure calling, interrupts, and I/O. Definitely want to make some peripherals, and I'd dearly love to implement them via some sort of pluggable interface, though I don't know how portable that would be. Plus there's currently no assembler, so I'm hand-assembling the machine code for now, which is even less fun than it sounds if you have to test a feature that requires any real set up.

I'll post a build once it has coalesced into something useful, if you can call it that.

2 things...

#1: If you work a desk job, don't sprain your ankle. Sprained ankles hate desk jobs.

#2: If you sit under the AC vent at said desk job, don't get sunburned.

Just sayin.

Stupid postfix tricks

I've recently had a need to redirect mail from a herd of linux boxen; the reason isn't interesting. The bottom line is that I wanted to somehow pipe all the mail emanating from these systems - mostly cron errors - through a script. Turns out that Postfix makes this fairly easy to do. (Note that I'm using Ubuntu 10.04 LTS - your file locations may differ).

Step 1. Edit /etc/postfix/master.cf

This file configures the Postfix master process, and is essentially a set of named services. The first argument in a line is the name of a service. Let's call ours "myscript". There's a bunch of other arguments, do man 5 master for the nitty-gritty details.

The last argument is a command + any required arguments for said command. A lot of the pre-existing lines in this file are calling one of the other daemons in the Postfix suite, but the one we're calling will let us do pretty much anything we like - it's called pipe. Again, do man 8 pipe to learn more, but I will share the basics. What pipe does is calls any executable you'd like and pipes its output into it. So let's pipe this into mailscript.php, which I prepared earlier:

myscript unix  -       n       n       -       -       pipe
  flags=O user=mail argv=/usr/bin/php5 /home/dante/mailscript.php ${recipient} ${sender}

Yes, yes, this is two lines (it's only two - the second one wrapped). It's not, though - it uses the standard "folding" technique whereby you continue a line by starting the next one with whitespace. The flags param mostly adds various email headers that you may want - check man 8 pipe for details. The user param tells master what user should this command run as. argv is the actual executable (plus arguments) that master will call and pipe its output to. ${recipient} and ${sender} will expand to the full email addresses of recipient and sender respectively. Note that the format and order of the argv param is entirely up to you - if you want to make your script executable and run it directly, that's cool.

So the net effect of this is the equivalent of the "mail" user making this call from the shell:

/usr/bin/php5 /home/dante/mailscript.php ${recipient} ${sender}

Step 2. Edit /etc/postfix/main.cf

You need to locate (or add) the line that begins with "default_transport". This defaults to "smtp", which is normally how one wants one's mail to be delivered. Change it to read:

default_transport = myscript

And you're pretty much done.

Step 3. Write your script

In our example here, ${recipient} and ${sender} will be $argv[1] and $argv[2] respectively, and STDIN will contain a standard RFC 2822 message. That is, the first set of lines coming in will contain the email headers, followed by a blank line, followed by the message itself. My current M.O. is to call trim(fgets(STDIN)) in a while loop to gather up the headers and let the loop die when it sees an empty string - then you can just suck up the rest of the input.

Step 4. Restart Postfix

Step 5. Profit

That's pretty much it, I think. As I (hopefully) implied, you can pipe the output into pretty much anything that will accept it, so write a Python or Perl script, a C/Java/Erlang/etc program, pipe it into /dev/audio if that's the cut of your jib. Make Postfix your bitch.

Column oriented databases

If you find yourself in the market for something to handle your analytics - or any other application that generates tens of millions of rows in your database - do yourself a favour and look at InfiniDB. It's a database engine for MySQL, and it works exceptionally well.

We tried a number of the popular "NoSQL" options, but they're all incredibly difficult to set up, and they all require a separate API to use, and it's generally a Java API. I have nothing against Java, but given the choice I'd rather use the PHP MySQL API since we already have a tons of code using that. The final nail in the coffin for these systems was that our management wanted to be able to do realtime ad-hoc queries, and we haven't seen a NoSQL option yet that offers such a thing. They'll grind through ten billion rows, but they aren't going to do it with you sitting there watching - they'll get back to you in a little while.

Yes, it's a commercial product. That being said, until you need to run a distributed (i.e. multi-machine) setup, you can use the "community edition" for nothing, regardless of what you're using it for.

The Soul of a New Machine

I just re-read The Soul of a New Machine by Tracy Kidder. Just stellar. If you're interested at all in the history of computing, you really must read this book. It follows a team of engineers designing and building an early 32-bit minicomputer in the late 70's. Kidder was brought in quite early in the development to document the process and he did an exemplary job of it.

If you're too cheap to buy it and your local library doesn't have it, you can borrow my copy. Yeah, that's right. It's now part of my library, it's that good. (I read 20 or 30 novels a year, and they normally go out the door as quickly as they came in.)