VS Code

 The other day I mentioned to somebody that I started writing code A Long Time Ago (my guess is 1982 or 83); among other things, this means I've witnessed the evolution of many programing languages and tools over the years.

It's unclear -- to me, anyhow -- whether languages have progressed much in that time. There's always something going on that "feels" new, but many up-and-comers languish in obscurity, and anything good eventually gets frozen in time because nobody wants to obsolete millions of lines of code. OOP isn't new, closures aren't new, strong/weak/dynamic typing isn't new, and whatever Javascript framework you're about to mention isn't interesting or useful. Or new; it's just Javascript.

I'm happy to defend this viewpoint for hours, but that's not my intended focus. I really wanted to talk about tools. Tools have absolutely progressed, mainly because (A) we've had lots of time to make them better, and (B) we have so much more speed and memory with which to drive them. Plus, you can update a tool without breaking people's code (are you listening Clang?). Since I'm a programmer, debugging tools is where I'm headed specifically here.

I would assert that Visual Studio has been setting the standard for debuggers for the last 20 years or so. Some would say, "On Windows, sure, but not on <platform XXX> or <language that visual studio doesn't do>". Maybe, but do the tools for that platform or language provide the kind of usability and programmer productivity that Visual Studio does? No they don't, put your hand down.

XCode has come pretty close as of late, though with all of the context-specific toolbars and menus that spring into place every time you select a different item in one of those menus, it's sometimes hard to find what it was you were looking at 10 seconds and 3 clicks ago. Not to mention updating the thing is a 10-12GB download, which is a little irritating.

Enter Visual Studio Code; hereafter referred to as "vscode". Don't confuse this with Visual Studio; for one thing, vscode is open source. You can check it out from GitHub, and as I understand it, they do accept external contributions. I don't care quite as much about that as I once did, but what I do care about is that it runs on Windows, Linux, and macOS. It has a plugin interface that allows anybody to add support for their language of choice. And it's fast.

All the languages I work with day-to-day -- probably 4 or 5 -- are debuggable in vscode with extensions freely available in the Visual Studio Marketplace. The configs are all json and extendable enough themselves that you can probably create a build/run setup that will enable you to do it all with a single keystroke. For bonus points, you can do "compound" launches, so if you needed to, say, run 2 game clients simultaneously in order to test PVP, it can run them both. In their own debugger. In the same vscode window. True story.

There are extensions to edit and run applications via SSH; I regularly sit at my Windows machine and write Python backend code. With the Flask extension, it will run that application on my Linux box, with my debugger attached.

You can't necessarily throw away all your existing tools. I still need Visual Studio to create a base Windows project, or Android Studio to get gradle set up, but once that's done, I can push them to the side. Most importantly, I never have to use a Java program to debug my Java Android code ever again. All other IDE's, don't let the door hit you in the ass on your way out.

A Spanish Christmas (nearly)

We're off to Spain until Dec 24th. Herself and I have spoken often about going away for Christmas -- though to be honest, the talk was usually about Whistler or Harrison Hot Springs. Still, Christmas in a different culture seems like a thing I'd like to try at least once.

Airline check in went poorly as it always does for us: We're not actually checked in yet, because apparently online check in is not a solved problem. I did manage to reserve seats, so I suppose it'll probably be fine, but when the support line's tech support doesn't know what went wrong, it doesn't inspire a high degree of confidence.

Remember when "checking in" meant just showing up, and you were assigned seats when you booked the flight months and months in advance? Maybe I'm getting too old for this.

I think once we're wheels down in Madrid, everything's gonna be fine.

Hasta próxima

Whether I need to or not

Just having a whip round my bookmarks, and I realize that it's been exactly a year since my last post. Seems like something a guy should do at least annually, so let's do a review of 2013.

• Went to Spain
• Quit smoking. Again.
• Left my job at a game studio, invested in a new game studio. What the hell was I thinking?

There's probably some other major stuff, but it's a bit of a blur now. Other minor (but perhaps fun) things:

• Taught my sister's kids how to say "What's that do?" just like Gir
• Joined the 21st century and got a smart phone (albeit a crappy one)
• Started losing weight instead of gaining it for a change
• Finally set up an LDAP server successfully.
• Read a surprisingly large amount of surprisingly good sci-fi, both old and new

Is that enough? Can I go?

CPU emulator - Part II

OK, posting version 1.0 - hopefully somebody finds it interesting. Everything I mentioned in the previous post is done. No hardware interrupts at this point, but there's no peripherals that would need one yet.

Get it here.

The docs need some work, but if you're the sort of person who's interested in this kind of thing, the architecture and brief instruction set are probably all you need. Maybe the peripherals. I've included a test assembly file that I think exercises the entire instruction set.

BTW, the files are hosted at cx.com, as an ex-coworker of mine works for them now, and it seems pretty cool.

CPU emulator - Part I

I messaged the mailing list a few days ago regarding a CPU emulator that I found last week. I played around with it for a day or two, and then decided that,

1. It was missing at least one instruction I wanted, and
2. the source is Delphi, and I don't want to go down that road.

So of course now I have to make one myself, so that I can alter the instruction set and add peripherals. And perhaps make it a little more configurable - put the peripherals on ports of my own choosing, select interrupts, etc.

All this is in aid of getting closer to a machine, any machine. Sure, I could just learn/write x86 assembly, but that just doesn't seem like enough "fun" to me. Plus, this gives me a little more freedom to be imaginative.

I'm currently writing it for Win32, simply because it's easier to hook up a GUI. If it makes you feel any better, the core is pretty much standard C++98 (I don't use any exotic C++ features), and I'm building it with a cross-compiler on my Ubuntu VM. This is mainly because modern MS compilers seem to require a fairly advanced version of Windows, and I want some backwards compatibility (Luddites rejoice!). You don't need Windows 8, or any .NET bullshit, or much of anything except a bare-bones Win32 installation; in theory it could run on Win98, but Win2K and up for sure.

At some point I might explore some sort of Linux GUI, but regardless of that, I'm fully committed to making sure the core CPU classes are portable enough that one could easily connect it to a simple console program. Load machine code, step, examine registers/memory, step, etc. We'll see. FYI, bitching always sounds to me like volunteering...

Current Feature Set
It's an 8-bit microprocessor, with (predictably) 256 bytes of RAM. It has 4 general purpose registers - AL, BL, CL, DL - and 4 special purpose registers - IP, SP, BP, SR (aka FLAGS). I've implemented all the move, arithmetic, binary logic, and comparison instructions. The GUI can load a machine code file, and you can step through it; RAM and registers are on display full-time.

Coming soon...
Obviously need to expand the instruction set - branching, procedure calling, interrupts, and I/O. Definitely want to make some peripherals, and I'd dearly love to implement them via some sort of pluggable interface, though I don't know how portable that would be. Plus there's currently no assembler, so I'm hand-assembling the machine code for now, which is even less fun than it sounds if you have to test a feature that requires any real set up.

I'll post a build once it has coalesced into something useful, if you can call it that.

Stupid postfix tricks

I've recently had a need to redirect mail from a herd of linux boxen; the reason isn't interesting. The bottom line is that I wanted to somehow pipe all the mail emanating from these systems - mostly cron errors - through a script. Turns out that Postfix makes this fairly easy to do. (Note that I'm using Ubuntu 10.04 LTS - your file locations may differ).

Step 1. Edit /etc/postfix/master.cf

This file configures the Postfix master process, and is essentially a set of named services. The first argument in a line is the name of a service. Let's call ours "myscript". There's a bunch of other arguments, do man 5 master for the nitty-gritty details.

The last argument is a command + any required arguments for said command. A lot of the pre-existing lines in this file are calling one of the other daemons in the Postfix suite, but the one we're calling will let us do pretty much anything we like - it's called pipe. Again, do man 8 pipe to learn more, but I will share the basics. What pipe does is calls any executable you'd like and pipes its output into it. So let's pipe this into mailscript.php, which I prepared earlier:

myscript unix  -       n       n       -       -       pipe
  flags=O user=mail argv=/usr/bin/php5 /home/dante/mailscript.php ${recipient} ${sender}

Yes, yes, this is two lines (it's only two - the second one wrapped). It's not, though - it uses the standard "folding" technique whereby you continue a line by starting the next one with whitespace. The flags param mostly adds various email headers that you may want - check man 8 pipe for details. The user param tells master what user should this command run as. argv is the actual executable (plus arguments) that master will call and pipe its output to. ${recipient} and ${sender} will expand to the full email addresses of recipient and sender respectively. Note that the format and order of the argv param is entirely up to you - if you want to make your script executable and run it directly, that's cool.

So the net effect of this is the equivalent of the "mail" user making this call from the shell:

/usr/bin/php5 /home/dante/mailscript.php ${recipient} ${sender}

Step 2. Edit /etc/postfix/main.cf

You need to locate (or add) the line that begins with "default_transport". This defaults to "smtp", which is normally how one wants one's mail to be delivered. Change it to read:

default_transport = myscript

And you're pretty much done.

Step 3. Write your script

In our example here, ${recipient} and ${sender} will be $argv[1] and $argv[2] respectively, and STDIN will contain a standard RFC 2822 message. That is, the first set of lines coming in will contain the email headers, followed by a blank line, followed by the message itself. My current M.O. is to call trim(fgets(STDIN)) in a while loop to gather up the headers and let the loop die when it sees an empty string - then you can just suck up the rest of the input.

Step 4. Restart Postfix

Step 5. Profit

That's pretty much it, I think. As I (hopefully) implied, you can pipe the output into pretty much anything that will accept it, so write a Python or Perl script, a C/Java/Erlang/etc program, pipe it into /dev/audio if that's the cut of your jib. Make Postfix your bitch.

Column oriented databases

If you find yourself in the market for something to handle your analytics - or any other application that generates tens of millions of rows in your database - do yourself a favour and look at InfiniDB. It's a database engine for MySQL, and it works exceptionally well.

We tried a number of the popular "NoSQL" options, but they're all incredibly difficult to set up, and they all require a separate API to use, and it's generally a Java API. I have nothing against Java, but given the choice I'd rather use the PHP MySQL API since we already have a tons of code using that. The final nail in the coffin for these systems was that our management wanted to be able to do realtime ad-hoc queries, and we haven't seen a NoSQL option yet that offers such a thing. They'll grind through ten billion rows, but they aren't going to do it with you sitting there watching - they'll get back to you in a little while.

Yes, it's a commercial product. That being said, until you need to run a distributed (i.e. multi-machine) setup, you can use the "community edition" for nothing, regardless of what you're using it for.

The Soul of a New Machine

I just re-read The Soul of a New Machine by Tracy Kidder. Just stellar. If you're interested at all in the history of computing, you really must read this book. It follows a team of engineers designing and building an early 32-bit minicomputer in the late 70's. Kidder was brought in quite early in the development to document the process and he did an exemplary job of it.

If you're too cheap to buy it and your local library doesn't have it, you can borrow my copy. Yeah, that's right. It's now part of my library, it's that good. (I read 20 or 30 novels a year, and they normally go out the door as quickly as they came in.)

Back to Basics

Back in September, I had some words to say about Java. Now I'm gonna say a few about C++.

While I've never been a big fan of templates, I'll admit that I use the STL from time to time just to get things done. On a recent project, I had a need for string-based maps, and as we all know, C/C++ just barely acknowledges strings as a data type. I used the examples, but was running into all kinds of memory errors. Eventually I came to the conclusion that I just don't have the template-fu to do this properly.

Fuck it, I'm a programmer, I should be able to just make something that works for me. So I did. And while I was at it, I made my map keyable on integers as well (I needed to search via two different criteria, and maintaining two containers is wicked error-prone). I learned a lot about searching, and hashing, and a bunch of other miscellanea that I wouldn't have otherwise had the opportunity to explore.

While I'm the first guy to say, "Look around for a library, somebody's probably already done this" - and they usually have - I'm wise enough (I hope) to know when it's time to roll my own. It's a little extra effort, but sometimes it's the only way to go. If you need a Swiss Army knife, but everybody's making plasma torches, make your own knife.

A Sad Day Indeed

Last Saturday when we were all looking the other way, one of the Titans of computer science left us. If there's anybody I would aspire to emulate, it's Dennis Ritchie. The man gave us C and Unix, and really, the entire idea of portability sprang from his lab. Every aspect of modern computing - save networking - is a descendant of his work in some way.

Bon Voyage Dennis, you were one of the great ones, and you'll always be my hero.

Sometimes the latest and greatest just isn't

Is it just me?

Thunderbird updated the other day, breaking yet another one of my plugins. I think they're all kaput now. Thanks, Mozilla. iOS 5 is coming out soon, which means we have to tear around and make all of our apps "compliant" or they'll get pulled from the AppStore. Thanks, Apple. We're mostly moved onto Blogger, and that's fine, but now my RSS feeds are all hosed (TBird thinks all the imported entries have the same date), and there's no option to have more than 25 items in the feed. Thanks, Google.

Everybody's in a hurry to update their software, but nobody gives a shit about what people already have in place. I'm all for progress to a point as long as it doesn't involve discarding everything I have right now. Everybody beat up Microsoft for years for doing this, yet for some reason we've all decided now that it's a perfectly acceptable practice.

Screw that. Stop breaking your software. You're not going to inspire much loyalty to your brand if moving to another system is just as easy as upgrading yours. And increasingly it is.

The End of an Era

So long, Steve. For better or worse, you changed the world.
http://www.apple.com/stevejobs/

My Old Nemesis

xkcd reference

I'm not really writing any Python, but I've been having a similar feeling lately about an old nemesis: Java.

There's been some Android talk around the office lately, and while I did do an Android port of one of our games, it wasn't the best experience. That's not Java's fault, it's the fault of the original programmers, who apparently thought that C++ was the only language they - and everybody else here - would ever need. People who aren't well-versed in all the languages that you might use in-house shouldn't write libraries, or at least they shouldn't design them. But I digress.

As I mentioned, there has been talk. We have partners (who can't be named) that are interested in publishing on Android, so I've been doing some light R&D. No porting, no real direction, just checking the lie of the land. I have to say, I'm rather enjoying it. I still have plenty of legitimate concerns - they're legitimate in my head at least - but I'd forgotten how many wonderful high-level constructs are pretty much built right into the language.

Need threads? Have your class extend Thread. Most concurrency issues can be addressed with synchronized blocks, no need for mutexes. Ah, you say, but I don't want to have to create a whole bunch of threads all the time, that's expensive. That's ok, Java has ThreadPoolExecutor. You want fries with those worker threads?

We use a lot of JSON round here - XML is so last decade - and it turns out there's a parser built in. Tick that box. Oh, and we use http as a transport for that, and of course that's been present for years, thanks to Apache. Redirects are handled automatically, and all the compression/encryption functions are also in there, and they "just work". If you just need to show the user a web page, they've got a webkit and you can just slap a browser window right there on your view.

All the usual hardware geegaws are exposed for you - USB, BlueTooth, Wifi, accelerometer, audio. And you're not left just sitting there having to figure those out, there's high-level classes to assist you - gestures, audio fx, sip, telephony, the list goes on.

I've also discovered all sorts of nifty things that have been in Java practically from the start, but was too green (or too stubborn) to use. Container classes out the ying-yang. Object serialization. Reflection - are you kidding me? I still struggle with the I/O stream hierarchy pretty much every session (what do I need here? a BufferedArrayInputSequenceFileStreamReader? what does that even mean?), but the docs are almost always helpful, and eventually I figure out the right thing.

I guess the bottom line is that the honeymoon period for me and programming is over, and I'm finding myself seduced by the higher level concepts that come with Java, and not with C. It might be all the PHP I've been writing for the last year and a half, but the appeal of the low level isn't what it used to be.

Quit screwing around... you screw around too much

A word of advice. Don't screw around with /etc/sudoers unless you have a root shell open. Shouldn't need to be said, but I just did it, so clearly somebody should have said it to me.

The best defense... is a good defense

We moved offices recently, and in every sense of the word it was a good move. The office is closer to the Skytrain (that's good and bad - sometimes I like the walk), the HVAC system is 100 times better, and we switched ISPs. This is all to the good.

One sort of bad thing came out of it: our trusty server box really didn't like being shut down. When we set it up at the new place, it didn't POST the first time we turned it on. But did we panic? We did not. Hit the reset button and try again. Ah there, it's fine. We shifted it a week later because its location was less than optimal, pretty much the same story. Again, we're not worry warts, we just hit reset until it booted.

That was stupid move #1.

Stupid move #2. The unit was giving off some sort of high-pitched whine - I have to take everybody's word for it, because I lost that range a long time ago. Anyhow, to combat the noise, we put the box in a cabinet. The cabinet isn't hermetically sealed or anything, but there was only about 5 or 6 inches of space at most around the unit, and I expect there wasn't a great deal of circulation happening in there.

Stupid move #3. SVN started acting funky. People would check stuff in, and the next time somebody updated, they'd get various error messages and the update invariably failed. Huh. So my partner in crime/sysadmin and I went to work on the SVN revisions with a pair of pliers and a blow torch. We restored what we could from the backups, and found some scripts that were known to fix symptoms of this kind for the newer revisions.

After 11 hours of this, we had a kernel oops. For those who have not yet experienced an oops, think of it as a not-quite-panic. The system definitely freaks out, but the kernel doesn't lock it up; it carries on and hopes for the best. The best was not our lot that evening, and a lot of zombie processes starting showing up. Fine. shutdown -r now. Except that didn't work either - shutdown was zombifying (if I may coin that term). Sigh. Hard boot.

Now. After 3 weeks of being cooped up in a hot cabinet, the system decided that I should serve as the example to others. Wouldn't POST. I thought perhaps if I could bring up the BIOS setup, maybe I could figure out what was up. The setup screen froze after about 10 seconds. 3 times in a row. Suck. So I waited 15 minutes and tried again. No dice. 30 minutes. Fuck you, Jack. Gave it an hour, the thing finally POSTs. I have 15 minutes to spare before I'd miss my last bus. Not happy, but at least everything's working and I can go home.

I vowed to get the boss to buy us new hardware the very next day. He wasn't planning on replacing it until the end of the year, but he's a technical person too, so he understands this needs to be done now. He commits to getting the hardware next week (this is now Friday) and starts speccing new gear. Great.

However, the cosmos had other plans for me. Over the long weekend, unbeknownst to me, the SVN is still acting up and the system gets 2 or 3 more hard reboots. I come in on Tuesday to find the RAID on which the repositories live is dead. D-E-A-D. As a fucking doornail. Neat.

I have a Linux box under my desk that I was using as a test bed for MySQL cluster. That testing's all done now, and except for the database, everything else is basically a stock install. I installed what I needed to get SVN working more or less as it did on the old box, and copied the (week-old) backups over. Took a little while because our repos are huge - well over half my day is eaten up by copying, exporting, reloading, verifying.

But now it's all done, and with a little care we managed to save all the history minus the past week, and no data was lost because of course everybody has the files they worked on still sitting on their own desktops.

So what's the moral of the story? If it looks like a duck, walks like a duck, and quacks like a duck - it's probably a duck. Pretending that a dying system isn't really dying is probably not the best strategy for data integrity.

Uptime

Last night:
[dante@roxy dante]$ uptime
01:52:10 up 371 days, 4:31

This morning:
[dante@roxy dante]$ uptime
13:56:54 up 16 min

Not that, you know, we use uptime as any sort of currency or anything... Anyhow, the long and short of it is that BC Hydro did some work in the area this morning, and they cut the power for 5 hours. So I had to shut down roxy for the first time in over a year. Pretty good for a 9+ year-old box. The drives aren't 9 years old (well, one of them might be), but everything else is for sure.

What's my secret? Frequent vacuuming ;)

Pay attention

I remember chatting with one of the engineers some years ago when I worked at the ferries. He was telling me a story about somebody who'd bumped into something and knocked off a bolt or some such, and caused a few thousand dollars worth of damage. The moral of the story was, "If you touch anything accidentally, look around and make sure you didn't fuck something up." Sage advice.

Fast forward to late 2008. I was editing my Samba configs, and was trying to figure something out, so I cranked up the logging to full debug (see where this is going yet?). Once I was satisfied, I closed smb.conf and walked away, leaving the log level at 9. Tsk.

This evening I was doing some PHP on the old box via Samba (why use vim when you have EditPlus). For some reason it was taking a long time - 2 or 3 seconds on occasion - to open the files. WTF. Top didn't show anything too untoward, except... hold on, the swapping daemon seemed to be working a lot harder than it should have been. What the hell is swapping? I restarted a few services, but nothing really made a difference.

I started poking through logs, and eventually found out the Samba logs were approaching 500MB in size. So of course Samba itself is the culprit, because not only is it logging like mad, it's appending that text onto the end of a HUGE file. So I cranked the logging level back down to 2, rotated the logs, and whaddaya know - files are opening quickly again.

Seriously - if you touch anything accidentally, look around and make sure you didn't fuck something up.

iPhone Blogging

For reasons not interesting to anybody, I was looking at our blogs with an iPod and noticed how nicely Todd's was formatted. Apparently he mentioned this a while ago.

So I enabled the plugin on my own blog, simple as that. Thanks Todd, it looks fabulous.

Cosmos

My last post was about James Burke and the genius that is Connections. Next on my to do list is Carl Sagan's Cosmos. Sagan's style is certainly different than Burke's, but no less visionary, and certainly no less interesting.

Strangely enough, I never watched this series when I was a kid, I'm not sure how it slipped past me. I haven't finished watching them all yet, but really enjoying it so far. One of his favourite phrases is, "I wonder...", and you can tell that he really did wonder.

We miss you, Carl.

Connections

I've been re-watching the Connections series on YouTube over the last few weeks. You remember all those great James Burke shows - technology... history... that stuff. Anyhow, I wrapped up the first series over the weekend, and I was just stunned at Burke's conclusions in the last minutes of episode 10. I don't want to give the surprise away, but keep in mind when you watch it (you are going to watch it, right?) that this was 1979.

For anybody who wasn't around in '79, let me paint a picture for you: there were no PCs - at least not like we have now; the Apple ][ got you 48KB of RAM, and the TRS-80 had 4K. We still dialed telephones, and Pink Floyd reminded us that we had "thirteen channels of shit on the tv to choose from". The Rolling Stones were still relevant, and fashion, well... we're still trying to forget all of that.

But I digress. Burke's observations were nothing less than prophetic - one or two details were not quite on the nose, but he clearly knew where we were heading. It's a fabulous programme in any case, and his white leisure suit is worth the price of admission. Watch them all, watch Connections2, Connections3 and The Day The Universe Changed. Science and history together make for outstanding television, and James Burke is nothing short of a genius.

James Burke playlists