Tuesday, September 14, 2010

Old Dogs, New(ish) Tricks

One of the many little things I've wanted to have a crack at over recent years but couldn't really do was tackle writing a Regular Expression library for the GC'd runtime I designed and wrote to use inside Ghost Solution Suite. When Russ Cox went and wrote a series of awesome articles on that after I'd written a big chunk of a Javascript interpreter for that runtime, I wanted to do it even more.

Alas, one of the things that I lost when Symantec cut me loose was access to all that code which I'd created and looked after over more than a decade. It's not of any use to them, but that's the way it goes and so I need to start my personal collection of Useful Stuff from scratch again, only this time I'll take a lot more care to avoid losing ownership of it. And to be fair, starting again means this time I don't have to worry quite so much about compatibility with broken C++ compilers, as I had to back in 1995 when even 1990-level template support and return type covariance were considered supremely exotic.

Anyway, while reading Russ's articles again and contemplating where I'd fit such a thing into my long-term roadmap, a noticed an offhand reference to this gem of an idea. It's not a total game-changer, in that most of the times I've hit this kind of problem no sparse representation was feasible, but now I do find myself contemplating ways of trying this out.

As with so many great ideas, as Russ points out it's not really new at all; there are hints of it being known back at least as far as 1974. Some really neat ideas like this languish in obscurity for a long time; some of my favourite CS papers took a huge amount of time to become well-known like this classic paper by Richard Brent which is a simple and practical method for high-occupancy hash tables very few people ever knew about.

It's really fun trying these things out and demonstrating just how well they work - often, so well that it's truly baffling why they aren't better known. So, extra kudos to Russ not just on the Regexp front but for popularising a classic algorithm like this.

On the more practical front, just a small addendum to yesterday's note on MSBuild and Visual Studio 2010; something that had been puzzling me; for years I've been used to setting up dependent projects as I'd always done so an application which depends on a DLL gets the .LIB for the DLL automatically linked with it. However, in VS2010 it wasn't working, which was perplexing me.

It turns out that this is a consequence of a peculiar change in the build system; the old project dependency system still exists (and it's easy to encounter in the VS UI since it's in the same context menu it's always been in) but the old dependency system only deals with build order. Instead, the old-style dependencies are now handled exclusively through the "Framework and References" part of project settings, which is especially bizarre since this mainly affects C++ projects but the relevant UI is very .NET-oriented. So, there's no way this would be easy to discover, but there you have it - treat your DLL projects like .NET assembly references and you're back at the classic behaviour that's been around since at least Visual C++ 6.0 circa 1998.

Monday, September 13, 2010

A Handy Versioning Trick

As I've been setting myself up so that I can develop code at home again - something I couldn't do before because Symantec automatically owned everything I did making personal projects impossible even if I had the emotional energy left after the kinds of high pressure we were basically always under. One of the things that has involved has been setting up all the normal process stuff you really want to have in place these days for everything but the simplest work.

Fortunately there are lots of good tools around to start using now; for source control there's Mercurial which I've been happily using for a while along with BitBucket as a hosted repository which allows private storage (and has reasonable prices) and comes with the usual accoutrements like a wiki and bug-tracker. Joel Spolsky's Kiln+FogBugz is also excellent but the pricing bump from the free tier to paid is a little steep by comparison (which isn't a complaint, it's worth it) and I may use that for side stuff too - the main drawback there is that they don't appear to have OpenID yet.

Building code is where things get more interesting. Thanks to some sterling work by Anuradha Dissanayake who set up a fantastic CM system for Ghost, I've long been a fan of continuous build/continuous integration systems and there's some good automation out there. It's amusing how almost all the main CI tools you can get are Java-based, but fortunately here since most of what they are doing is coordination work - polling the version-control system for work and launching scripts - this is not the problem it would normally be. In fact, they are one of the few kinds of thing which really benefit from Java's famously over-sold "run anywhere" because they don't have classic UIs, aren't innately very complex, but can derive a lot of benefit from dynamic class loading for extensions.

On top of that, some of the CI systems are actually pretty good. Hudson and Teamcity both seem up to the job - I'm going with Hudson to start because TeamCity's supposed great feature is IDE integration which is actually pretty useless to me, and if I ever need to get into writing any Hudson extensions for myself I can. Although really, the only thing it doesn't have right out of the box is support for Windows slave instances on Amazon EC2, only UNIX (well, OpenSolaris or Linux) slave instances, but that's hardly a deal-breaker.

However, one interesting thing I've been looking at today is the irritation of managing build numbers; most of the CI systems have something-or-other to try and do for this, and there are all kinds of obscure work-arounds to install .NET extensions to get build numbers into MSBuild, but it's certainly not exactly seamless and it's nice if developer builds in Visual Studio (even plain old Express) can do this too.

Turns out it's actually pretty easy nowadays, thanks to some long overdue improvements in Visual Studio 2010. The old project and solution build system for C++ code was, frankly, hideous, and so part of the solution has been that Visual Studio now uses the same unified build tool for C++ as has been used in the .NET world, MSBuild. And fortunately, to go with that MSBuild has been helped a lot with the .NET 4.0 release that came with VS2010 in terms of filling in a lot of the basic missing features.

Probably the single neatest thing in MSBuild 4.0 is property functions, which allow build scripts to call into arbitrary parts of the .NET framework and get results as strings which can be used to parameterize the build actions, without having to go to the tedium of writing and building .NET assemblies to extend the build system. With these plus the general improvements to the built-in MSBuild "Tasks" elements, a regular vcxproj file (typically an otherwise empty project of "Utility" type) in Visual Studio can contain something like this inside the outermost <Project> element:

<Target Name="Build">
  <MakeDir Directories="$(OutDir)" />
  <WriteLinesToFile File="$(Name)" Lines="#define VER_MAJOR $(Major)" Overwrite="True" />
  <WriteLinesToFile File="$(Name)" Lines="#define VER_MINOR $(Minor)" Overwrite="False" />
  <WriteLinesToFile File="$(Name)" Lines="#define VER_BUILD $(Year)$(Day)" Overwrite="False" />
  <WriteLinesToFile File="$(Name)" Lines="#define VER_REV ;  $(Time)" Overwrite="False" />

This is just the simplest kind of example, using hours and minutes as the last number part; one of the tricks to watch for is that each component of the total version number is 16-bit and so limited to 65535. That's why I used the day-of-year after the year (so this won't overflow until 2065) and why another popular build number format is to use half the number of seconds elapsed in the day instead of hours or minutes (unfortunately there are slightly too many seconds per day to use it raw, so we need to divide it down a little):


In practice the hours-and-minutes time is more than good enough, since if you're doing your builds through an automated system they aren't going to be scheduled more than one per minute anyway, and the result is slightly easier to interpret. You can also choose to use "Now" instead of "UtcNow" in the example above for simplicity if you aren't particularly worried about the effects of timezones.

If you do that, then there are some nice benefits: Hudson has a plug-in which will post build results to a Google Calendar, so it's simplicity itself to correlate any given binary with the build that produced it just by looking it up in a browser. Of course that's not particularly hard with UTC or seconds-per-day either, but if you're a one-man-band it's your choice to make it easier on yourself.

Note that by overriding the "Build" target in the XML inserted into the vcxproj file, we don't get the normal inherited behaviour for the "Build" target (which comes from some of the imported boilerplate) of doing minimal builds, so this action always runs which is what we want. In a minimal vcxproj file you will still want to have some of the regular IDE crud present just to stop the IDE property editors going haywire, and one advantage of that is that you will get properties like $(OutDir) and friends set the right way if you use property sheets to ensure those are consistent across all your projects (as you definitely should).

Another thing to consider is that this is just one way of using the above data; you can also pick it up in other MSBuild projects and tasks in your overall build for e.g. publishing the build data to a version-dependent location and all kinds of other steps, or write it out in other formats for other tools (the above example is just the first thing I did because I happen to work mostly in C++, the language which has only just joined the MSBuild family in Visual Studio).

And now with this in place I can start to get on with some simple coding.

Friday, September 10, 2010

The Magic of Differential Pricing

It's really odd to watch the Amazon store at work for Kindle books, even free ones. As I'd previously noted, there basically aren't any free Kindle books. Except, there are, but just not for me.

An odd quirk of the Amazon storefront is that the first time I load it up on a machine after a couple of days, I get to see Kindle books based on what I last viewed or in various other sections of the page, all with alluring prices - and in one notable case, it did this for an item I remembered the price of - USD$6.95 (now USD$5.95 when I checked it again for this article) for the otherwise freely downloadable Romance of the Three Kingdoms.

Except, this time it was showing me the price as USD$3.95 ... so I clicked on the item, and hey presto! the price suddenly jumps by USD$2.00 - and if I refresh the Amazon landing page, suddenly the prices of half the Kindle books (most notably the "free" ones) change to reflect this.

It's very repeatable, and very frustrating, since it seems to only happen for Kindle books and not for anything else I've ever looked at on the Amazon store; Kindle books are "special" in several ways to the Amazon storefront, partly because of the bizarre regional availability (so the store tries very hard to pretend books I can't buy don't exist), but also because they seem to be the only thing to which Amazon are applying regionally differentiated pricing.

Basically, there seems to be a bug in the storefront which isn't picking up on my geographical location at first visit, so I'm given a tantalizing glimpse of the pricing US customers see, just as I sometimes get a glimpse of Kindle editions I can't buy. However, then it adds the "regional surcharge" to the browser session if I click on an item and thereafter the storefront only shows me the regionally adjusted prices.

The really interesting thing is that when this happens I can use "open in a new tab" to check all kinds of things, and it appears thus far that this hidden regional surcharge doesn't exist for full-priced Kindle e-texts. Presumably, for e-texts over a certain publisher price Amazon are happy just getting their cut; it's only for ones that fall under some mysterious threshold price to which a crude surcharge is applied.

Ultimately, I could live with a small regional surcharge, because I'm aware of what Amazon have sunk into their incredible infrastructure where small regional price differentials do exist. But what really bites is the scale of the discrepancy; it's 4 cents per Gigabyte difference in S3 pricing for AsiaPac, but being charged two dollars for a megabyte is just wrong, given that even the dastardly local telcos don't charge that for 3G data (and even the NZ$1/Mb rate is just to soak the unwary, it's easy to avoid).

So, I've almost finished Accelerando and the Kindle has really proven itself (barring some odd formatting in the book, which may be an artifact of the preparation - 1023 was rendered without superscripts as 1023, the kind of thing that makes a difference in a book using Big Numbers). It's really a great device, and the notion that in 5-10 years something like it could be almost ubiquitous is pretty exciting, although I imagine the textbook publishers will have a much harder time adapting their business models to e-text than mass-market fiction/non-fiction publishers are managing now (where although it's a bumpy ride, it's at least slightly less disruptive).

Thursday, September 9, 2010

Through the Looking Glass

Or possibly the most bizarre road trip ever: Jeffrey Goldberg goes to Cuba and has a series of astonishing conversations with Fidel Castro and (Che Guevara's daughter makes a cameo appearance).

To pad out the post, The Economist points at OECD data on graduates in unskilled jobs, which as one commenter notes is interesting when compared to this OECD press release on expanding tertiary education. Meanwhile on the Schumpeter column, worries about the U.S. University system (although I'm not position to critically evaluate his argument, the decline of science and engineering in western-country universities generally has concerned me for a while, as young people demonstrate by their choices how the status of scientists and engineers has fallen).

Also via The Economist's Free Exchange blog, a thoughtful article from Martin Wolf of the Financial Times on the role of the state in Democratic societies which has an enormous amount of interesting comment arising from it.

Tuesday, September 7, 2010

The Race To The Bottom

Although it's been fascinating to watch how my particular craft has become demeaned and devalued over the years, it's good to be reminded that others have it worse.

The above is the last in a series run on the Dish while its normal editor is away, where readers were asked to write in and explain popular misconceptions about their job - it's been quite informative.

Monday, September 6, 2010

Atlanta Motor Speedway

So, I've just caught up on this weekend's NASCAR race at Atlanta. Man, mixed emotions; but that's the epic quality of NASCAR and why for me it is basically the ultimate in professional motorsport and an incredible sport, with great athletes. An incredible race, the last 100 laps were just gripping.

The mixed part of those feelings come mainly from seeing Mark Martin's chance of making the Chase evaporate, because I am *such* a Mark Martin fan. Last year's thrilling finale to the Cup Chase when Mark yet again came in runner-up - the greatest driver never to be the champion - was an incredible achievement and as long as he's there I'll be watching.

Man, I could rave for hours about NASCAR; this year's season and the drivers and the drama and the wing versus spoiler and aero push and track position and clean air versus tyres.... wow. And that's before getting into the personalities and pageantry.

There is much to dislike about U.S. popular culture, and U.S. politics is while not quite the most corrupt on the earth is still mostly despicable, but let me say as some other intellectual have that I have no hesitation in celebrating NASCAR. I may have little in the way of shared cultural heritage with its main target audience, but man do I love it.

Saturday, September 4, 2010

Romance of the Three Kingdoms on Kindle

I have first here to confess that I learned of the Three Kingdoms through playing one of the earliest editions of the computer game version by Koei, specifically Romance of the Three Kingdoms on the Amiga (I also greatly enjoyed Nobunaga's Ambition, published around the same time).

Ever since then, reading a translation of the original book has been on my to-do list, and lo and behold there's a free Edition available. From a quick look it doesn't have any of the extra material or historical notes of the acclaimed Moss Roberts Translation, so there's real value in still going for one of those, but given the massive size of the book I suspect I'm much more likely to succeed in reading it as an E-text. Note that from Silk Pagoda you also get an HTML version.

Note that the Silk Pagoda edition linked above is also on the Amazon store as The Romance of Three Kingdoms, where you might note a small price differential. This can really only be ascribed to the way Amazon's burying of the 3G data charges - note that most "free" e-texts on Amazon are about USD$2, but ROTK is a monster book of about 2Mb in a compressed .PRC MobiPocket form - and WiFi Kindle owners can't escape that cunningly hidden 3G surcharge even though we get none of the benefit.

Probably the main thing I expect to lose out on with this is syncing, since I have a feeling it would be nice to have the option of reading this on PC from time to time. However, the Kindle for PC reader application only works with texts you've purchased and paid for via Amazon (which is disappointing but hardly surprising - as with the 3G data surcharges, it makes rather less sense for WiFi-only Kindles but everything in Kindle world is still oriented around 3G-only devices). I'm also experimenting with MobiPocket (which has RSS feed conversion and other interesting capabilities - it has PDF conversion but unfortunately doesn't handle the PDFs I have which are from old TeX-formatted->PS->PDF scientific papers) and thus far that seems to work just fine including syncing the read points with the Kindle.

By the way, ROTK isn't the only one of those classics I'm hoping to take a stab at: thanks to the excellent Dan Simmons books Ilium and Olympos I've got a hankering to at least take a crack at The Iliad and Odyssey. Eventually, anyway - I do feel a bit of an ignoramus for having never tried before.

Friday, September 3, 2010

Divine Discontent

Although I'm not normally big on link dumps, I may as well do one here. No comment is offered, except that which is implicit in the offering of these things in conjunction.

Tylen Cowen on interestingness and happiness. A fine example of Marginal Revolution; put it in your feed reader. Economics and happiness is a fascinating topic of research, and Tyler's postings are nicely provocative (in the best way, provoking thought). Having read the blog for many years, I am sorely tempted by his book but what holds me back is the question of how much, as a regular reader of the blog, would be genuinely new to me. Probably a lot, but it's hard to be certain.

Cal Newport posts again, as always on the theme of developing an exceptional life. From his point of view, I've probably followed most of his recommendations: the difference is that I've never really understood my place in the world, and living in a tiny part of the world it has never been easy to understand whether it's a matter of being an outlier in a small sample population or ... well, the impact of large numbers generally.

This morning on Radio New Zealand was a rare interview with a Waikato University researcher on gifted children and high-achieving adults on the Nine To Noon magazine programme; the current presenter Kathryn Ryan does a good job; although normally a poor science interviewer (if only because she's inevitably compared to the excellent broadcaster Kim Hill), she does a fine job here by just giving the interviewer the right kind of leads and just letting him speak to them. Get the MP3.

Kindle 3 Second Impressions

After having a little more time to settle in with the device, I'm getting more comfortable with what it can and can't do, at least with the current firmware. But more importantly I've started actually reading on it, and that's an experience I can't fault.

But first, a warning: after discovering Feedbooks and in particular the essential Kindle Download Guide, which is a MobiPocket-format e-book with hyperlinks to all the other Mobipocket free content they have, things became... strange.

After downloading a couple of texts (because there's some great old stuff there including many beloved childhood classics: Edgar Rice Burroughs! H.P. Lovecraft! E.E. "Doc" Smith!), suddenly the Kindle froze and rebooted... and then a couple more and it rebooted again... and if I attempted to add a book to a collection, instant repeatable reboot. Try and read any of what I downloaded, instant reboot.

The only way to resolve this was to remove the Download Guide e-book completely. I can't say why, but that thing is evidently anathema to the Kindle 3.0 firmware - even though reading the guide itself was fine, simple having it on the device rendered the device unstable. Remove that, however, and suddenly all was well.

And so I began to settle in, beginning with Accelerando, which I've mostly been reading in bed at night or in the early morning. And here the Kindle ... well, it Just Works(tm). The display is clear and easy to read in just about any lighting conditions, enough so that I wouldn't bother springing for a case with light. I do, however, seriously regret not getting a plain leather case when I ordered the Kindle, because the shipping on the case alone is quite expensive (Amazon by not actually calling the new Kindle something distinctive haven't made it easy to know what case fits what device, so I have no idea whether anything I can order in New Zealand would suit it or not).

The next-page/back buttons on the side of the device work well so I've never had a problem getting comfortable and holding the Kindle with either hand. It hasn't yet quite "disappeared" for me yet, to the point where I've forgotten that it's there, but when reading it I'm definitely able to get into the flow of reading just as well as if it was a print book.

Now, things that don't work so well: well, the Webkit-based browser built into it is a mixed bag. The most baffling thing of all about it is that you get no control at all over font size in pages, even though regular desktop browsers all provide this and so does the Kindle itself for e-texts. This means that you're always forced into using one of the zoomed-in views to read the teeny-tiny text which is clunky and just... ugh, especially for pages with navigation sidebars.

However, pages designed for mobile phones are mostly a joy on the Kindle. Plain Gmail has the teeny-tiny-font problems and runs very slowly (and has crashed the browser outright at least once). Mobile Gmail is just perfect, though; it was good on a Symbian S60 phone and it's equally good on the Kindle. The stand-out exception where neither works well is Google Reader, since the mobile version isn't good even on phones where it uses numeric keys for its limited interactions, which means that the Kindle's alpha-oriented keyboard makes for an even worse experience. Not that I'd want Google Reader except that a quality RSS reading experience is something that the Kindle is crying out for and is unavailable outside North America.

Apropos of which, the regional availability is just driving me nuts, and there doesn't seem to be any way around it. Although the Steam Game Service has a lot of the same region problems (including Australia/NZ being charged over 3 times the USD price for some games), at least there gifting is an option, or would be if I knew anyone in the U.S. willing to help. There's no gifting for the Kindle, however, even for items where the print version can be gifted.

So, while I'd like to get Illuminatus! again (and I could briefly find it on my Kindle before it fully region-locked itself, which is why it's on my Amazon Wish List), I can't. I'd like to (re)read some other books like some Thomas Pynchon and Infinite Jest but even though for some of those I know Kindle editions exist, Amazon won't even show me the items existing at all in e-book format unless I find the URLs for them externally via Google.

So, most of the content I actually want for the device I simply can't buy. Fortunately there are other sources of e-books I can use, and I can see that the Kindle definitely will still increase my reading volume. If only it wasn't for the bizarre region-locking it would be an unhesitating recommendation, but as it is I'm not unhappy and definitely looking forward to a Wi-Fi DX joining the line-up. It also won't be supplanting my print subscription to The Economist as the Kindle version is remarkably expensive and doesn't give access to the Web version, something that comes with a print subscription; and although I'd found that out already while looking for content after ordering the Kindle, now that I actually have the device it's vanished off Amazon's web site for me now - it's evidently region-locked, and thus hidden by the "doesn't exist" filter. /sigh

Update: Oh yeah, one small thing I didn't pick up on before buying the Kindle is that it uses a Micro-USB B connector, like the slimline Nokia phones (e.g. the E71) do. Which is fine and dandy, no problem, except that generic Micro-USB B cables are stupidly hard to get hold of at the moment, at least in New Zealand - no-one carries them retail (except as an overpriced Nokia accessory) and there's only one vendor on TradeMe selling cables. That problem will hopefully go away over time but right now I have no spares.

Thursday, September 2, 2010

Object Orientation 101 Part 1

Since Rad is starting to look at Object-Oriented programming, a few quick notes to hopefully clarify a few things that he (rightly) expressed uncertainty about in response to his recent post talking about his first readings on the subject.

Before I begin, a disclaimer: there is a special kind of object-oriented programming based on a concept called delegation instead of inheritance, but for Rad's benefit I'm pretending that doesn't exist, because what he's reading doesn't address that yet either. But I'm sure Rad will look at Javascript one day and then we can look at that.

First things first: a textbook author who writes that Object-Oriented programming
[is] the most powerful programming model yet devised
is hard to take seriously, because it's arrant nonsense. OOP is certainly very popular (and set to remain so by being the only model taught in most universities), and also very useful, given that a great deal of software is written to model real-world things, which is what object-orientation is essentially about, and why object-oriented programming is in fact pretty easy to learn.

However, in terms of expressive power it cannot be compared to the functional languages, which both derive and ultimately pay for their immense versatility and precise expression because their core concepts don't match to simple real-world reasoning by analogy. OOP works by making it easy to create program structures which are direct analogues of real-world things, and indeed it's generally presented that way. However, reasoning by analogy is also horribly imprecise, so while OOP is comparatively "easy to learn" because it's about drawing simple analogies, that learning process is in fact pretty hit-and-miss because people can and usually do draw incorrect inferences from analogies.

This is why rigorous teaching programs start with something like Scheme; it's small, and about as simple as a functional-type language can be, but it's very precisely specified - so when you learn it, you don't end up with misconceptions that have to later be unlearned - and it's built on concepts that are universally applicable.

Most college programs use Java precisely because it's object-oriented, and thus its' entire logic can be demonstrated by analogy, which by "happy" coincidence is the mode of reasoning it's suited to address. Since making analogies is very simple, people get a big sense of power very quickly, even though what they are learning is very imprecise and that imprecision can lead people quite far astray.

The main thing about object-oriented programming is that it's got things (i.e., nouns) and actions (i.e., verbs), but in OOP it is the nouns that are in charge all the time. Verbs are applied to nouns, and have no meaning by themselves.

Things, just like nouns, come in several forms; there are nouns which describe categories of things (for example, "cats"), and nouns which describe particular actual things (for example, "my car").

All the world of nouns in OOP can be dealt with by one of a type of categorical statement, of these forms:
  1. An X is a kind of Y, e.g. a Cow is a kind of Animal.
  2. An X is a Y, e.g. Bessie is a Cow.
  3. An X has a Y called Z, e.g. a Cow has a Tail called "the tail".
This leads us to the things we have in OOP programs, since everything gets tied to a noun and they are all related to each other in one of these ways. Many languages also have a fourth kind of statement which is of the form X "uses" Y as opposed to straight "has a", but that's a detail we can put off to start with.

Classes are where we make statements of the first kind, to start to describe a category by asserting what it is kind of like (or analogous to), which we can then elaborate by describing it in more detail using the third kind of statement, where we can describe the attributes of the thing. And lastly, almost as an afterthought, we can tack on verbs:
  • An X can do a verb, e.g. a Cow can Moo.
When it comes to the heavy lifting of the verbs, the principal aim in object-oriented programming is to define those as lists of statements like object verb other nouns.

Congratulations! That's basically the entire conceptual foundation of OOP, and if you are happy expressing everything like this (i.e., talking like a child with a picture book) then you could well have a promising career as an Enterprise Java Developer in front of you.

So, to address Rad's (implied) questions:
In PHP, characteristics are called FIELDS, and behaviors are called METHODS (.. F&M). Okay, methods I can see, but not sure why they chose the term 'fields'.
Actually, they aren't called fields. The term used by convention in most programming language is "properties", and a trick to remember this is by recalling it's all about saying has a. If you have something you own it, therefore it's your property.

The term "field" isn't used in much programming language literature (and isn't used that way in PHP either), because it comes from a specific application area: filling in forms, which is of course a lot of what very simple programming is about. A field is an area in a form you can fill out; basically, calling a "property" a "field" is something you might do if the code you were writing was nothing more than conveying form fields.

So, it's a terminology thing which you'll only encounter in the context of PHP which is oriented around HTML, and thus where simple applications receives most of its data via forms. The other place the term "field" used to be common was in COBOL applications (which similarly was almost exclusively about processing forms) where it was actually used in the language: data in COBOL was organized into structures called records and the elements of a record were called fields. Later on, SQL standardized the terminology for the quite versatile (not just for forms) representation of data it supported to "rows" and "columns" although few languages incorporated SQL directly into themselves so there were separate names for persistent data in a database on disk and data which is "live" in memory in a program.

Most of the rest of the things such as abstraction and modularity as they exist in OOP aren't actually fundamental. Rather, they are best considered as consequences of the noun focus. If you're restricted to describing nouns in the prescribed way, using only the simple kinds of statements you're allowed, then modularity is enforced by fiat. Similarly, abstraction is a consequence of the fact that verbs can be attached to objects which have more specialised "kinds of" described later.

Modularity and abstraction are just as important in other kinds of programming language, but they arise in different ways because the other kinds of language have a different way of composing things which aren't just about noun verb. So by all means treat them as virtues, but bear in mind that they aren't particularly unique to OOP at all.

By the way, the whole noun-verb obsession is brilliantly lampooned by Steve Yegge in Execution in the Kingdom of Nouns which shows how awkward and stilted it becomes trying to reason using only sentences at the level of grammatical complexity of "See Spot Run", which is actually the goal of much OOP programming.


A short while ago I dropped a line to the always thoughtful Rad to let him know of my changed circumstances since he might be interested in talking about some tech things, and he sent me back this interesting link on the ageist culture in technology companies.

There are a fair number of comments posted, showing near-unanimous agreement that a) this particular phenomenon is very real (and I don't doubt it for a moment), and that b) the suggestion that we more seasoned folk should enter management or sales-type roles is both insulting and impractical (which is certainly how I find it).

Like a number of those commenters, I'm now in my mid-40's and as I'm supposed to be writing a CV or resume or some such thing (having never had to bother before) I've found myself puzzled by the problem of explaining all the things I've done over the past 30 years in terms a modern employer might understand, given that most of them are either going to go well over their heads, or be thought of as obsolete (which is of course utter nonsense - having written a TCP/IP stack and the OS it lived in isn't diminished by the fact that the 68000 processor it targeted is now a museum piece, since the understanding and insight that comes from having that experience never goes away).

This is one of the big differences between merely having read of something and having done it: a potted summary, even a reasonably deep one, can really only cover the topmost layer of engineering decisions that go into any substantial piece of technology. However, when you're creating an entire large ecosystem from scratch without relying on others, there are a myriad small details that you have to get right, many of which are actually related to deep engineering problems you'll encounter again and again.

The same thing also applies in spades to the "snap-together" programming culture which is utterly dominant today. Modern developers appear to simply be uninterested in learning the inner workings of the artifacts they are throwing together, and are equally unconcerned that the results of their labour are huge, slow, and inflexible.

There are a lot of articles I intend to write on the beauty of the small - an engineering virtue on many levels from the practical to the aesthetic - but one of the best things about there being no mysteries is that you can fix anything. In the computer systems I use, there is simply nothing that I couldn't write for myself; I've done text editors, compilers, interpreters, GUI widgets, OS kernels, all from the ground up. Not all of them were as pretty as you see today, mind you, being constrained by the hardware at the time and by the fact modern commercial systems have been polished to a fine sheen by the labour of thousands, but in functional terms there are no important secrets at any level of the software stack.

That understanding confers some real power, because as a consequence you are never constrained at all to colour within the lines. If you have a problem, it's possible to take things apart and fix them. It's one thing to make a hard-headed economic decision to build or buy, another entirely to be unable to build software to get from A to B unless someone has already done it before.

Martin Gardner

The blame for me becoming an insufferable nuisance to the teachers at Glenfield College in 1980 can't be laid entirely at the feet of the Texas Instruments corporation. The setup for that was laid down rather earlier, thanks to the writing of one of the world's great educators: Martin Gardner, who was to mathematics what the great Sir David Attenborough is to naturalists.

When my parents divorced in 1976 I chose to live with my father, who continued to work, thus leaving me a lot of time alone. A great deal of that free time was spent with a pile of back issues of Scientific American which he had subscribed to in the mid-60s when he was a young man - before the unplanned arrival of myself thoroughly derailed his life in 1967.

Scientific American was a great magazine for a 10-year-old boy; articles written by eminent scientists, but carefully targeting a reading age of (I'd guess) about 14 it did a fantastic job of explaining science clearly, but without sacrificing too much to make it truly "popular" in the oversimplified way that is more common nowadays (with, say, the Discovery Channel).

And as great as every issue was, the very best part of all lurked at the back (just before "The Amateur Scientist" column), in the form of the "Mathematical Games" column in which Martin Gardner regularly served up an essay devoted to recreational - but still quite serious - aspects of mathematics.

From him I learned about counting on your fingers in Binary - spending hours counting up to 1023 and back down - and how the so-called "Russian Peasant" multiplication worked, which was a curiosity for decimal but was exactly how computer arithmetic worked. Or studying the Towers of Hanoi puzzle, the solution to which can again be understood based on the properties of numbers when represented as binary.

Of course those were the more serious articles but Martin Gardner's columns also included plenty of things which were nothing but sheer fun, and probably the greatest of these was Flexagons, from simple flats to the wonderful solid Tetrahexaflexagons. Just as I was getting fascinated by these a truly wonderful book was published, which my Father spent a horrendous amount of money to buy for me: M. C. Escher Kaleidocycles, because flexagons and the magnificent tiled drawings of Maurits Escher go together perfectly.

With that kind of early start, it's hard not to become math-obsessed, and since there weren't computers around to be used back in the late 70's I was absolutely primed to become completely fascinated by them when one finally came within reach. And indeed, that's probably why jumping into writing assembly language appeared to come so easily, because I'd years before then absorbed most of the fundamental prerequisites.

These days perhaps it would be harder to get hooked on maths quite the same way, with so many distractions around - not just mobile telephones and the internet were yet to appear, but back then there was precious little worth watching on New Zealand's two television channels either (although to be fair we did get to see some superb BBC programming, from the classic Horizon documentaries to James Burke's incredible Connections TV series.

Today, all that raw information (and much more) is out there and in principle readily available, but it's mostly drowned out by a cacophony of ephemera. I can only hope that youngsters today are getting steered at this kind of material, and can draw from it a sense of the richness of the world and the power of science, and thus choose to embark on a career in science or engineering out of real passion.

Wednesday, September 1, 2010

I Am A Free Man

Because it's just so damn cool (and yes I have the DVD set and it's as good as I remember from my youth), in celebration of my status today as an independent:

The Prisoner original theme by none other than Ron Grainer, who famously composed the haunting Doctor Who Theme music performed by Delia Derbyshire of the BBC Radiophonic Workshop.

The video linked above is from an obscure (excellent, but long out of print) set of remixes of classic British TV shows that was called "Powerthemes 90".