Irate Nate's weblog

re-reading robert jordan

Sat, 08 May 2010 14:44:00 CDT

I was in an airport bookstore last week and picked up one of the newer Wheel of Time books. I opened it to a random page and read:

Nynaeve crossed her arms under her breasts and sniffed.

Yes, well, shocking. I stopped reading after Winter's Heart, but I have found something that almost makes me want to start reading them again. It's Leigh Butler's Wheel of Time Re-Read, wherein she, well, re-reads Jordan's epic, complete with varying amounts of breezy, pop-culture commentary on each bit. My favorite bit so far, having only “re-read” The Shadow Rising, was an musing on Mat as the classic American hero.

She's in the middle of The Path of Daggers right now. I find myself strongly tempted to re-read all of the novels--and read the new ones--through the lens of this blog. I haven't looked at the inspiration for this project, the Lord of the Rings re-read, but I suspect that one would be equally compelling.

analogical thinking

Wed, 05 May 2010 08:26:00 CDT

We were riding around in the car this past weekend and Becca was complaining that Ally was so enamored with everything she did. Tricia was attempting to explain that this is because Ally thinks Becca is so cool and someone to look up to. Tricia told Becca, “She thinks you're the cat's meow!”

Becca thought about that for a second, and Ally piped up with, “You're the dog's woof!”

Hooray! My child is thinking analogically! This feels like a big step. We compliment Ally on her observation.

To which Ally responds, “You're the pooping dog's poop!”

OK, still thinking analogically, but...

functional language proponents

Wed, 28 Apr 2010 15:55:00 CDT

Seen on an LtU thread:

Actually, I think one of the reasons FP proponents are more convinced of the superiority of functional programming than is necessarily warranted is due to their high degree of applicability to writing compilers, where almost all I/O is input to a function at one end and output at the other end, where the input and output are defined in specifications, where there is an unambiguously correct mapping from source to target, where the data is inherently tree structured, where the structure of the computation involves recursive pattern matching and transformation, where there is virtually no dynamic code loading necessary, etc. It's a match made in heaven; and if all you do is geek on languages and compilers - especially in an academic context - of course it's self-evident that functional programming is superior to everything else.

How true, how true.

pwnat

Tue, 27 Apr 2010 15:21:00 CDT

Cool tool of the day: pwnat. From the description:

pwnat, pronounced “poe-nat”, is a tool that allows any number of clients behind NATs to communicate with a server behind a separate NAT with *no* port forwarding and *no* DMZ setup on any routers in order to directly communicate with each other. The server does not need to know anything about the clients trying to connect.
Simply put, this is a proxy server that works behind a NAT, even when the client is behind a NAT, without any 3rd party.
There is no middle man, no proxy, no 3rd party, no UPnP/STUN/ICE required, no spoofing, and no DNS tricks.

The above page also features a description of how it works. Very cool.

commentary on peter pan

Sun, 18 Apr 2010 21:23:00 CDT

I have meant to blog about Sympathy for Hook: Toward a Christening of Peter Pan ever since I read it several months ago. Even re-reading it today, it stirred up the same mix of emotion (“cool!”, melancholy, thoughtfulness) that it did originally. To whet your appetite:

Peter Pan is about the death of children, long a daunting challenge to both simple faith and learned soteriology. It is, however, equally about deadly parental neglect--about not wanting or loving children. It is all too timely for our era of selfishly infertile and casually feticidal adults. Perhaps we miss Peter Pan's central themes in part because child death post partum has become far rarer than it was a century ago. We also may miss them, at least in part, because we dismiss as mere caricature the story's main adult character, Captain Hook: We fail to recall and give due weight to the tradition, which goes back to the play's first staging, under Barrie's direction, that Hook be played by the same actor who plays Mr. Darling, a flawed father. Mr. Darling and Hook are, in fact, the same character, operating respectively in this world and in Barrie's limbus puerorum, Neverland. Although, as G.B. Shaw noted, Peter Pan “is ostensibly a holiday entertainment for children but really a play for grown-up people,” appreciation of its burden for grown-ups requires attention to its main adult character. Appreciation of Peter Pan requires sympathy for Hook.

Recommended reading, particularly for parents and doubly so for fathers.

Tricia and I started reading Peter Pan this past summer and were somewhat taken aback at the tone and violence in the book (thanks, Disney!). But this essay makes me think that completing the book will definitely be worthwhile. Now to find the road trip to read to each other...

financial regulation

Mon, 12 Apr 2010 15:36:00 CDT

Eliot Spitzer on the need for government financial regulation:

When I was Attorney General of New York, we investigated what I refer to as the "analyst cases." The cases were about investment banking. To understand them, all you need to know about investment banking is that the business has two sides: you have supposed analysts, who recommend stocks for investors to buy, and you have underwriters, who sell IPOs, secondary offerings, and other things that raise capital for the companies. We argued that there is an inherent conflict of interest when you have people in the same business who are both recommending a stock to retail investors--saying, "this is a great stock, buy it"--and doing the underwriting. The more successful the underwriting, the more fees it will generate, and the more favorable the analysis, the more likely the underwriting will succeed.
This is clearly a conflict of interest. In fact, Jack Grubman, the telecom analyst banned from the securities industry in 2002, brilliantly encapsulated this whole era: what used to be viewed as a conflict of interest, he said, is now viewed as "synergy." Think about that. What used to be viewed as a conflict of interest--hence dangerous to people, something to warn them about--is turned into something that creates value. This was the way we masked the problem. In the analyst cases, we didn't discover the conflict of interest: everyone understood it and had accepted it for a long time. But at the attorney general's office, we called it what it was...
The lawyers for Merrill came into my office. Now, when you're a white-collar defense lawyer, you argue in the alternative. First you say the emails were taken out of context. Then you say: "you don't fully understand them." Then you say, "no, they were fabricated." Then you say the person doesn't really speak on behalf of the company. And then you check the limits on your insurance. I was a white-collar defense attorney for a number of years, so I know the drill.
They didn't make any of those arguments. They came to me and said, in essence, "Eliot, you're right. Absolutely right about the conflicts, the tensions, the problems. But we are not as bad as our competitors." That was their defense to the charge that they were defrauding their customers and the marketplace. When I asked to hear more about it, they said, "let me tell you about Goldman and Citi." I've turned defendants over the years, but these guys were the easiest flip I've ever done.
There is a very important point here: the people at Merrill Lynch understood that their business model was problematic. They understood that there was something wrong with recommending stocks that they didn't think were any good. But they had to choose between ethics and profits, and they made the choice that harmed individuals and undercut the integrity of the market. And then they said it wasn't up to them to enforce the rules of transparency. Somebody else should do it.

greenspun makefiles

Fri, 09 Apr 2010 22:23:00 CDT

Greenspun's Tenth Rule:

Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.

Can we apply that to Makefiles, too? This is what I wrote tonight:

find-hwm-sources = $(subst $(srcdir)/arch/$(ARCH_DIR)/,,\
                            $(wildcard $(HWM_DIR)/$(1)/*.$(2)))
hwm-c-objfile-name = $(subst /,-,$(1:%.c=%.o))
hwm-S-objfile-name = $(subst /,-,$(1:%.S=%.o))
hwm-sources-varname = HWM-$(1)-$(2)
hwm-objects-varname = HWM-$(1)-$(2)-objects

define hwm-object-template
$(call hwm-sources-varname,$(1),$(2)) := $(call find-hwm-sources,$(1),$(2))
$(call hwm-objects-varname,$(1),$(2)) := \
  $$(foreach o,$$(value $$(call hwm-sources-varname,$(1),$(2))),\
     $$(call hwm-$(2)-objfile-name,$$(o)))
$$(value $$(call hwm-objects-varname,$(1),$(2))) \
  : hwm-$(1)-%.o : hwm/$(1)/%.$(2) $(if $(subst S,,$(2)),,assym.s)
	$$(CC) $$(if $$(subst S,,$(2)),,-D_ASM -include assym.s) $$(ALL_CFLAGS) $$< -c -o $$@
HWM_ALL_OBJ_$(2) += $$(value $$(call hwm-objects-varname,$(1),$(2)))
endef

HWM_C_SUBDIRS := timer rhymer nickel dimer dcache icache ucache wecache zing
HWM_S_SUBDIRS := base
$(foreach s,$(HWM_C_SUBDIRS),$(eval $(call hwm-object-template,$(s),c)))
$(foreach s,$(HWM_S_SUBDIRS),$(eval $(call hwm-object-template,$(s),S)))

(newlines and backslashes inserted for sanity in word-wrapping; had they actually been there, make probably would have choked.)

Just think how much easier that would have been in a language with real macros. Of course, it's also possible that there's a much simpler way to do what I wanted...

looping issues

Tue, 06 Apr 2010 23:37:00 CDT

I really enjoy using loop. Collapsing a surrounding let in a with binding clause or lifting an if expression into a conditional execution clause brings me happiness. One of the things I dislike about it, though, is collecting with nested loops. In highly repetitive macroized code, for instance, it's not unusual to want to do something like:

(loop for ...
  do (loop for ...
        collect ...))

Except that doesn't work, because you're only collecting in the inner loop, not in the outer loop where you want. There are ways around this; you can do:

(loop with list = nil
  for ...
  do (loop for ...
       do (push ... list))
  finally (return (nreverse list)))

Or, if you like using loop's features:

(loop for ...
  nconc (loop for ...
           collect ...))

I don't particularly care for either of those, though I'll use the last one if the situation arises.

Recently, though, I found a shortcut for a special case. I needed to loop over several quantities, each of which could be true or false. Extending the second workaround to a multilevel loop...well, let's not go there. However, by doing some bit-twiddling, I arrived at:

(loop for bits from 0 below (ash 1 n-quantities)
  for q0 = (logbitp 0 bits)
  for q1 = (logbitp 1 bits)
  for q2 = (logbitp 2 bits)
  ...
  collect ...)

which works well and avoids deeply nesting code. It even works if you have one quantity that iterates over a number of values that's not a power of two. Say you have three true/false things and one tri-valued thing:

(loop for bits from 0 upto #b10111
  for q0 = (logbitp 0 bits)
  for q1 = (logbitp 1 bits)
  for q2 = (logbitp 2 bits)
  for q3 = (ecase (ldb (byte 2 3) bits)
             (0 ...)
             (1 ...)
             (2 ...))
  ...)

Perhaps not overly useful, but a nice trick to have in your bag.

love my debugger

Thu, 25 Mar 2010 21:11:00 CDT

I love my debugger.

I realize that this is not the usual stance among developers. The Practice of Programming by Kernighan and Pike states in the chapter on debugging:

As a personal choice, we tend not to use debuggers beyond getting a stack trace or the value of a variable or two. One reason is that it is easy to get lost in details of complicated data structures and control flow; we find stepping through a program less productive than thinking harder and adding output statements and self-checking code at critical places. Clicking over statements takes longer than scanning the output of judiciously-placed displays. It takes less time to decide where to put print statements than to single-step to the critical section of code, even assuming we know where that is. More important, debugging statements stay with the program; debugger sessions are transient.

Now, “thinking harder” is certainly a valuable skill. Debugging kernel oopses without the running kernel is certainly a valuable skill. (Read that link; it's the sort of debugging you ought to aspire to.) But I think it's fair to say that the above paragraph advocates two approaches to debugging: thinking hard and putting in print statements.

The critical observation that's made me love my debugger in the last several months is this: a debugger is a tool for inserting print statements at runtime. Stick a breakpoint where you think things are going wrong and poke around when you stop there. Stick a breakpoint a little further along if you need to. Disable the previous one because it's not helpful. If your debugger supports it, tell it to automatically print out some information when it hits a breakpoint and/or make that breakpoint depend on some condition you think is interesting. If you're feeling daring and you think you botched an assignment somewhere, tell the debugger to modify some variable values when you hit a breakpoint and continue execution. Yes, it's all boiling down to print statements, but you're doing it at runtime.

Plus, you can do all of these things without recompiling your program. And that is a huge huge win. With a huge program to debug, even with a beefy machine for compilation, you're stuck twiddling your thumbs waiting for a compile/link cycle. Ooops, wait, you have to do another one because you screwed up a semicolon or some matter of syntax. Oh, hm, that print statement isn't giving me enough information; I need to go insert another one somewhere. Bother, that's too much information, I should set up a debug level framework for enabling these messages separately. Wait, I need some information over here to obtain more clarity; should that be debug level one or debug level two or another debug level entirely? Whoops, fix that format string mismatch with my printf...you see how this can get tedious.

I was starting down the above path earlier this week when I decided to use the debugger instead. (And in a humorous twist, I was working on GDB, so reaching for the debugger should have been the first thing on my mind!) And the experience was significantly more pleasant and took me much less time than bothering with debug levels and printfs and staring at log output.

I'm not as good as Kernighan and Pike; perhaps I don't think hard enough about where to place my debug statements. Or I make too many simple typos and require too many recompile steps. Or the programs I work on don't follow the Unix Way strictly enough and are too big. And yes, I've worked on several codebases where using a debugger was painful and print statements are a better option. But the next time you're about to write a bevy of print statements to tell you everything you need to know about what your program is doing, try using a debugger instead. I bet you will be surprised at how much better the debugging experience is.

movie recommendations

Thu, 18 Mar 2010 00:02:00 CDT

Tricia and I are planning on activating a free 1-month Netflix subscription in the next few days. Our short list of movies we'd like to rent during the subscription is:

Julie and Julia
Star Trek (the new one, obviously)
The Soloist

Anybody have any other suggestions?

book report

Fri, 05 Mar 2010 19:08:00 CDT

I've had a bit more time to read than normal these past two months because I've been traveling for work. Even though I don't feel like I've been reading a book a week (my goal), the list below suggests I'm not doing too badly.

Daemon by Daniel Suarez
Who's Afraid of Postmodernism? Taking Deriida, Lyotard, and Foucault to Church by James K.A. Smith
The Price of Spring by Daniel Abraham
The Next Reformation by Carl Raschke
Whole Earth Discipline by Stewart Brand
Sidewalks in the Kingdom by Eric O. Jacobsen
Assorted Sherlock Holmes short stories by Sir Arthur Conan Doyle
Cradle to Cradle by William McDonough and Michael Braungart
Foucault for Beginners by Lydia Alix Fillingham
Unlocking the Clubhouse: Women in Computing by Jane Margolis and Allan Fisher

My recommendations from the above list are: Daemon, which is essentially Halting State cranked to 11. The assembly line was an excellent representation of the lure behind MMOs. Whole Earth Discipline was a challenging book, especially if you lean “green.” I understand the positive side of postmodern thought much better after reading The Next Reformation; Who's Afraid of Postmodernism can serve the same purpose, but as it focuses on Foucault, Derrida, and Lyotard, it doesn't give one quite as much history or as broad an understanding as The Next Reformation.

Honorable mentions: Unlocking the Clubhouse gave me a lot to think about with raising two little girls and exposing them to technology (which I admittedly haven't done very much of beyond twiddling with iTunes and explaining how to play CDs in the PS2). I appreciated the vision that Cradle to Cradle presented; I wish all books were made out of similar material, both for the pleasantness of heft and for the eco-friendliness. The Price of Spring was just as good as I expected it to be. If you like fantasy and you haven't read Daniel Abraham's Long Price Quartet, you ought to start now.

What didn't make the list above? I tried reading One Market Under God by Thomas Frank, but I only made it about a quarter of the way through before I got sick of his smug know-it-all tone. I've mostly finished Discussion of the Method by Billy Vaughn Koen, which serves much the same purpose for the engineer-ingly bent among us as Gödel, Escher, Bach did for computer scientists. I skimmed through No Easy Answers by Allan Franklin; his descriptions of famous physics experiments were interesting, but past the first one or two, they didn't really draw me in. The description of the Meselson-Stahl experiment that proved how DNA replicated is worth a read, though.

cities and redemption

Thu, 18 Feb 2010 14:30:00 CDT

From Sidewalks in the Kingdom by Eric Jacobsen:

...This radical change in the status of the city from a place of sin and unfaithfulness to the ultimate context for our redemption raises at least one important question: What can account for such a radical shift? Why did God take what looks to be one of humanity's worst inventions and use it as the primary context for our redemption?
The answer to this question is to be found in the question itself. Our God is fundamentally a God of redemption. God takes us (and our decisions) seriously. When we make mistakes--even profound ones--God does not obliterate the scars these mistakes leave on our character and start over. Rather, God takes our mistakes and reworks them into his divine plan and transforms our scars into something beautiful. Joseph said to his brothers when they repented of selling him into slavery, “Even though you intended to do harm to me, God intended it for good.” So also it seems that God has said of our cities, “You meant them to be a form of escape from me, I used them to draw you back to me.”
And we see hints of this redemption theme all around the cities of the Bbible. Cain's desire to flee to Enoch after killing his brother later finds a redeemed expression in the cities of refuge. The fear of the residents of Babel that they might be scattered is redeemed when the Isrelites find cities in the Promised Land in which they can gather. The Hebrews long to be free from their tyrant, Pharoah, and eventually found a good king in the city of David. And finally, the Babel residents' desire to make a name for themeselves is met in Jerusalem, where God has caused his name to dwell.
Redemption shows that God's power to redeeem and restore is stronger than our ability to alienate and break down. But redemption is not always the strategy that we would chose if it were up to us. Often what we seek is a return to innocence. We want to forget about the past and start over. Psychologically, we repress painful memories. Relationally, we cut ourselves off from people who remind us our past. And culturally, we ignore our history in favor of what is new and current.
...But for those who seek to take the Bible seriously, this cannot be our strategy. We cannot hate the city when our God is using it for good. We can suport the city from a truly rural setting, but we cannot retreat to our own private gardens. That way has been closed off. We must at some level learn to take our cities seriously. Whether live, work, worship, and play in our imperfect cities, or even just for them at a distance, we need to look to our cities if we hope to catch a glimpse of what God has in store for us.

I approve of the strategy of incrementalism. Starting over can be an option, but the opportunity to do so happens so infrequently that we are danger of idolizing it. Not just in cities, but in other contexts, too.

banks asking for regulation

Fri, 22 Jan 2010 21:52:00 CDT

If banks are going to continue in their expectations that the government is going to bail them out, then the government is naturally--regardless of arguments from the left for increased regulation or from the right for steering clear of the self-correcting market--going to impose regulations to make it less likely that banks will place themselves in a position in which a bailout is necessary. Simple economics. You can debate back and forth about the popular merits of such a decision and the political maneuvering that comes with that. But the government is acting in its own self-interest in this case, plain and simple. The banks are getting exactly what they asked for.

grouchy c programmer

Fri, 15 Jan 2010 22:05:00 CDT

Whenever possible, feature macros in C should be #define'd to 0 or 1, rather than letting the definedness or lack thereof be the switch for the feature. This convention means that you can talk about the feature macro in conditions, rather than having to jump through hoops to do so. (Doing so also means that you can play clever tricks in code with said feature macros if need be...) Of course, this implies that your compiler needs to be intelligent about dead code elimination in the presence of disabled features, but what compiler isn't?

Also, to all the hotspot rockstar programmers out there who are enormously impressed with their ability to record key routines from the C library using whizbang vector instructions and cache control instructions and show massive speedups: please try to actually test your code to ensure that it has the same correctness properties as the code in the C library. Nobody cares how fast your routines are if they cause buffer overflows on real-world code, rather than the trivial benchmark you threw together.

ice skating

Fri, 01 Jan 2010 23:33:00 CDT

We took the girls ice skating yesterday. Becca liked it so much that she declared that she wanted to go back today. So we did that, with one change: I put on skates today and went out on the ice. Yesterday was mostly holding onto Ally while Becca was on the ice or vice versa. Tricia was going to send me out on my own.

“I'm not sure I've ever been out without holding onto somebody before,” I said.

“How many times have you been ice skating before?”

“Two.”

“How old were you?”

“Twenty-one.”

As you can guess, I haven't had much experience ice skating. By the end of the public skate session, though, I had managed to make it around the rink a couple times at a fair distance from the side. I even managed to take both of the girls out all by myself--and let me tell you, that was about the most fun I've had with my children in a while.

I was so proud of Becca too, for really trying to skate both days. She even managed to travel short distances (a couple of yards) on her own.

a geek's lament

Wed, 30 Dec 2009 22:51:00 CDT

Somebody more skilled in humorish things than I needs to write a piece in the style of A college admission essay. In this piece, the author would set forth his geek credentials. Examples:

I have a four-digit Slashdot user id.
I installed Slackware when it only had 12 diskettes.
I won an argument with Ulrich Drepper.
I -funroll-all-loops.
I taught Chuck Norris everything he knows.
I know Bruce Schneier's password.
I have successfully rickrolled somebody.
I survived the Eternal September.
I have ascended with every character class.

...and so forth, perhaps more outlandishly than I have done above. After a suitably long listing of all these feats of geekdom in suitably stylistic prose, the essay should conclude with:

“But I have not yet figured out the usefulness of Google Wave.”

suitcase parent hack

Tue, 29 Dec 2009 22:03:00 CDT

When we asked the girls what they wanted to buy with their Christmas gift money this year, they responded that they wanted little travel suitcases. Neither one of us wanted to buy a little rickety suitcas with thin vinyl siding plastered with the cartoon character du jour, though. We wanted to buy something that would last for a while and even be useful beyond these next couple of years.

We looked around online, but children's luggage is rather boutique-ish. So it looked like we'd have to go with the cheap plastic luggage or with some boutique luggage that would very nearly break the bank.

We were able to go out on a date today and were wandering around TJ Maxx when we noticed a bright orange Columbia rolling carry-on. It was a standard adult size, but it was definitely on the small side. It was also heavily discounted. We looked around a little more and found a slightly smaller red Wenger rolling carry-on as well as a blue Columbia bag of the same style as the orange one.

The Wenger bag was about half again as much as your standard plastic luggage; the Columbia bag was a little bit more than twice as much. Both were significantly cheaper than the other options we'd seen online, though. Both bags can easily accommodate a week's worth of clothes for several years. Even when the girls get older, they can fit a couple days' worth of clothes in the bags. So they win on the price and practicality fronts.

At this point, however, the girls appreciate the color most. I suppose they appreciate the size of the bags, since they can fit a lot of doll clothes in them. (We got the blue Columbia one and the red Wenger one.)

Anyway, that's the parent hack of the day: discounted adult luggage for children.

clos initialization protocol

Tue, 22 Dec 2009 16:40:00 CDT

Update: Tobias Rittweiler pointed out that I don't need &ALLOW-OTHER-KEYS in my method definitions; &ALLOW-OTHER-KEYS was already declared in the DEFGENERIC and isn't required in individual methods. Furthermore, putting &ALLOW-OTHER-KEYS in your method definitions when it's already been declared in DEFGENERIC inhibits useful argument checking. So don't do that. I've fixed the examples below.

From the last blog post, we saw that REINITIALIZE-INSTANCE is a useful way to reduce consing. You say, “well, sure, that's why I have RESET-FOO functions in the library/application that I'm writing.” I'd like to suggest that if you've written such functions, you ought to be writing REINITIALIZE-INSTANCE methods (if you're resetting structures) or writing :AFTER methods on the usual CLOS initialization protocol methods instead. Doing so is more Lispy than the usual reset functions. Writing to the existing protocol also encourages you to more clearly define what's initialization, what's reinitialization, and what can be shared between them.

“OK,” you say, “but what are these CLOS methods and how do I use them?” Glad you asked.

There are three major generic functions that participate in the protocol:

As you might imagine, INITIALIZE-INSTANCE is called from MAKE-INSTANCE and REINITIALIZE-INSTANCE is called from, well, REINITIALIZE-INSTANCE. SHARED-INITIALIZE is called from both INITIALIZE-INSTANCE and REINITIALIZE-INSTANCE and handles bits common to both methods. We'll focus on SHARED-INITIALIZE because it underpins the other two and because SHARED-INITIALIZE is also called when updating class definitions and changing the class of an instance. You get a lot of things for free when writing methods for SHARED-INITIALIZE.

First things first: you generally only want to write after methods on SHARED-INITIALIZE, viz:

(defmethod shared-initialize :after ((object my-class) slot-names &rest initargs &key)
  ...)

The reason you do this is because the primary method on SHARED-INITIALIZE already does a lot of useful things, like initializing slots from their :INITFORMs and :INITARGs. (For the same reason, you should generally use :AFTER methods for INITIALIZE-INSTANCE and REINITIALIZE-INSTANCE.) You can, of course, write:

(defmethod shared-initialize ((object my-class) slot-names &rest initargs &key)
  (call-next-method)
  ...)

But that's generally not how things are done. I think writing :AFTER methods makes your intent clearer: you're doing extra work after the standard bits have been done. It's also easy to forget to CALL-NEXT-METHOD, which would lead to puzzling behavior and also might lead you to submit spurious bug reports like “CLOS initialization doesn't work with user-defined classes”. Also, method combination is often more useful and more declarative than trying to get CALL-NEXT-METHOD in the right place. That doesn't mean CALL-NEXT-METHOD doesn't have its place, it just means I think method combination should be preferred when possible.

OK, so, onto initializing things. What's this SLOT-NAMES parameter? SLOT-NAMES is a (possibly empty) list of slot names that need to be initialized from their :INITFORMs, or T to stand in for “all slots”. For our purposes here, we are going to assume that SLOT-NAMES is only ever T (for when SHARED-INITIALIZE is called from INITIALIZE-INSTANCE) or NIL (for when SHARED-INITIALIZE is called from REINITIALIZE-INSTANCE). In the T case, the primary method on SHARED-INITIALIZE handles the initialization logic. So we don't have to worry too much about SLOT-NAMES.

You can also mostly ignore INITARGS. Most of the keyword-argument pairs in it will have been handled elsewhere by standard CLOS bits. Any bits that are interesting to you will be pulled out via separate &KEY arguments; we'll get to that use-case in a second.

OK, so what useful things can we do in SHARED-INITIALIZE? We can recompute slots that depend on the values of other slots and therefore can't be usefully initialized with :INITFORM or :INITARG:

(defclass triangle ()
  ((a :initarg :a :reader a)
   (b :initarg :b :reader b)
   (c :initarg :c :reader c)
   (area :reader area)))

(defmethod shared-initialize :after ((o triangle) slot-names &rest initargs &key)
  (let ((area (compute-area-of-triangle (a o) (b o) (c o))))
    (setf (slot-value o 'area) area)))

In cases like this, you might be tempted to write some sort of wrapper (with the appropriate modification to the definition of TRIANGLE):

(defun make-triangle (a b c)
  (let ((area (compute-area-of-triangle a b c)))
    (make-instance 'triangle :a a :b b :c c :area area)))

which is certainly doable (I'll leave the subject of whether to wrap MAKE-INSTANCE or expose it directly to users in your API for a future post). But this sort of wrapper doesn't easily enable reinitialization. Consider what your hand-crafted RESET-TRIANGLE function might look like to handle all the cases the SHARED-INITIALIZE method would handle for free:

(defun reset-triangle (triangle &key a b c)
  (let ((a (or a (a triangle)))
        (b (or b (b triangle)))
        (c (or c (c triangle))))
    (setf (slot-value triangle 'a) a
          (slot-value triangle 'b) b
          (slot-value triangle 'c) c
          (slot-value triangle 'area) (compute-area-of-triangle a b c))
    triangle))

Notice that you've duplicated slot setting (the CLOS internals handle that for you) and you're computing the area of the triangle in two different, but related (both part of initialization) places. You might say that you only want users resetting all three sides of a triangle, so the API should reflect that. Maybe. But in that case, you should be the one calling REINITIALIZE-INSTANCE, and you'd wind up going the SHARED-INITIALIZE route anyway. I think the SHARED-INITIALIZE method also better captures what it means to compute the area: the computation of the area is an integral part of initialization, rather than something that you do beforehand prior to creating the object.

For a less pedagogical example, let's say you have a cool encryption algorithm:

(defclass les () ; the LISP encryption standard
  ((round-keys :reader round-keys)
   (n-rounds :reader n-rounds)))

(defmethod shared-initialize :after ((o les) slot-names &rest initargs &key key)
  (multiple-value-bind (round-keys n-rounds) (schedule-key key)
    (setf (slot-value o 'round-keys) round-keys
          (slot-value o 'n-rounds) n-rounds)))

You may have noticed that SHARED-INITIALIZE takes a KEY keyword argument above, even though there's no :INITARG :KEY in the DEFCLASS form. One cool thing about SHARED-INITIALIZE is that any &KEY arguments to it automatically become candidates for use with MAKE-INSTANCE and the other generic functions that participate in the initialization protocol. This feature enables you to pass keyword arguments to MAKE-INSTANCE that aren't directly connected with slots, but require massaging in some way to produce something suitable for slot values. So using the above, you say:

(defvar *l* (make-instance 'les :key #(#xde #xad #xbe #xef)))
...lots of code...
;; sometime later
(reinitialize-instance *l* :key #(#xca #xfe #xbe #xbe))

Again, you could use wrapper functions. But I think the same arguments cited above for clarity and for not repeating yourself apply here as well. (And if you're really serious about this encryption stuff, you also want to be able to reset the initialization vector/nonce for your encryption mode and possibly even to change the encryption mode entirely. Once you've done that, you've basically rewritten SHARED-INITIALIZE, and you have potentially duplicated logic between your MAKE-INSTANCE wrapper and your reset function.)

Another reason to have SHARED-INITIALIZE methods is because they naturally cooperate with subclassing. Let's say you're dealing with some packaging format and you're creating entries from octet vectors:

(defclass entry ()
  ...slots...)

(defun make-entry-from-buffer (buffer &key (start 0))
  (let (...parse out individual slots from BUFFER...)
    (make-instance 'entry ...initargs for slots...)))

Fine and dandy. Now let's say that your customers tell you about packaging format v2, which mostly retains the format of v1, but adds a way of specifying additional metadata as part of the entries. OK:

(defclass entry-v2 (entry)
  ...more slots...)

(defun make-entry-from-buffer (buffer &key (start 0))
  ;; A `2' at the start of the buffer indicates a version 2 entry.
  (if (= (aref buffer start) 2)
      (make-entry-v2-from-buffer buffer :start start)
      (make-entry-v1-from-buffer buffer :start start)))

Hm. We want to initialize the slots in an entry in a common place, so as to avoid code duplication:

(defun initialize-common-entries (entry buffer &key (start 0))
  (let (...parse out individual slots from BUFFER...)
    (setf ...lots of slots...)
    entry))

(defun make-entry-v2-from-buffer (buffer &key (start 0))
  (let ((entry (make-instance 'entry-v2)))
    (setf ...new slots for ENTRY-V2...)
    (initialize-common-entries entry buffer :start start)))

and so on. Possibly with slight adjustments because parsing the slots for ENTRY-V2 might depend on the values of one or more of the common slots in ENTRY. I claim that this is more elegantly handled by:

(defmethod shared-initialize :after ((o entry) slot-names &rest initargs &key buffer start)
  (let (...parse out individual slots from BUFFER...)
    (setf ...lots of slots...)
    o))

(defmethod shared-initialize :after ((o entry-v2) slot-names &rest initargs &key buffer start)
  (let (...parse out individual slots from BUFFER...)
    (setf ...slots for ENTRY-V2...)
    o))

(defun make-entry-from-buffer (buffer &key (start 0))
  ;; A `2' at the start of the buffer indicates a version 2 entry.
  (if (= (aref buffer start) 2)
      (make-instance 'entry-v2 :buffer buffer :start start)
      (make-instance 'entry :buffer :start start)))

Since :AFTER methods run in least-specific-first order, the :AFTER method on ENTRY will be called first. The :AFTER method on ENTRY-V2 can therefore freely use the value of any slots from ENTRY. Using SHARED-INITIALIZE here not only enables REINITIALIZE-INSTANCE, it also does the right thing in handling common code from a software engineering perspective.

Of course, if you have code that specifically needs to run at MAKE-INSTANCE time, or REINITIALIZE-INSTANCE time, then you can add :AFTER methods to INITIALIZE-INSTANCE or REINITIALIZE-INSTANCE as appropriate. For instance, if all of your objects need to have a unique ID, you surely don't want to assign a unique ID in SHARED-INITIALIZE.

(defclass unique-id-mixin ()
  ((unique-id :reader unique-id)))

(defmethod initialize-instance :after ((o unique-id-mixin) &rest initargs &key)
  (setf (slot-value o 'unique-id) (get-unique-id-for-instance o)))

(You might, however, want to assign a different unique ID if somebody change-class'ed your object. That's a topic for future discussion; I only sort-of understand the protocol involved in CHANGE-CLASS and I have yet to see really pragmatic reasons for adding methods to the generic functions involved.)

I know that my code doesn't always work according to the ideas I've laid out above; it's taken me a while to wrap my head around what goes where and why. But now that I understand, I'm slowly modifying my code to use these ideas. Where can your code benefit from these ideas?

archive release

Fri, 18 Dec 2009 23:11:00 CDT

I've released ARCHIVE 0.8. It features an assortment of bugfixes, including fixes for extraction of tar files and fixes for Lispworks/Win32 users. This version also understands POSIX global headers in tar files, so if you've been getting complaints about “unknown typeflag 103” or similar, now would be a good time to upgrade. It is available from the usual places, including straight from github. (And like a git n00b, I've managed to botch the tagging of 0.8. The tag of 0.8 is almost right, except the version number in the .asd is wrong...sigh.)

surveillance clock

Fri, 18 Dec 2009 20:24:00 CDT

One of the more interesting (and frightening) pieces I've read in a while is Jeff Jonas's Six Ticks till Midnight: One Plausible Journey from Here to a Total Surveillance Society. He explains the ACLU's Surveillance Society Clock and exactly why it's set where it is by laying out the rough sketches of what needs to happen before the activities of everybody can be tracked 24-7. (Aside: the ACLU utterly failed at this. If you look at their site without reading Jonas's post, it looks like general fear-mongering, since they don't provide any explanation of why the clock is where it is.) Also note the clock has moved up one minute since Jonas's post two years ago.

The really amusing/interesting/chilling part is this:

Oh yeah, one more thing, no more need for facial recognition (a very hard problem many years off anyway). In this coming world, all that useless video being collected can now be efficiently recalled because GPS data provides the missing link … who was where when?
While the exact technologies or the exact sequence of events may unfold quite differently, nonetheless such a future is coming. And this future is being created by us consumers, not the government!
Consumers are funding the surveillance economy, with the blistering pace of this extraordinary surveillance being driven by ordinary people who relish all the technological advances and willing to entirely trade in their information and privacy as they optimize their life.

One of the reasons I've become very interested in reading about history--a subject which I disliked in school--is this idea of unintended consequences. (The same principles come up in economic/political discussions too, except nobody seems to care...wait, that's exactly how history works, too.) Luther, for instance, almost certainly had no intention of producing the individualistic evangelical culture we have today--but you can trace a pretty convincing path between the two. Or economists making decisions based on some particular model of things and convincing themselves that said decisions will have the desired effect--without realizing that those same decisions are changing fundamental assumptions undergirding the model. (William Greider's Secrets of the Temple, which discusses the Federal Reserve's actions under Paul Volcker, has several examples of this.) There's no grand cabal of people, no portly, mustached executives and bureaucrats carefully designing a system to track everybody everywhere--we are doing this to ourselves.