» Saturday, 24 January A.D. 2009

common lisp and javascript

I haven't done a lot of Lisp hacking lately. I've done a little bit of experimentation with assembly routines in Ironclad and a little hacking for GNU tar extensions and pax headers in ARCHIVE. The bulk of the twiddling with Lisp lately has been experimenting with a visualization tool for our home finances.

I record all our finance bits (well, at least what we spend on a day-to-day basis; nothing like tracking my 401(k) or anything like that) in Ledger. Ledger is great as far as it goes, but is somewhat lacking in being able to make pretty pictures--which is fine. But my wife would like to get a picture of where the money goes in our household, so I sat down to try to put something together for her.

Version 0.1 was about ten minutes of thinking about creating hand-drawn graphs with something like CL-GD. That sounded like a lot of tedious work, and it wouldn't necessarily be more accessible to my wife than teaching her the various invocations of Ledger at the command line, which was right out. So I quickly moved onto version 1.0.

Version 1.0 was an exercise in having Ledger generate XML output, parsed by CXML, which was then turned into web pages served by Hunchentoot, screen-scraped by MochiKit, and then be turned into snazzy plots by PlotKit. This actually worked pretty well; programming in JavaScript is fairly painless, MochiKit is pleasant to use, and PlotKit is straightforward enough. Even better was that I could just stick the whole thing on a server in our house and my wife could view the ins and outs of our budget with no difficulty. But scraping these tables was tedious and it seemed like there ought to be a better way.

Version 2.0 came about when I realized that I could expose the XML that Ledger generated as JSON URIs, and virtually all of the page layout and so forth could be driven by JavaScript. So I grabbed YASON for generating the JSON, scrapped a good bit of the JavaScript I had written, and went off to the races. This worked much better and I was able to learn how to use the nifty asynchronous bits of MochiKit to boot.

One bit about using YASON: the approved way of turning Lisp bits into JavaScript objects is to send a hashtable through YASON. This is uncontroversial, but I found myself wanting to pass a dictionary to JavaScript to fake keyword arguments. Common Lisp's hashtables are not really suited for such one-off bits (no literal syntax)--using an alist or a plist is much easier for this sort of thing. (Yes, I could use custom Lisp-side objects, but that's almost more heavyweight than hashtables.) YASON didn't support this sort of usage, so I tweaked it accordingly:

(defmethod json:encode ((list list) &optional (stream *standard-output*))
  (labels ((json-alist-p (list)
             (or (null list)
                 (and (consp (first list))
                      (json-alist-p (rest list)))))
           (json-plist-p (list)
             (or (null list)
                 (and (keywordp (first list))
                      (consp (rest list))
                      (json-plist-p (cddr list)))))
           (encode-key/value (key value)
             (let ((string (symbol-name key)))
               (json:encode-object-element string value)))
           (json-encode-alist (alist)
             (json:with-output (stream)
               (json:with-object ()
                 (loop for (key . value) in alist
                    do (encode-key/value key value)))))
           (json-encode-plist (plist)
             (json:with-output (stream)
               (json:with-object ()
                 (loop for (key value . rest) on plist by #'cddr
                    do (encode-key/value key value))))))
    (cond
      ((json-alist-p list) (json-encode-alist list))
      ((json-plist-p list) (json-encode-plist list))
      (t
       (write-char #[ stream)
       (let (printed)
         (dolist (value list)
           (if printed
               (write-char #, stream)
               (setf printed t))
           (json:encode value stream)))
       (write-char #] stream)))
    list))

Much better. I considered string-downcase'ing the symbols used for keys, but while that would be easier for simple cases (no need to || symbols), it prevents you from actually having capital letters in your object properties.

The result so far is that I have about 400 lines of Common Lisp and about 150 lines of JavaScript; that gets several useful views onto the data: a yearly summary, a breakdown of each spending category by month, and a breakdown of each month by category. The next steps are to add some links between them, an easy way to get information on individual transactions in the latter two views, and some information on how we're doing versus the budget that we've decided on. Ironclad and ARCHIVE can wait, for the moment; I'm having too much fun with this.

posted by Nate @ 9:29PM