Is Abstraction a Double Edged Sword?

I've been writing a lot about functional programming. I do so out of a two pronged interest, the first prong of which is that I'm interested in understanding things myself, and I find there are few better ways to test one's understand than to try and explain that understanding to others. The second reason is that I'm interested in education for its own sake. I've been drafting some chapters for a book in pure functional game development, mostly for reason 1, and I've run across some insights I thought I'd blog about, to see if anyone has any helpful comments.

The issue at stake is that abstraction cuts both ways. Abstract solutions are clean, simple, re-usable and reliable. But that very abstraction makes them hard to get a handle on. They seem to be cognitively slippery.

For Instance

Consider that we wish to represent a game state purely functionally, as an association list, for instance. The game I'm working on as an example is a kind of farming simulation where you grow letters and build items by spelling words. Its game state (in Racket) may look like:

(define initial-state
  '((stamina . 6)
    (total-turns . 0)
    (points . 0)
    (letters . ((f . 2) 
                (o . 4)
                (d . 2)))
    (items   . ())
    (field . ())))

This is pretty straightforward and concrete, such as these things go. Each field represents a part of the game state. stamina, for instance, represents how many more turns we can take before we we have to rest for the day, while letters is an association-list storing the number of each letter/crop we've got stocked up.

We want, however, to avoid side effects. That means we need functions to create a new association list whenever we want to "change" a field. Eg:

(define (increment-points state by)
  (let ((current-points (dict-ref state 'points)))
    (dict-set state (+ current-points by))))

Or:

(define (increment-stamina state by)
  (let ((current-stamina (dict-ref state 'stamina)))
    (dict-set state (+ current-stamina by))))

My aim is to write for a completely novice programmer, and both these solutions, I think, are pretty comprehensible for such a programmer. More advanced functional programmers might prefer to write:

(define (increment-field state field by)
  (let ((value (dict-ref state field)))
   (dict-set state field (+ value by))))

And then, perhaps:

(define (increment-stamina state by)
  (increment-field state 'stamina by))

(define (increment-points state by)
  (increment-field state 'points by))

Both functions are probably still comprehensible to new programmers. increment-field probably seems like a kind of detour or distraction, on the way to the concrete goal of changing our game state. But its a small one, and relatively concrete. Hardly distracting at all.

If I were writing this code for myself, without worrying much about teaching someone what I was doing, I'd take this approach:

(define (partial-left f . partial-args)
  (lambda rest
    (apply f (append partial-args rest))))
(define (partial-right f . partial-args)
  (lambda rest
    (apply f (append rest partial-args))))
(define (always v)
  (lambda rest v))
(define (compose f1 f2)
  (lambda (args)
    (f1 (f2 args))))

(define (dict-transform dict keys-or-key fun)
  (match keys-or-key
    [(or (? symbol? key)
         (list key))
     (let ((current (dict-ref dict key)))
       (dict-set dict key (fun current)))]
    [(list key subsequent-keys)
     (let ((sub-dict (dict-ref dict key)))
       (dict-set dict key
                 (dict-transform sub-dict subsequent-keys fun)))]))

(define (increment-field dict field by)
  (dict-transform dict field (partial-right + by)))

(define (increment-stamina dict by)
  (increment-field dict 'stamina by))
(define (increment-points dict by)
  (increment-field 'points by))

Or, if we want to go really nuts:

(define (curry-left f)
  (lambda uncurried 
    (lambda curried 
      (apply f (append uncurried curried)))))

(define (curry-right f)
  (lambda uncurried
    (lambda curried
      (apply f (append curried uncurried)))))

(define (compose-partially f . transformers)
  (lambda args
    (apply f 
           (let loop ((fs transformers)
                      (as args)
                      (acc '()))
             (if (or (empty? as)
                     (empty? fs))
                 (reverse acc)
                 (loop (cdr fs)
                       (cdr as)
                       (cons ((car fs) (car as)) acc)))))))

(define (id x) x)

(define increment-field-2 
  (compose-partially dict-transform id id (curry-left +)))

I might be ashamed to admit it publically, but I think this implementation of increment-field is kind of rad (although compose partially could probably be made more efficient or given static semantics). Using combinators like compose-partially, curry-* and partial-* we abstract away all the details involved in writing the our functions. It takes a bit of thought to write this way, but this implementation has many fewer tokens into which a mistake could creep.

And yet its completely billy bollucks pedogogically. For one thing, we're writing a ton of functions which have nothing to do with the goal at hand. They'll be useful later because they solve a class of functional programming problems. And they are useful now because they make the solution to this problem simpler, but they seem like a divergence to the reader.

They are also abstract - not only do they not refer specicically to our problem, they don't seem to refer specifically to anything. Some of that is naming (curry is not very descriptive) but some of it is that we are programming at the "function level," in APL/J parlance. The objects we are manipulating are the fundamental objects of our code themselves (functions). These functions (curry-*, partial-*, compose, etc) are all about controlling when, exactly, things are "done." They are conspicuosly not about doing things.

And then, even when we get to our function, increment-field, the implementation seems to have nothing at all to do with what we seem to want to be doing. No names appear, no values seem to be calculated (although functions are calculated, they seem invisible to a new programmer).

The Problem

Who cares, though? If you are teaching novices, just avoid this approach, you might say. Fair enough, but it seems that when functional purity is a design goal, these kinds of higher order functions really help to avoid verbose and repeatative code. How would you find a happy medium? When you restrict yourself to low level idioms, purely functional code seems to become ugly, deeply nested, and difficult to read and write. Even given that I've hammed it up seriously in this post, it seems hard to find a happy medium. What do my readers (if any there be) think?