SPDE, Semi-functional programming for Processing

Written by Jeff Heard on January 15th, 2010

I’ve been using Processing for awhile now as a tool for prototyping visualizations and getting visualizations on the web.  The main problem with Processing is all the hideous programming habits it encourages; I feel at the end of a long bout of Processing like I’m my 5 year old self back at the keyboard of my Commodore 64, typing 10 PRINT “Hello World!” The only thing it doesn’t have is a goto statement.  So imagine my pleasant surprise when I found you could code in Processing without having to use their awful P4 subdialect of Java.  Instead, I can code in Scala, which while not as purely functional as Haskell, does allow functional programming and encourages more sensible programming. The environment is called SPDE and at its most basic level it consists of The Graft, a specially constructed catch-all package that contains the scala compiler, the simple-build-tool, and the Processing runtime and bells and whistles.

Unlike Processing, SPDE is command-line oriented, but all the familiar things (except one niggling detail, and that is the creation of fonts, which still has to be done in the Processing tool) are there, including applet creation, one-step running, etcetera.  Actually applets are a little less work in SPDE, as typing applet in the tool will cause a browser window to pop up with the created applet, instead of merely exporting the directory and showing it in the Finder (I use a Mac).

To get going, download the graft jarfile and execute it in your operating system’s favorite fashion (e.g. java -jar graft-setup-0.2.2.jar).  You should get a fresh new directory with a Sketch.spde, several directories, and an sbt.bat or sbt file (depending on your OS).  If everything’s worked perfectly, it’ll even pop up a sketch that looks like an epilepsy-triggering barcode.

Now to start doing your own thing.  sbt is the Simple Build Tool.  Sketch.spde is the file you’ll start modifying.  Any file with an .spde extension will be compiled by the compiler into a single sketch (note these are not really modules and you don’t get separate namespaces for each file, unlike Java, so if you have name clashes, they will produce esoteric error messages that e.e. cummings would have been proud to have written).  SPDE and Processing both follow the same model of faking a toplevel by enclosing whatever you type in the runtime class, so you get a lot for free, and although you’re still writing Scala, it’s a little freer than the straight-up Scala you’d write normally.  Here’s an applet that i created in SPDE. Let’s look at some of the code that produced it:

import scala.io._
import org.joda.time._

val minR = 3
val maxR = 50
val minVR = 100
val maxVR = 3000000
val maxVC = 9000000
val rS = color(26,27,48,198)
val rF = color(78,102,136,198)
val xS = color(52,16,30,232)
val xF = color(111,61,72,232)
val pSD = "start_date"
val pED = "end_date"
val pV = "amount"
val pN = "title"

val rFs = Array(color(17,118,109,128), color(65,9,54,128))
val xFs = Array(color(217,251,25,232), color(228,111,10,232))
val rSs = rFs
val xSs = xFs
val rWs : Array[Float] = Array(/*0,*/0.9,0.9)

val nc = new Geography(
    Point(-84.322, 33.845),
    Point(-75.459, 36.588),
    Point(200+0,35),
    Span(1066,331),
    Right(loadImage("nc.png"))
)

val world = new Geography(
    Point(-180, -89.902),
    Point(180,83.677),
    Point(370+(1066.0-684)/2, 455),
    Span(684,331),
    Right(loadImage("world.png"))
)

val historicalNCGrants = Features.fromStringConcrete(nc, Source.fromInputStream(createInput("historical_nc_grants.ows")).mkString)
val activeNCGrants = Features.fromStringConcrete(nc, Source.fromInputStream(createInput("active_nc_grants.ows")).mkString)

val historicalUSAGrants = Features.fromStringConcrete(world, Source.fromInputStream(createInput("historical_us_grants.ows")).mkString)
val activeUSAGrants = Features.fromStringConcrete(world, Source.fromInputStream(createInput("active_us_grants.ows")).mkString)

val hNCGrantsOnMap = new TimeseriesMapBubbles(minR, maxR, minVR, maxVR, maxVC, rS, rF, xS, xF, historicalNCGrants, pSD, pED, pV, pN)
val aNCGrantsOnMap = new TimeseriesMapBubbles(minR, maxR, minVR, maxVR, maxVC, rS, rF, xS, xF, activeNCGrants, pSD, pED, pV, pN)

val hUSAGrantsOnMap = new TimeseriesMapBubbles(minR, maxR, minVR, maxVR, maxVC, rS, rF, xS, xF, historicalUSAGrants, pSD, pED, pV, pN)
val aUSAGrantsOnMap = new TimeseriesMapBubbles(minR, maxR, minVR, maxVR, maxVC, rS, rF, xS, xF, activeUSAGrants, pSD, pED, pV, pN)

val hNCGrants = new TimeIntervalSeriesAreaChart(
      FaceUp
    , rF
    , rS
    , rWs(0)
    , 5
    , 790
    , 1056
    , 50
    , Days.days(15).toPeriod
    , hNCGrantsOnMap.begin
    , aNCGrantsOnMap.end
    , new DateIntervalLinearInterpolator(historicalNCGrants, pSD,pED, 30)
    , pV)

val aNCGrants = new TimeIntervalSeriesAreaChart(
      FaceUp
    , rF
    , rS
    , rWs(0)
    , 5
    , 0
    , 790
    , 50
    , Days.days(30).toPeriod
    , aNCGrantsOnMap.begin
    , aNCGrantsOnMap.end
    , new DateIntervalLinearInterpolator(activeNCGrants++activeUSAGrants, pSD,pED, 30)
    , pV)   

val grantInceptionTimeline = new Timeline(
      FaceUp
    , rS
    , 0.8
    , 5
    , 0
    , 790
    , 50
    , aNCGrantsOnMap.begin
    , aNCGrantsOnMap.end
    , activeNCGrants ++ activeUSAGrants
    , pSD
    , pV) 

val grantEndTimeline = new Timeline(
      FaceUp
    , xF
    , 1.0
    , 5
    , 0
    , 790
    , 50
    , aNCGrantsOnMap.begin
    , aNCGrantsOnMap.end
    , activeNCGrants ++ activeUSAGrants
    , pED
    , pV) 

object PauseButton {
    def draw = {
        strokeWeight(0.8)
        stroke(rS)
        fill(rF)
        rect(268,759,60,24)
        fill(200,203,212)
        text("Pause", 272,778)
    }

    def mousePressed : Boolean = {
        if(mouseX > 268 && mouseY > 759 && mouseX < (268+60) && mouseY < (759+24)) {
            isPaused = !isPaused
            return true
        }
        else return false
    }
}    

// End data definition.  Begin drawing code.

size(1266,800)
background(237)
smooth
frameRate(25)
val dateFont = loadFont("FreeSansBold-24.vlw")
var isPaused = false

aNCGrantsOnMap.date = aNCGrantsOnMap.begin
aUSAGrantsOnMap.date = aNCGrantsOnMap.begin

def draw = {
    background(200,203,212)
    world.draw
    nc.draw

    pushMatrix
    rotate(PI/2)
    grantInceptionTimeline.draw
    grantEndTimeline.draw
    aNCGrants.draw       

    popMatrix

    aNCGrantsOnMap.draw
    aUSAGrantsOnMap.draw

    if(!isPaused) {
        grantInceptionTimeline.date = if(grantInceptionTimeline.date isAfter grantEndTimeline.end) {
            grantInceptionTimeline.begin
        } else {
            grantInceptionTimeline.date plusDays 2
        }
        grantEndTimeline.date = grantInceptionTimeline.date
        aNCGrantsOnMap.date = grantInceptionTimeline.date
        aUSAGrantsOnMap.date = grantInceptionTimeline.date
    }

    PauseButton.draw

    fill(rS)
    textFont(dateFont)
    text(aNCGrantsOnMap.date.toString("YYYY.MM"), 180,780)
    textFont(dateFont,18)

}

override def mousePressed = {
    if(!PauseButton.mousePressed) {
        if(!(grantInceptionTimeline.mousePressed || grantEndTimeline.mousePressed)) {
            if(!aNCGrantsOnMap.mousePressed && !aUSAGrantsOnMap.mousePressed) {
                saveFrame("snapshot.png")
            }
        }
        else {
            grantEndTimeline.date = grantInceptionTimeline.date
            aNCGrantsOnMap.date = grantInceptionTimeline.date
            aUSAGrantsOnMap.date = grantInceptionTimeline.date
        }
    }
}

This Sketch.spde file produced something similar to (the colors are different, the vis is branded, and the data’s not private are the differences, not differences in code) the linked-to visualization. Note that quite a lot of these classes, like Geography and Features are things I defined myself in other .spde files. Processing is an immediate-mode framework for visualization, rather than a scene-graph framework, but scene-graphs can be made to be more functional than immediate mode visualizations are make for a whole lot better reuse, so I tend to encapsulate my data and visuals semantically like I do in Haskell.

The best thing about Spde is probably the development cycle.  To get a development environment that tests the sketch every time you save it, open Sketch.spde in your favorite text editor, and then in a terminal, type “./sbt ~run”.  Now every time you save a file with the extension spde in that directory, the compiler will attempt to compile and run the test in the background.  If all you want to do is check to make sure you’re still able to compile after a save, the command is “./sbt ~compile”  The “every time I save the file” part is the tilde before “compile” and “run.”  It’s fast and convenient, even more so than vanilla Processing.

There are things about Spde that I don’t like out of aesthetic reasons, like the fact that Scala makes procedural programming both too necessary and too easy, and the fact that it requires type-casts all together too often, but if I’m going to have to compromise to get something to show up in a browser, I’d prefer to compromise with Scala rather than some other language.

The final visualization took a few days to write instead of an afternoon, but what I came out with was much more reusable than anything I would have come out with in Processing, which discourages the use of classes and the encapsulation of drawable data into objects (doesn’t prohibit, but it certainly doesn’t help, either).

Leave a Comment