defservice

These are my notes and (shoddy) slides for a short talk I gave at the Berlin Lispers meeting, September 14 2010.

The code for this system can now be found at http://github.com/marijnh/defservice.

I'll talk about a set of macros that we use in the HTTP code for AllegroGraph

some of you have no doubt defined similar frameworks
in fact, this is about the fourth system of this type that I wrote
i feel I've hit something of a sweet spot this time

no theoretically interesting advanced macrology, except maybe some remarks about redefinition handling

code example slide

this is what code using the macros looks like

it defines, one by one, HTTP services, which are tied to a certain URL and method
each service definition is based on a context, which represents a start url in whose context it is defined
it then extends this url with a path containing zero or more elements

so /users/johnny and /users/marijn both match the example handler

a parameter minilanguage makes fetching and parsing HTTP parameters easy
(name type [default])
meaningful errors returned when required parameters are omitted, or wrong format is given
(:string :boolean :integer :float :list)

service def ends with a body, a piece of code that implements the service logic, optionally outputs the response

new contexts can be defined with defcontext, and (optionally) given a body of code that is executed when they are entered
contexts without bodies are simply shorthands
multiple context definitions of the same name are possible, creating multiple paths into that context and the services below it

DAG slide

the structure built up by the defservice/defcontext forms is a DAG
in the picture, gray ovals represent fixed-word edges, blue ones variable words
multiple edges to fixed words can come out of a node, only a single variable one
(they alternate here, and usually do, but don't have to. for example ...)

the dispatcher splits a url on slashes, url-unescapes the parts, and works its way through this DAG

data structure slide

each 'URL' (roughly, the nodes in the graph we just saw) has a set of handlers, one per method (GET/POST/PUT/DELETE)
during dispatch, when the whole url has been consumed, the right method is looked up in the final node
if no handler for the method, return a 405 (method not allowed) error

each node also has an outgoing edge

when, during dispatch, a node without outgoing edge is reached, or a split with no matching label, we have a 404 error

edges may also have a 'wrap' function, which originates in a defcontext body, and is executed when traversing that edge

the library stores a table mapping context labels to nodes
a 'start context' is simply an orphan node that is used as the start state in dispatch

i'd say that the defservice and defcontext forms are easier to think about than such a DAG
which isn't to say that there aren't some gotchas

re-evaluation slide

an important consideration with macros that maintain a global data structure is re-evaluation

on re-evaluation, the new definition should replace the old
redefining a context should not kill all nodes under it

lookup in a split edge is linear (beats hash table for typical split sizes)
the order of the edges is definition order, so new definitions append, not push
on redefinition, the old position of the edge is reused

error checking. it is possible to define 'impossible' edges, which are both variable and split
the macroexpansion just raises an error, you have to clear your start-context if you want to really change an edge
(this would actually be a great use of a restart)

success story slide

we're using this system to great effect in a server implementing some 100 different services

the context wrappers can nicely 'centralize' some code, making individual services simpler

DAGs rather than trees are actually useful

and of course, declarative parameters

trivial to add new services, even for people not very familiar with the code

utilities slide

other utilities that help make service definitions succinct

a hierarchy of condition types for HTTP responses

a macro that wraps a body in a handler that

content-negotiation system that allows the handlers to just specify a value and a type
will find the correct 'writer' for the value, given the Accept header that the request specifies
or return a not acceptable error if no writer available