Dispatch: Binding URLs to Procedures

Dave Gurnell

dave at untyped

Dispatch is a web development tool for creating a two-way mapping between permanent URLs and request-handling procedures known as controllers. The library provides a simple means of dispatching requests to matching controllers and of reconstructing URLs from controller calls.

The first section of this document provides a brief overview of the features of Dispatch. The second section provides a working example of how to set up a simple blog from scratch using Dispatch. The third section provides a reference for the Dispatch API.

1 Overview

1.1 URLs to controllers

The namesake feature of Dispatch is the ability to dispatch HTTP requests to controller procedures. Imagine you are writing a blog application and you want the following URLs to point to the most important parts of the site:

• the URL "http://www.example.com/" should map to the procedure call (list-posts);

• URLs like "http://www.example.com/posts/hello-world" should map to procedure calls like (review-post "hello-world");

• URLs like "http://www.example.com/archive/2008/02" should map to procedure calls like (review-archive 2008 2).

Dispatch makes it very easy to create this kind of configuration using code like the following:

(define-site blog

([(url "") list-posts]

[(url "/posts/" (string-arg)) review-post]

[(url "/archive/" (integer-arg) "/" (integer-arg))

review-archive]))

; request -> response

(define-controller (list-posts request)

...)

; request string -> response

(define-controller (review-post request slug)

...)

; request integer integer -> response

(define-controller (review-archive year request month)

...)

1.2 Controllers to URLs

Dispatch helps further by providing a way of recreating URLs from would-be calls to controllers. For example, the code:

  (controller-url display-archive 2008 2)

applied to display-archive from the example above would construct and return the value "/archive/2008/2".

1.3 Clean separation of view and controller

The define-site macro binds identifiers for the site and all its controllers. define-controller mutates the controllers defined by define-site so that they contain the relevant controller bindings.

This separation of interface and implementation means that there is a simple way of accessing all your controllers from anywhere in your application, without having to worry about cyclic module dependencies. Simply place the define-site statement in a central configuration module (conventionally named "site.ss") and require this module from all other modules in the application to gain access to your controllers. As long as the various define-controller statements are executed once when the application is started, the majority of the application only needs to know about "site.ss".

2 Quick Start

This section provides a worked example of using Dispatch to set up the blog described earlier. The example also uses Instaservlet to simplify the web server configuation. Some details are skipped over here: see the API Reference for more information on the macros and procedures used.

2.1 Create the site

The first step is to create a site definition using define-site. Create a directory called "blog" and in it create a file called "blog/site.ss". Edit this file and type in the following:

#lang scheme/base

(require (planet untyped/dispatch))

(define-site blog

([(url "/") index]

[(url "/posts/" (string-arg)) review-post]

[(url "/archive/" (integer-arg) "/" (integer-arg))

review-archive]))

(provide (site-out blog))

Now that the site has been defined we just need a servlet to create a working web application. We will simplify the creation of our servlet by using the serve/dispatch procedure. Create a file called "blog/run.ss", edit it and type in the following:

#lang scheme/base

(require (planet untyped/dispatch)

"site.ss")

(serve/dispatch site)

serve/dispatch starts a web server and populates it with a single servlet that dispatches to site whenever it receives an HTTP request.

We should now be able to test the site. On the command line type:

  mzscheme run.ss

and go to "http://localhost:8000/" in your web browser. You should see an error page saying something like “Controller not defined”. Also try "http://localhost:8000/posts/hello-world" and "http://localhost:8000/archive/2008/02".

serve/dispatch provides a default 404 handler that it uses when it cannot find a matching rule. Test this by going to "http://localhost:8000/foo" in your browser.

2.2 Define some controllers

The Controller not defined error pages above are appearing because there are no define-controller statements for our controllers. We will write a define-controller statement for review-post now. Create a directory "blog/controllers" and in it a file called "blog/controllers/posts.ss". Edit this file and type in the following:

#lang scheme/base

(require (planet untyped/dispatch)

"../site.ss")

; request string -> html-response

(define-controller (review-post request slug)

`(html (head (title ,slug))

(body (h1 "You are viewing " ,(format "~s" slug))

(p "And now for some content..."))))

We need to make sure "posts.ss" gets executed so that this definition gets loaded into blog. To do this, add an extra clause to the require statement in "run.ss" so that it reads:

(require (planet untyped/dispatch)

"site.ss"

"controllers/posts.ss")

Now re-run the application and go to "http://localhost:8000/posts/hello-world" in your browser. You should see the web page we just created.

2.3 Insert links from one controller to another

Now that we are able to write controllers and dispatch to them, we need to know how to create links from one controller to another. Dispatch lets us do this without having to remember the URL structure of the site. Return to "blog/controllers/posts.ss" and add the following code for the index controller:

; request -> html-response

(define-controller (index request)

`(html (head (title "Index"))

(body (h1 "Index")

(ul ,@(map index-item-html

(list "post1"

"post2"

"post3"))))))

; string -> html

(define (index-item-html slug)

`(li (a ([href ,(controller-url review-post slug)])

"View " ,(format "~s" slug))))

In this code, the index controller is generating a list of posts using a helper procedure called index-item-html. index-item-html is using a procedure from Dispatch called controller-url to create URLs that point to review-post. controller-url takes as its arguments the controller to link to and the values of any URL pattern arguments: note that there is no request argument.

Note that review-post is being provided from the define-site statement "site.ss", not from the define-controller statement in the local module. We can easily move index-item-html out into a separate module of view code without creating a cyclic module dependency. For the moment, however, we just need to see the code working. Re-run the application and go to "http://localhost:8000/" in your browser.

You should see a list of three links. Inspect the HTML source of the page and notice that the links point to URLs like "/posts/post1". These are not continuation links - they are permanent, memorable, bookmarkable links to the posts. What is more, these URLs are generated from the URL patterns in the definition of blog in "site.ss": we can change these patterns in this one place and generated URLs will change accordingly throughout the site.

Note that we can still use continuations to call review-post. Simply wrap a normal procedure call in a lambda statement as normal:

; string -> html

(define (index-item-html slug)

`(li (a ([href ,(lambda (request)

(review-post request slug))])

"View " ,(format "~s" slug))))

The URLs generated by this approach will expire after a finite time, but in exchange we get the full state-passing power of continuations.

2.4 Define a custom 404 handler

It is worth noting that we can replace Dispatch’s default 404 Not Found handler with our own code by passing an extra keyword argument to serve/dispatch:

(serve/dispatch

site

#:file-not-found-responder

(lambda (request)

'(html (head (title "404"))

(body (p "Oops! We could not find what you were looking for.")))))

Note that, unlike in Dispatch 1.x, we cannot achieve the same result using a catch-all "not found" rule at the end of the site. This is because serve/dispatch looks for static files such as CSS files and images after it scans the rules in the site. Placing a 404 rule in the site would prevent th web server serving these static files.

2.5 Next steps...

The quick start has demonstrated how to get up and running with Dispatch. However, Dispatch contains many more features that we have not covered. You can find more information in the API Reference documentation below, including:

• how to define your own argument types for use in URL patterns;

• how to define controllers that can only be called by continuation;

• how to abstract common setup tasks (for example user identification, authentication, exception handling and cookie handling) into request pipelines.

3 API Reference

The API for Dispatch is made available by requiring a single file, "dispatch.ss":

(require (planet untyped/dispatch))

The following sections document the forms and procedures provided.

3.1 Defining sites and controllers

(define-site id (rule ...) site-option ...)

rule   =   (condition controller-id)           site-option   =   #:other-controllers (controller-id ...)     |   #:rule-not-found (request -> response)           condition   =   (url url-part ...)           url-part   =   string     |   arg

Creates a new site and a set of controllers and binds them to id and each unique controller-id.

Controllers are referenced within the site via a collection of rules. Each controller is bound to a single identifier, but may be referenced by as many rules as desired. When a request is dispatched to the site using dispatch, the rules are evaluated in the order specified until a match is found. The corresponding controller is called and passed the request and any arguments from the rule’s condition(s).

Currently only one type of condition is supported: the url form creates a regular expression pattern that is matched against the the path part of the request URL. String arguments to url are matched verbatim; arg arguments capture patterns in the URL and convert them to Scheme values that are passed to the controller. Anchor strings ("#anchor"), request arguments ("?a=b&c=d") and trailing slashes ("/") are ignored when matching.

The optional #:other-controllers argument can be used to specify controllers that are not bound to any URL. These controllers may be called like normal procedures (including by continuation) but cannot be used with controller-url.

The optional #:rule-not-found argument can be used to specify a procedure to call when no matching controller is found. This is useful if you want to override the default 404 page without defining a special controller.

(site-out site)

Provide form that provides site and its associated controllers.

(define-controller (id arg ...) expr ...)

(define-controller id pipeline procedure)

pipeline   =   (listof stage)

Initialises id, which must be a controller bound using define-site. The first form is the equivalent of a standard PLT procedure definition:

(define (id arg ...)

expr ...)

allowing all the same features including keyword arguments, optional arguments and multiple return values. The second form allows you to specify a request pipeline to use with the controller. Pipelines are a useful abstraction for common tasks to perform when the controller is called and/or returns. Pipelines are part of the unlib.plt package, and are beyond the scope of this document. See the documentation for unlib.plt for more information.

Controllers can be called directly just like normal Scheme procedures. If a controller has no pipeline, calling it is equivalent to calling its body procedure. For example, given an appropriate site definition and the code:

(define-controller (my-controller request a b c)

; ... )

calling (my-controller request 1 2 3) is equivalent to calling:

((lambda (request a b c)

; ... )

request 1 2 3)

If a controller is defined with a pipeline:

(define-controller my-controller

my-pipeline

(lambda (request a b c)

; ... ))

calling (my-controller request 1 2 3) is equivalent to calling:

(call-with-pipeline my-pipeline

(lambda (request a b c)

; ... )

request 1 2 3)

3.2 Standard URL pattern arguments

Dispatch provides several built-in types of URL pattern arguments:

(integer-arg) → arg?

Creates an argument that captures a section of the URL, converts it to an integer and passes it as an argument to the controller. Equivalent to the regular expression #rx"[-]?[0-9]+".

(real-arg) → arg?

Similar to integer-arg except that it captures real numbers. Equivalent to the regular expression #rx"[-]?[0-9]+|[-]?[0-9]*.[0-9]+".

(string-arg) → arg?

Creates an argument that matches one or more non-slash characters and passes them as an argument to the controller. Equivalent to the regular expression #rx"[^/]+".

(symbol-arg) → arg?

Similar to string-arg except that the captured pattern is converted to a symbol before it is passed to the controller.

(rest-arg) → arg?

Similar to string-arg except that it captures any characters including slashes. Equivalent to the regular expression #rx".*". Note that trailing slashes in the URL never get matched.

You can also make your own types of pattern argument in addition to the above. See Custom URL pattern arguments for more information.

3.3 Dispatching an initial request

(dispatch request site) → any

request : request?

site : site?

Dispatches request to the relevant controller in site. The rules in site are examined in sequence, and the request is dispatched to the controller in the first matching rule found. Default error pages are provided in case no rules match (a 404 response) or no matching define-controller statement is found.

If you are writing a servlet directly you should call dispatch directly from your start procedure:

(define (start initial-request)

(dispatch initial-request my-site))

If you are using the web-server/servlet-env module you should call to dispatch from the procedure you pass to serve/servlet:

(serve/servlet (lambda (initial-request)

(dispatch initial-request my-site)))

(serve/servlet       site+start      [ #:command-line? command-line?       #:launch-browser? launch-browser?       #:quit? quit?       #:banner? banner?       #:listen-ip listen-ip       #:port port       #:ssl? ssl?       #:servlet-path servlet-path       #:servlet-regexp servlet-regexp       #:stateless? stateless?       #:stuffer stuffer       #:manager manager       #:servlet-namespace servlet-namespace       #:server-root-path server-root-path       #:extra-files-paths extra-files-paths       #:servlets-root servlets-root       #:servlet-current-directory servlet-current-directory       #:file-not-found-responder file-not-found-responder       #:mime-types-path mime-types-path       #:log-file log-file       #:log-format log-format])

→ void

site+start : (or/c site? (-> request? response/c))

command-line? : boolean? = #f

launch-browser? : boolean? = (not command-line?)

quit? : boolean? = (not command-line?)

banner? : boolean? = (not command-line?)

listen-ip : (or/c string? #f) = "127.0.0.1"

port : number? = 8000

ssl? : boolean? = #f

servlet-path : string? = "/"

servlet-regexp : regexp? = #rx""

stateless? : boolean? = #f

stuffer : (stuffer/c serializable? bytes?) = default-stuffer

manager   :   manager?       =   (make-threshold-LRU-manager #f (* 1024 1024 64))

servlet-namespace : (listof module-path?) = empty

server-root-path : path-string? = default-server-root-path

extra-files-paths   :   (listof path-string?)       =   (list (build-path server-root-path "htdocs"))

servlets-root   :   path-string?       =   (build-path server-root-path "htdocs")

servlet-current-directory : path-string? = servlets-root

file-not-found-responder   :   (-> request? response/c)       =   dispatch-not-found-responder

mime-types-path : path-string? = ...

log-file : (or/c path-string? #f) = #f

log-format : log-format/c = 'apache-default

A wrapper for the Web Server’s serve/servlet. Quickly configures default server instance. Most arguments are the same as those for serve/servlet, with three notable exceptions:

The start argument from serve/servlet has been replaced with site+start, which can be a procedure from a request to a response or a Dispatch site. If this argument is a site, the web server dispatches incoming requests straight there. If the argument is a procedure, it should call dispatch with an appropriate request and site.

The default values of servlet-path and servlet-regexp have been changed to "" and #rx"" respectively. This causes all requests to be sent to site+servlet. If Dispatch cannot find a matching rule it raises exn:dispatch, passing control back to serve/servlet.

The default value of file-not-found-responder has been replaced with Dispatch’s controller not found message.

3.4 Custom URL pattern arguments

In addition to the arguments described in Standard URL pattern arguments, you can also create your own arguments that capture/serialize arbitrary Scheme values. A pattern argument consists of four things:

• a symbolic name, used when printing the argument;

• a regular expression fragment, used by dispatch to determine whether a URL matches the pattern;

• a decoder procedure, used by dispatch to convert a captured URL fragment into a useful Scheme datum;

• an encoder procedure, used by controller-url to convert a Scheme datum into a URL fragment.

(make-arg name pattern decoder encoder) → arg?

name : symbol?

pattern : string?

decoder : (-> string? any)

encoder : (-> any string?)

Creates a URL pattern argument. name is a symbolic name used in debugging output. pattern is a regular expression fragment written as a string in the pregexp language. decoder and encoder are used to convert between captured URL fragments Scheme values.

When dispatch is trying to match a request against a rule, it uses a regular expression that it assembles from the parts of url clause. For example, consider the form:

  (url "/posts/" (integer-arg) "/" (integer-arg))

Literal strings in the pattern are passed through pregexp-quote to remove the special meanings of any reserved characters. Args are converted to fragments using their pattern fields, which are wrapped in parentheses to enable regular expression capture:

; The pattern of an integer-arg is "[-]?[0-9]+":

(string-append "\\/posts\\/"

(string-append "(" "[-]?[0-9]+" ")")

"\\/"

(string-append "(" "[-]?[0-9]+" ")"))

The whole expression is wrapped in beginning- and end-of-text anchors, and an extra fragment is added to the end of the expression to account for trailing slashes:

(string-append "^"

(string-append "\\/posts\\/"

(string-append "(" "[-]?[0-9]+" ")")

"\\/"

(string-append "(" "[-]?[0-9]+" ")"))

"\\/?$")

The request URL is matched against the final regular expression. If a match is found, the captured substrings are converted into useful values using the decoder procedures of the relevant arguments, and the values are passed as arguments to the controller. If no match is found, dispatch procedures to the next rule in the site.

Conversely, controller-url assembles a URL from the first pattern it finds with the correct controler and arity. It passes the controller arguments through the encoder fields of the relevant pattern args, and assembles a URL from the complete pattern.

As an example, here is an argument that captures co-ordinate strings like "1,2" and converts them to cons cells:

(make-arg 'coord

"[-]?[0-9]+,[-]?[0-9]+"

(lambda (raw)

(define x (string-index raw #\,))

(cons (string->number (substring raw 0 x))

(string->number (substring raw (add1 x)))))

(lambda (pair)

(format "~a,~a" (car pair) (cdr pair))))

3.5 Useful predicates, accessors and mutators

(site? site) → boolean?

site : any

Returns #t if the argument is a site, #f otherwise.

(site-id site) → symbol?

site : site?

Returns a symbolic version of the identifier to which site is bound.

(site-controllers site) → (listof controller?)

site : site?

Returns a listof the controllers that are part of site.

(controller? site) → boolean?

site : any

Returns #t if the argument is a controller, #f otherwise.

(controller-id controller) → symbol?

controller : controller?

Returns a symbolic version of the identifier to which controller is bound.

(controller-site controller) → site?

controller : controller?

Returns the site associated with controller.

(controller-pipeline controller)

→ (listof (request? -> response/c))

controller : controller?

Returns controller’s pipeline, or null if controller has no pipeline. Raises exn:fail:contract if controller has not been initialised with define-controller.

(controller-body controller) → procedure?

controller : controller?

Returns controller’s body procedure. Raises exn:fail:contract if controller has not been initialised with define-controller.

(controller-url controller arg ...) → string?

controller : controller?

arg : any

Returns a host-local URL that, when visited, would result in controller getting called with the specified arguments. Raises exn:fail:dispatching if there is no rule of matching arity associated with controller.

(arg? arg) → boolean?

arg : any

Returns #t if the argument is a URL pattern argument, #f otherwise.

(arg-id arg) → symbol?

arg : arg?

Returns the name of arg, for use in debugging output.

(set-arg-id! arg id) → void?

arg : arg?

id : symbol?

Sets the name of arg to id.

(arg-pattern arg) → string?

arg : arg?

Returns the regular expression fragment of arg.

(set-arg-pattern! arg pattern) → void?

arg : arg?

pattern : string?

Sets the regular expression fragment of arg to pattern, which should be written in the pregexp language and should not contain capturing parentheses or beginning- or end-of-text markers ("^" or "$").

(arg-decoder arg) → (-> string? any)

arg : arg?

Returns the decoder procedure associated with arg.

(set-arg-decoder! arg proc) → void?

arg : arg?

proc : (-> string? any)

Sets the decoder procedure of arg to proc.

(arg-encoder arg) → (-> any string?)

arg : arg?

Returns the encoder procedure associated with arg.

(set-arg-encoder! arg proc) → void?

arg : arg?

proc : (-> any string?)

Sets the encoder procedure of arg to proc.

4 Acknowledgements

Many thanks to the following for their contributions: Jay McCarthy, Karsten Patzwaldt and Noel Welsh.