Whalesong: a Racket to JavaScript compiler
Danny Yoo <dyoo@hashcollision.org>
Source code can be found at: https://github.com/dyoo/whalesong. The latest version of this document lives in http://hashcollision.org/whalesong.
1 Introduction
Whalesong is a compiler from Racket to JavaScript; it takes Racket programs and translates them so that they can run stand-alone on a user’s web browser. It should allow Racket programs to run with (hopefully!) little modification, and provide access through the foreign-function interface to native JavaScript APIs. The included runtime library supports the numeric tower, an image library, and a framework to program the web in functional event-driven style.
The GitHub source repository to Whalesong can be found at https://github.com/dyoo/whalesong. If you have any questions or comments, please feel free to use the Racket-users mailing list.
Prerequisites: at least Racket 5.1.1. If you wish to use the JavaScript compression option, you will need Java 1.6 SDK.
1.1 Examples
raphael-demo.html [src] Uses features of the JavaScript FFI to access the RaphaelJS vector graphics library.
google-maps.html [src library src] Uses features of the JavaScript FFI (js-function->procedure, js-async-function->procedure, make-world-event-handler) to treat a Google Maps instance as a source of events for a World program. Click on the map, and the coordinate stored in the world should synchronize with the clicked location.
attr-animation.html [src view.html style.css] Uses update-view-attr and on-tick to perform a simple color animation.
color-buttons.html [src view.html] Uses view-bind-many to bind several events at once. Clicking on a button should change the color of the header by adjusting its CSS color attribute.
boid.html [src view.html] Uses update-view-css and on-tick to perform an animation of a flock of boids.
dwarves.html [src view.html] Uses view-show and view-hide to manipulate a view. Click on a dwarf to make them hide.
dwarves-with-remove.html [src view.html] Uses view-focus? and view-remove to see if a dwarf should be removed from the view.
field.html [src view.html] Uses view-bind to read a text field, and update-view-text to change the text content of an element.
phases.html [src view1.html view2.html] Switches out one view entirely in place of another. Different views can correspond to phases in a program.
tick-tock.html [src view.html] Uses on-tick to show a timer counting up.
redirected.html [src view.html] Uses on-tick to show a timer counting up, and also uses open-output-element to pipe side-effecting printfs to a hidden div.
where-am-i.html [src view.html] Uses on-location-change and on-mock-location-change to demonstrate location services.
hot-cross-buns.html [src view.html] Demonstrates use of checkboxes. Uses view-has-attr? to see if a checkbox has been checked, and remove-view-attr to change the checked attribute when the user wants to reset the page.
rain.html [src] Uses the image libraries to show droplets of water falling down.
pacman.html [src] Pacman.
2 Getting started
2.1 Installing Whalesong
Please go to the Racket submenu.
Select the Limit Memory item.
Change the setting to Unlimited.
#lang racket/base (require (planet dyoo/whalesong:1:21/make-launcher))
This may take a few minutes, as Racket is compiling Whalesong, its dependencies, and its documentation. When it finally finishes, you should see a "whalesong" launcher in the current directory.
You should also see a "whalesong-gui" launcher that includes a minimal graphical user interface.
$ ./whalesong |
Usage: whalesong <subcommand> [option ...] <arg ...> |
where any unambiguous prefix can be used for a subcommand |
|
The Whalesong command-line tool for compiling Racket to JavaScript |
|
For help on a particular subcommand, use 'whalesong <subcommand> --help' |
whalesong build build a standalone html and javascript package |
whalesong get-runtime print the runtime library to standard output |
whalesong get-javascript Gets just the JavaScript code and prints it to standard output |
To repeat: whenever Whalesong’s source code is updated from Github, please re-run the raco setup step. Otherwise, Racket will try to recompile Whalesong on every single use, which can be very expensive.
2.2 Making .html files with Whalesong
Let’s try making a simple, standalone executable. At the moment, the program must be written in the base language of (planet dyoo/whalesong). This restriction unfortunately prevents arbitrary racket/base programs from compiling at the moment; the developers (namely, dyoo) will be working to remove this restriction as quickly as possible.
"hello.rkt"
#lang planet dyoo/whalesong (display "hello world") (newline)
$ racket hello.rkt |
hello world |
$ |
$ whalesong build hello.rkt |
Writing program #<path:/home/dyoo/work/whalesong/examples/hello.js> |
Writing html #<path:/home/dyoo/work/whalesong/examples/hello.html> |
|
$ ls -l hello.html |
-rw-r--r-- 1 dyoo dyoo 3817 2011-09-10 15:02 hello.html |
$ ls -l hello.js |
-rw-r--r-- 1 dyoo dyoo 841948 2011-09-10 15:02 hello.js |
|
Visit hello.html to execute this program.
Visit dom-play.html to execute this program.
"dom-play.rkt"
#lang planet dyoo/whalesong ;; Uses the JavaScript FFI, which provides bindings for: ;; $ and call-method (require (planet dyoo/whalesong/js)) ;; insert-break: -> void (define (insert-break) (call-method ($ "<br/>") "appendTo" body) (void)) ;; write-message: any -> void (define (write-message msg) (void (call-method (call-method (call-method ($ "<span/>") "text" msg) "css" "white-space" "pre") "appendTo" body))) ;; Set the background green, and show some content ;; on the browser. (void (call-method body "css" "background-color" "lightgreen")) (void (call-method ($ "<h1>Hello World</h1>") "appendTo" body)) (write-message "Hello, this is a test!") (insert-break) (let loop ([i 0]) (cond [(= i 10) (void)] [else (write-message "iteration ") (write-message i) (insert-break) (loop (add1 i))]))
2.3 Using Whalesong functions from JavaScript
Whalesong also allows functions defined from Racket to be used from JavaScript. As an example, we can take the boring factorial function and define it in a module called "fact.rkt":
The files can also be downloaded here:with generated JavaScript binaries here:
"fact.rkt"
#lang planet dyoo/whalesong (provide fact) (define (fact x) (cond [(= x 0) 1] [else (* x (fact (sub1 x)))]))
$ whalesong get-javascript fact.rkt > fact.js |
$ ls -l fact.js |
-rw-r--r-- 1 dyoo dyoo 27421 2011-07-11 22:02 fact.js |
$ whalesong get-runtime > runtime.js |
$ ls -l runtime.js |
-rw-r--r-- 1 dyoo dyoo 544322 2011-07-11 22:12 runtime.js |
"index.html"
<!DOCTYPE html>
<html>
<head>
<script src="runtime.js"></script>
<script src="fact.js"></script>
<script>
// Each module compiled with 'whalesong get-runtime' is treated as a
// main module. invokeMains() will invoke them.
plt.runtime.invokeMains();
plt.runtime.ready(function() {
// Grab the definition of 'fact'...
var myFactClosure = plt.runtime.lookupInMains('fact');
// Make it available as a JavaScript function...
var myFact = plt.baselib.functions.asJavaScriptFunction(
myFactClosure);
// And call it!
myFact(function(v) {
$('#answer').text(v.toString());
},
function(err) {
$('#answer').text(err.message).css("color", "red");
},
10000
// "one-billion-dollars"
);
});
</script>
</head>
<body>
The factorial of 10000 is <span id="answer">being computed</span>.
</body>
</html>
See: bad-index.html.
3 Using whalesong
Whalesong provides a command-line utility called whalesong for translating Racket to JavaScript. It can be run in several modes:
To create HTML + js documents
To output the compiled JavaScript as a single ".js" file
$ whalesong build [name-of-racket-file] |
The whalesong commands support these command line options:
--compress-javascript
Use Google Closure’s JavaScript compiler to significantly compress the JavaScript. Using this currently requires a Java 1.6 JDK.--verbose
Write verbose debugging information to standard error.--dest-dir
Write files to a separate directory, rather than the current directory.--split-modules
Write each dependent module as a separate file, rather than in one large ".js". This may be necessary if your browser environment prohibits large ".js" files. The files will be numbered starting from 1.
For more advanced users, whalesong can be used to generate JavaScript in non-standalone mode. This gives the web developer more fine-grained control over how to control and deploy the outputted program.
3.1 build
Given the name of a program, this builds ".html" and ".js" files into the current working directory.
The ".html" and ".js" should be self-contained, with an exception: if the file uses any external resources by using define-resource, those resources are written into the current working directory, if they do not already exist there.
3.2 get-javascript
Given the name of a program, writes the JavaScript to standard output, as well as its dependent modules. The outputted file is meant to be used as a SCRIPT source.
By default, the given program will be treated as a main module. All main modules will be executed when the JavaScript function plt.runtime.invokeMains() is called.
3.3 get-runtime
Prints out the core runtime library that the files generated by get-javascript depend on.
4 Including external resources
(require (planet dyoo/whalesong:1:=21/resource)) |
Programs may need to use external file resources that aren’t themselves Racket programs, but instead some other kind of data. Graphical programs will often use ".png"s, and web-related programs ".html"s, for example. Whalesong provides the (planet dyoo/whalesong:1:=21/resource) library to refer and use these external resources. When Whalesong compiles a program into a package, these resources will be bundled alongside the JavaScript-compiled output.
syntax
(define-resource id [path-string])
#lang planet dyoo/whalesong (require (planet dyoo/whalesong/resource)) (define-resource my-whale-image-resource "humpback.png")
#lang planet dyoo/whalesong (require (planet dyoo/whalesong/resource)) (define-resource humpback.png)
".png"
".gif"
".jpg"
".jpeg"
If the resource has the extension ".html", then it will be run through an HTML purifying process to make sure the HTML is well-formed.
procedure
(resource? x) → boolean
x : any
procedure
(resource->url a-resource) → string?
a-resource : resource?
#lang planet dyoo/whalesong (require (planet dyoo/whalesong/resource) (planet dyoo/whalesong/image)) (define-resource my-whale-image-resource "humpback.png") (define WHALE-IMAGE (bitmap/url (resource->url my-whale-image-resource)))
5 The web-world API
(require (planet dyoo/whalesong:1:=21/web-world)) |
The web-world library allows you to write functional event-driven World programs for the web; the user defines functional callbacks to handle events, and receive and consume a world argument.
One difference introduced by the web is the web page itself: because the page itself is a source of state, it too will be passed to callbacks. This library presents a functional version of the DOM in the form of a view.
Visit tick-tock.html to execute this program.
"view.html"
<html>
<head><title>My simple program</title></head>
<body>
<p>The current counter is: <span id="counter">fill-me-in</span></p>
</body>
</html>
"tick-tock.rkt"
#lang planet dyoo/whalesong (require (planet dyoo/whalesong/web-world) (planet dyoo/whalesong/resource)) (define-resource view.html) ;; draw: world view -> view (define (draw world dom) (update-view-text (view-focus dom "counter") world)) ;; tick: world view -> world (define (tick world dom) (add1 world)) ;; stop?: world view -> boolean (define (stop? world dom) (> world 10)) (big-bang 0 (initial-view view.html) (to-draw draw) (on-tick tick 1) (stop-when stop?))
We require a few libraries to get us some additional behavior; in particular, (planet dyoo/whalesong:1:=21/web-world) to let us write event-driven web-based programs, and (planet dyoo/whalesong:1:=21/resource) to give us access to external resources.
We use define-resource to refer to external files, like "view.html" that we’d like to include in our program.
We use big-bang to start up a computation that responses to events. In this example, that’s clock ticks introduced by on-tick, though because we’re on the web, we can bind to many other kinds of web events (by using view-bind).
5.1 big-bang and its options
procedure
(big-bang w h ...) → world
w : world h : big-bang-handler
procedure
(initial-view x) → big-bang-handler
x : any
... (define-resource page1.html) ... (big-bang ... (initial-view page1.html))
procedure
(stop-when stop?) → big-bang-handler
stop? : ([w world] [dom view] -> boolean)
... (define-struct world (given expected)) ... ;; stop?: world view -> boolean (define (stop? world dom) (string=? (world-given world) (world-expected world))) (big-bang ... (stop-when stop?))
procedure
(on-tick tick-f delay) → big-bang-handler
tick-f : ([w world] [v view] [e event]? -> world) delay : real (on-tick tick-f) → big-bang-handler tick-f : ([w world] [v view] [e event]? -> world)
... ;; tick: world dom -> world (define (tick world view) (add1 world)) (big-bang ... (on-tick tick 5)) ;; tick every five seconds
procedure
(on-mock-location-change location-f) → big-bang-handler
location-f : ([w world] [v view] [e event]? -> world)
During the extent of a big-bang, a form widget will appear in the document.body to allow us to manually send location-changing events.
... ;; move: world view event -> world (define (move world dom event) (list (event-ref event "latitude") (event-ref event "longitude"))) ... (big-bang ... (on-mock-location-change move))
procedure
(on-location-change location-f) → big-bang-handler
location-f : ([w world] [v view] [e event]? -> world)
... ;; move: world view event -> world (define (move world dom event) (list (event-ref event "latitude") (event-ref event "longitude"))) ... (big-bang ... (on-location-change move))
procedure
(to-draw draw-f) → big-bang-handler
draw-f : ([w world] [v view] -> view)
... (define-struct world (name age)) ;; draw: world view -> view (define (draw world dom) (update-view-text (view-focus dom "name-span") (world-name world))) ... (big-bang ... (to-draw draw))
5.2 Views
A view is a functional representation of the browser DOM tree. A view is always focused on an element, and the functions in this subsection show how to traverse and manipulate the view.
procedure
(->view x) → view
x : any
procedure
(view-focus? v id) → boolean
v : view id : String
procedure
(view-focus v id) → view
v : view id : String
procedure
(view-left? v) → boolean
v : view
procedure
(view-left v) → view
v : view
procedure
(view-right? v) → boolean
v : view
procedure
(view-right v) → view
v : view
procedure
(view-up? v) → boolean
v : view
procedure
(view-up v) → view
v : view
procedure
(view-down? v) → boolean
v : view
procedure
(view-down v) → view
v : view
procedure
(view-forward? v) → boolean
v : view
procedure
(view-forward v) → view
v : view
procedure
(view-backward? v) → boolean
v : view
procedure
(view-backward v) → view
v : view
procedure
(view-text v) → string
v : view
procedure
(update-view-text v s) → view
v : view s : string
procedure
(view-bind v type world-updater) → view
v : view type : string world-updater : ([w world] [v view] [e event]? -> world)
Attach a world-updating event to the focus. When the world-updater is called, the view will be focused on the element that triggered the event.
Common event types include "click", "mouseenter", "change". Note that the name of the event should not include an "on" prefix.
syntax
(view-bind-many a-view [id type world-updater] ...)
Common event types include "click", "mouseenter", or "change". Note that the name of each event should not include an "on" prefix.
(define (click-handler w v) ...) (define (change-handler w v) ...) (define-resource view.html) (define my-static-view (->view view.html)) (define connected-view (view-bind-many my-static-view ["id1" "click" click-handler] ["id2" "click" click-handler] ["id3" "change" change-handler])) ...
If the collection of ids, types, and handlers can’t be represented as a static list, then view-bind-many* is an alternate helper function that may be helpful to bind a bulk number of handlers to a view.
procedure
(view-bind-many* v id+type+updater-list) → view
v : view id+type+updater-list : (listof (list string string world-updater))
Common event types include "click", "mouseenter", or "change". Note that the name of each event should not include an "on" prefix.
(define (click-handler w v) ...) (define (change-handler w v) ...) (define-resource view.html) (define my-static-view (->view view.html)) (define connected-view (view-bind-many* my-static-view `(["id1" "click" ,click-handler] ["id2" "click" ,click-handler] ["id3" "change" ,change-handler]))) ...
procedure
(view-show v) → view
v : view
procedure
(view-hide v) → view
v : view
procedure
(view-attr v name) → view
v : view name : String
procedure
(view-has-attr? v name) → boolean
v : view name : String
procedure
(update-view-attr v name value) → view
v : view name : String value : String
procedure
(remove-view-attr v name) → view
v : view name : String
procedure
(view-width v) → number
v : view
procedure
(view-height v) → number
v : view
procedure
(view-css v name) → view
v : view name : String
procedure
(update-view-css v name value) → view
v : view name : String value : String
procedure
(view-id v) → world
v : view
procedure
(view-form-value v) → view
v : view
procedure
(update-view-form-value v value) → view
v : view value : String
Dom nodes can be created by using xexp->dom, which converts a xexp to a node, and attached to the view by using view-append-child, view-insert-left, and view-insert-right.
procedure
(view-append-child v d) → view
v : view d : dom
procedure
(view-insert-left v d) → view
v : view d : dom
procedure
(view-insert-right v d) → view
v : view d : dom
procedure
(view-remove v) → view
v : view
5.3 Events
An event is a structure that holds name-value pairs. Whenever an event occurs in web-world, it may include some auxiliary information about the event. As a concrete example, location events from on-location-change and on-mock-location-change can send latitude and longitude values, as long as the world callback can accept the event as an argument.
struct
(struct event (kvpairs) #:extra-constructor-name make-event) kvpairs : (listof (list symbol (or/c string number)))
procedure
(event-keys evt) → (listof symbol)
evt : event?
5.4 Dynamic DOM generation with xexps
#lang planet dyoo/whalesong (require (planet dyoo/whalesong/web-world)) ;; tick: world view -> world (define (tick world view) (add1 world)) ;; draw: world view -> view (define (draw world view) (view-append-child view (xexp->dom `(p "hello, can you see this? " ,(number->string world))))) (big-bang 0 (initial-view (xexp->dom '(html (head) (body)))) (on-tick tick 1) (to-draw draw))
Normally, we’ll want to do as much of the statics as possible with ".html" resources, but when nothing else will do, we can generate DOM nodes programmatically.
We can create new DOMs from an xexp, which is a s-expression representation for a DOM node. Here are examples of expressions that evaluate to xexps:
"hello world"
'(p "hello, this" "is an item")
(local [(define name "josh")] `(p "hello" (i ,name)))
'(div (@ (id "my-div-0")) (span "This is a span in a div"))
`(div (@ ,(fresh-id)) (span "This is another span in a div whose id is dynamically generated"))
| ‹xexp› | ::= | ‹string› |
|
| | | ‹symbol› |
|
| | | ( ‹id› ‹xexp›* ) |
|
| | | ( ‹id› ( @ ‹key-value›* ) ‹xexp›* ) |
| ‹key-value› | ::= | ( ‹symbol› ‹string› ) |
procedure
(xexp? x) → boolean
x : any
procedure
(xexp->dom an-xexp) → dom
an-xexp : xexp
procedure
(fresh-id) → string
procedure
(view->xexp a-view) → xexp
a-view : view
5.5 Tips and tricks: Hiding standard output or directing it to an element
For a web-world program, output is normally done by using to-draw. However, side effecting functions, such as printf or display, are still available, and will append to document.body.
We may want to disable such printing or redirect it to a particular element on the page. For such purposes, use a combination of current-output-port and open-output-element to redirect the output of these side effect functions to somewhere else.
... ;; Redirect standard output to a div called "stdout-div". (current-output-port (open-output-element "stdout-div")) ... (big-bang ... (on-tick (lambda (world dom) (printf "Tick!\n") (add1 world))) ...)
All subsequent I/O side effects after the call to current-output-port will be written out to the stdout-div, which can be easily styled with display: none to hide it from normal browser display.
procedure
(open-output-element id) → output-port
id : string
6 The JavaScript Foreign Function Interface
procedure
(js-string? x) → boolean
x : any
procedure
(string->js-string str) → js-string
str : string
procedure
(js-string->string js-str) → string
js-str : js-string
procedure
(js-number? x) → boolean
x : any
procedure
(number->js-number num) → js-number
num : number
procedure
(js-number->number js-num) → number
js-num : js-number
procedure
(js-null? x) → boolean
x : any
value
js-null : js-value
procedure
(get-attr obj key) → js-value
obj : js-object key : (or/c string symbol)
procedure
(set-attr! obj key value) → js-value
obj : js-object key : (or/c string symbol) value : js-value
procedure
(load-script url) → void
url : string?
procedure
(js-function->procedure f) → procedure
f : (U string js-function)
(define primitive-* (js-function->procedure "function(x) { return x * x; }")) (primitive-* 42)
procedure
(js-async-function->procedure f) → procedure
f : (or/c string js-function)
(define primitive-* (js-async-function->procedure "function(success, fail, x) { success(x * x); }")) (primitive-* 42)
(define js-sleep (js-async-function->procedure "function(success, fail, x) { setTimeout(success, x); }")) (js-sleep 1000)
procedure
(alert msg) → void
msg : string?
value
body : any/c
procedure
(call-method object method-name arg ...) → any/c
object : any/c method-name : string? arg : any/c
procedure
($ locator) → any/c
locator : any/c
(call-method ($ "<h1>Hello World</h1>") "appendTo" body)
procedure
(in-javascript-context?) → boolean
procedure
(viewport-width) → number?
procedure
(viewport-height) → number?
6.1 Adding new event handlers to world programs with the FFI
(require (planet dyoo/whalesong:1:=21/js/world)) |
The callback-driven asynchronous APIs in JavaScript can act as sources of World events. The following function allows web-world programs to bind to these sources.
procedure
(make-world-event-handler setup shutdown) → event-handler
setup : procedure shutdown : procedure
The setup and shutdown functions are usually constructed with js-function->procedure in order to bind to native JavaScript APIs.
The setup function is called with an JavaScript function value that, when called, emits a new event into the world’s event loop. The return value of the setup function will be saved, and when the shutdown procedure calls, that value is passed to it, with the intent that shutting down a service will likely require information that’s produced at setup-time.
#lang planet dyoo/whalesong (require (planet dyoo/whalesong/js) (planet dyoo/whalesong/js/world)) (define setup-geo (js-function->procedure "function (locationCallback) { return navigator.geolocation.watchPosition( function(evt) { var coords = evt.coords; locationCallback(plt.runtime.makeFloat(coords.latitude), plt.runtime.makeFloat(coords.longitude)); })}")) (define shutdown-geo (js-function->procedure "function (watchId) { navigator.geolocation.clearWatch(watchId); }")) ;; We create a new event handler type here. (define on-geo (make-world-event-handler setup-geo shutdown-geo)) ;; Once defined, we can use on-geo as just another world-handler type. (define (move world view lat lng) (list lat lng)) (big-bang (list 'undefined 'undefined) (on-geo move))
7 Simple world programming
Whalesong provides a library to support writing functional I/O programs (A Functional I/O System). Here’s an example of such a world program:
[FIXME: embed a world program here.]
8 Acknowledgements
jshashtable (http://www.timdown.co.uk/jshashtable/)
js-numbers (http://github.com/dyoo/js-numbers/)
JSON (http://www.json.org/js.html)
jquery (http://jquery.com/)
Google Closure Compiler (http://code.google.com/p/closure-compiler/)
Base64 encode (http://www.webtoolkit.info/)
excanvas (http://excanvas.sourceforge.net/)
canvas-text (http://code.google.com/p/canvas-text/source/browse/trunk)
The following folks have helped tremendously in the implementation of Whalesong by implementing libraries, giving guidence, reporting bugs, and suggesting improvements.
Asumu Takikawa
Cristina Teodoropol
Doug Orleans
Emmanuel Schanzer
Eric Hanchrow
Ethan Cecchetti
Greg Hendershott
Gregor Kiczales
Jay McCarthy
Jens Axel Søgaard
Keith Decker
Matthew Flatt
Richard Cleis
Robby Findler
Sam Tobin-Hochstadt
Scott Newman
Shriram Krishnamurthi
Zhe Zhang