1 Introduction
2 Getting started
2.1 Installing Whalesong
2.2 Making Standalone .xhtml files with Whalesong
2.3 Using Whalesong functions from Java Script
3 Reference
3.1 The "whalesong" command-line
3.2 build
3.3 get-runtime
3.4 get-javascript
4 The Whalesong language
true
false
pi
e
null
let/ cc
null?
not
eq?
equal?
void
4.1 IO
current-output-port
current-print
write
display
newline
format
printf
fprintf
displayln
4.2 Numeric operations
+
-
*
/
=
add1
sub1
<
<=
>
>=
abs
quotient
remainder
modulo
gcd
lcm
floor
ceiling
round
truncate
numerator
denominator
expt
exp
log
sin
sinh
cos
cosh
tan
asin
acos
atan
sqr
sqrt
integer-sqrt
sgn
make-rectangular
make-polar
real-part
imag-part
angle
magnitude
conjugate
number->string
string->number
pair?
exact?
4.3 List operations
cons
car
cdr
list
length
append
reverse
map
member
4.4 Vector operations
make-vector
vector
vector-length
vector-ref
vector-set!
vector->list
list->vector
5 The Java Script API
alert
body
call-method
$
in-javascript-context?
viewport-width
viewport-height
6 World programming
7 Internals
7.1 Architecture
7.2 parser/ parse-bytecode.rkt
7.3 compiler/ compiler.rkt
7.4 js-assembler/ assemble.rkt
7.5 Values
7.5.1 Numbers
7.5.2 Pairs, NULL, and lists
7.6 Vectors
7.7 Strings
7.8 VOID
7.9 Undefined
7.10 EOF
7.10.1 Boxes
7.10.2 Structures
7.11 Tests
7.12 What’s in js-vm that’s missing from Whalesong?
8 Acknowledgements
Version: 5.1.1

Whalesong: a Racket to JavaScript compiler

Danny Yoo <dyoo@hashcollision.org>

Source code can be found at: https://github.com/dyoo/whalesong. The latest version of this document lives in http://hashcollision.org/whalesong.

Current commit head is 887e986487087532a314400de14a11e7798bfd41.

Warning: this is work in progress!

1 Introduction

Whalesong is a compiler from Racket to JavaScript; it takes Racket programs and translates them so that they can run stand-alone on a user’s web browser. It should allow Racket programs to run with (hopefully!) little modification, and provide access through the foreign-function interface to native JavaScript APIs. The included runtime library supports the numeric tower, an image library, and a framework to program the web in functional event-driven style.

The GitHub source repository to Whalesong can be found at https://github.com/dyoo/whalesong.

Prerequisites: at least Racket 5.1.1, and a Java 1.6 SDK.

2 Getting started

2.1 Installing Whalesong

At the time of this writing, although Whalesong has been deployed to PLaneT, the version on PLaneT is out of date. I’ll be updating the PLaneT package as soon as Whalesong starts to stabilize, but the system as a whole is still in some flux.

You may want to get the latest sources instead of using the version on PLaneT. Doing so requires doing a little bit of manual work. The steps are:

We can check it out of the source repository in GitHub; the repository can be checked out by using git clone. At the command-line, clone the tree with:

 $ git clone git://github.com/dyoo/whalesong.git

This should check the repository in the current directory.

Next, let’s set up a PLaneT development link. Make sure you are in the parent directory that contains the "whalesong" repository, and then run this on your command line:

$ planet link dyoo whalesong.plt 1 0 whalesong

(You may need to adjust the 1 and 0 major/minor numbers a bit to be larger than the latest version that’s on PLaneT at the time.)

Finally, we need to set up Whalesong with raco setup. Here’s how to do this at the command line:

$ raco setup -P dyoo whalesong.plt 1 0

This should compile Whalesong, as well as set up the "whalesong" executable. Any time the source code in "whalesong" changes, we should repeat this raco setup step again.

At this point, you should be able to rung "whalesong" from the command line.

$ ./whalesong

Expected one of the following: [build, get-runtime, get-javascript].

and if this does appear, then Whalesong should be installed successfully.

2.2 Making Standalone .xhtml files with Whalesong

Let’s try making a simple, standalone executable. At the moment, the program must be written in the base language of (planet dyoo/whalesong). This restriction unfortunately prevents arbitrary racket/base programs from compiling at the moment; the developers (namely, dyoo) will be working to remove this restriction as quickly as possible.

Write a "hello.rkt" with the following content

"hello.rkt"

#lang planet dyoo/whalesong
(display "hello world")
(newline)
This program is a regular Racket program, and can be executed normally,

$ racket hello.rkt

hello world

$

However, it can also be packaged with "whalesong".

$ whalesong build hello.rkt

 

$ ls -l hello.xhtml

-rw-rw-r-- 1 dyoo nogroup 692213 Jun  7 18:00 hello.xhtml

Running whalesong build on a Racket program will produce a self-contained ".xhtml" file. If you open this file in your favorite web browser, you should see a triumphant message show on screen.

We can do something slightly more interesting. Let’s write a Whalesong program that accesses the JavaScript DOM. Call this file "dom-play.rkt".

The generated program can be downloaded here: dom-play.xhtml

"dom-play.rkt"

#lang planet dyoo/whalesong
 
;; Uses the JavaScript FFI, which provides bindings for:
;; $ and call
(require (planet dyoo/whalesong/js))
 
;; insert-break: -> void
(define (insert-break)
  (call-method ($ "<br/>") "appendTo" body)
  (void))
 
;; write-message: any -> void
(define (write-message msg)
  (void (call-method (call-method (call-method ($ "<span/>") "text" msg)
                    "css" "white-space" "pre")
              "appendTo"
              body)))
 
;; Set the background green, and show some content
;; on the browser.
(void (call-method body "css" "background-color" "lightgreen"))
(void (call-method ($ "<h1>Hello World</h1>") "appendTo" body))
(write-message "Hello, this is a test!")
(insert-break)
(let loop ([i 0])
  (cond
   [(= i 10)
    (void)]
   [else
    (write-message "iteration ") (write-message i)
    (insert-break)
    (loop (add1 i))]))
This program uses the JQuery API provided by (planet dyoo/whalesong/js), as well as the native JavaScript FFI to produce output on the browser. If w run Whalesong on this program, and view the resulting "dom-play.xhtml" in your web browser, we should see a pale, green page with some output.

2.3 Using Whalesong functions from JavaScript

Whalesong also allows functions defined from Racket to be used from JavaScript. As an example, we can take the boring factorial function and define it in a module called "fact.rkt":

The files can also be downloaded here:
with generated JavaScript binaries here:

"fact.rkt"

#lang planet dyoo/whalesong
(provide fact)
(define (fact x)
  (cond
   [(= x 0)
    1]
   [else
    (* x (fact (sub1 x)))]))

Instead of creating a standalone .xhtml, we can use whalesong to get us the module’s code. From the command-line:

$ whalesong get-javascript fact.rkt > fact.js

$ ls -l fact.js

-rw-r--r-- 1 dyoo dyoo 27421 2011-07-11 22:02 fact.js

This file does require some runtime support not included in "fact.js"; let’s generate the runtime.js and save it as well. At the command-line:

$ whalesong get-runtime > runtime.js

$ ls -l runtime.js

-rw-r--r-- 1 dyoo dyoo 544322 2011-07-11 22:12 runtime.js

Now that we have these, let’s write an "index.html" that uses the fact function that we provideed from "fact.rkt".

"index.html"

<!DOCTYPE html>

<html>

<head>

<script src="runtime.js"></script>

<script src="fact.js"></script>

 

<script>

// Each module compiled with 'whalesong get-runtime' is treated as a

// main module.  invokeMains() will invoke them.

plt.runtime.invokeMains();

 

plt.runtime.ready(function() {

 

   // Grab the definition of 'fact'...

   var myFactClosure = plt.runtime.lookupInMains('fact');

 

   // Make it available as a JavaScript function...

   var myFact = plt.baselib.functions.asJavaScriptFunction(

        myFactClosure);

 

   // And call it!

   myFact(function(v) {

              $('#answer').text(v.toString());

          },

          function(err) {

              $('#answer').text(err.message).css("color", "red");

          },

          10000

          // "one-billion-dollars"

          );

});

</script>

</head>

 

<body>

The factorial of 10000 is <span id="answer">being computed</span>.

</body>

</html>

Replacing the 10000 with "one-billion-dollars" should reliably produce a proper error message.

3 Reference

(This section should describe the whalesong language.)

3.1 The "whalesong" command-line

(This section should describe the whalesong launcher and the options we can use.)

(We want to add JavaScript compression here as an option.)

(We also need an example that shows how to use the get-javascript and get-runtime commands to do something interesting...)

3.2 build

3.3 get-runtime

3.4 get-javascript

4 The Whalesong language

 (require (planet dyoo/whalesong:1:1/lang/base))

This needs to at least show all the bindings available from the base language.

true : boolean
The boolean value #t.
false : boolean
The boolean value #f.
pi : number
The math constant pi.
e : number
The math constant pi.
The empty list value null.

(let/cc id body ...)
(null? ...)
(not ...)
(eq? ...)
(equal? ...)
(void ...)

4.1 IO

(write ...)
(display ...)
(newline ...)
(format ...)
(printf ...)
(fprintf ...)
(displayln ...)

4.2 Numeric operations

(+ ...)
(- ...)
(* ...)
(/ ...)
(= ...)
(add1 ...)
(sub1 ...)
(< ...)
(<= ...)
(> ...)
(>= ...)
(abs ...)
(quotient ...)
(remainder ...)
(modulo ...)
(gcd ...)
(lcm ...)
(floor ...)
(ceiling ...)
(round ...)
(truncate ...)
(numerator ...)
(expt ...)
(exp ...)
(log ...)
(sin ...)
(sinh ...)
(cos ...)
(cosh ...)
(tan ...)
(asin ...)
(acos ...)
(atan ...)
(sqr ...)
(sqrt ...)
(sgn ...)
(make-polar ...)
(real-part ...)
(imag-part ...)
(angle ...)
(magnitude ...)
(conjugate ...)
(pair? ...)
(exact? ...)

4.3 List operations

(cons ...)
(car ...)
(cdr ...)
(list ...)
(length ...)
(append ...)
(reverse ...)
(map ...)
(member ...)

4.4 Vector operations

(vector ...)
(vector-ref ...)

5 The JavaScript API

 (require (planet dyoo/whalesong:1:1/js))
This needs to describe what hooks we’ve got from the JavaScript side of things.
In particular, we need to talk about the plt namespace constructed by the runtime, and the major, external bindings, like plt.runtime.invokeMains.
The contracts here are not quite right either. I want to use JQuery as the type in several of the bindings here, but don’t quite know how to teach Scribble about them yet.

(alert msg)  void
  msg : string?
Displays an alert. Currently implemented using JavaScript’s alert function.

body : any/c
A JQuery-wrapped value representing the body of the DOM.

(call-method object method-name arg ...)  any/c
  object : any/c
  method-name : string?
  arg : any/c
Calls the method of the given object, assuming object is a JavaScript value that supports that method call. The raw return value is passed back.

For example,

(call-method body "css" "background-color")

should return the css color of the body.

($ locator)  any/c
  locator : any/c
Uses JQuery to construct or collect a set of DOM elements, as described in the JQuery documentation.

For example,
(call-method ($ "<h1>Hello World</h1>")
             "appendTo"
             body)
will construct a h1 header, and append it to the document body.

(in-javascript-context?)  boolean
Returns true if the running context supports JavaScript-specific functions.

(viewport-width)  number?
Can only be called in a JavaScript context.

Returns the width of the viewport.

(viewport-height)  number?
Can only be called in a JavaScript context.

Returns the height of the viewport.

6 World programming

Whalesong provides a library to support writing functional I/O programs (A Functional I/O System). Here’s an example of such a world program:

[FIXME: embed a world program here.]

7 Internals

Please skip this section if you’re a regular user: this is really notes internal to Whalesong development, and is not relevant to most people.

These are notes that describe the internal details of the implementation, including the type map from Racket values to JavaScript values. It should also describe how to write FFI bindings, eventually.

7.1 Architecture

The basic idea is to reuse most of the Racket compiler infrastructure. We use the underlying Racket compiler to produce bytecode from Racket source; it also performs macro expansion and module-level optimizations for us. We parse that bytecode using the compiler/zo-parse collection to get an AST, compile that to an intermediate language, and finally assemble JavaScript.

                    AST                 IL

parse-bytecode.rkt -----> compiler.rkt ----> assembler.rkt

 

The IL is intended to be translated straightforwardly. We currently have an assembler to JavaScript "js-assembler/assemble.rkt", as well as a simulator in "simulator/simulator.rkt". The simulator allows us to test the compiler in a controlled environment.

7.2 parser/parse-bytecode.rkt

(We try to insulate against changes in the bytecode structure by using the version-case library to choose a bytecode parser based on the Racket version number. Add more content here as necessary...)

7.3 compiler/compiler.rkt

This translates the AST to the intermediate language. The compiler has its origins in the register compiler in Structure and Interpretation of Computer Programs with some significant modifications.

Since this is a stack machine, we don’t need any of the register-saving infrastructure in the original compiler. We also need to support slightly different linkage structures, since we want to support multiple value contexts. We’re trying to generate code that works effectively on a machine like the one described in http://plt.eecs.northwestern.edu/racket-machine/.

The intermediate language is defined in "il-structs.rkt", and a simulator for the IL in "simulator/simulator.rkt". See "tests/test-simulator.rkt" to see the simulator in action, and "tests/test-compiler.rkt" to see how the output of the compiler can be fed into the simulator.

The assumed machine is a stack machine with the following atomic registers:
  • val: value

  • proc: procedure

  • argcount: number of arguments

and two stack registers:
  • env: environment stack

  • control: control stack

7.4 js-assembler/assemble.rkt

The intent is to potentially support different back end generators for the IL. "js-assembler/assemble.rkt" provides a backend for JavaScript.

The JavaScript assembler plays a few tricks to make things like tail calls work:

Otherwise, the assembler is fairly straightforward. It depends on library functions defined in "runtime-src/runtime.js". As soon as the compiler stabilizes, we will be pulling in the runtime library in Moby Scheme into this project. We are right in the middle of doing this, so expect a lot of flux here.

The assembled output distinguishes between Primitives and Closures. Primitives are only allowed to return single values back, and are not allowed to do any higher-order procedure calls. Closures, on the other hand, have full access to the machine, but they are responsible for calling the continuation and popping off their arguments when they’re finished.

7.5 Values

All values should support the following functions

7.5.1 Numbers

Numbers are represented with the js-numbers JavaScript library. We re-exports it as a plt.baselib.numbers namespace which provides the numeric tower API.

Example uses of the plt.baselib.numbers library include:

Do all arithmetic using the functions in the plt.baselib.numbers namespace. One thing to also remember to do is apply plt.baselib.numbers.toFixnum to any native JavaScript function that expects numbers.

7.5.2 Pairs, NULL, and lists

Pairs can be constructed with plt.runtime.makePair(f, r). A pair is an object with first and rest attributes. They can be tested with plt.runtime.isPair();

The empty list value, plt.runtime.NULL, is a single, distinguished value, so compare it with ===.

Lists can be constructed with plt.runtime.makeList, which can take in multiple arguments. For example,

 var aList = plt.runtime.makeList(3, 4, 5);

constructs a list of three numbers.

The predicate plt.runtime.isList(x) takes an argument x and reports if the value is chain of pairs, terminates by NULL. At the time of this writing, it does NOT check for cycles.

7.6 Vectors

Vectors can be constructed with plt.runtime.makeVector(x ...), which takes in any number of arguments. They can be tested with plt.runtime.isVector, and support the following methods and attributes:
  • ref(n): get the nth element

  • set(n, v): set the nth element with value v

  • length: the length of the vector

7.7 Strings

Immutable strings are represented as regular JavaScript strings.

Mutable strings haven’t been mapped yet.

7.8 VOID

The distinguished void value is plt.runtime.VOID; functions implemented in JavaScript that don’t have a useful return value should return plt.runtime.VOID.

7.9 Undefined

The undefined value is JavaScript’s undefined.

7.10 EOF

The eof object is plt.runtime.EOF

7.10.1 Boxes

Boxes can be constructed with plt.runtime.makeBox(x). They can be tested with plt.runtime.isBox(), and they support two methods:

box.get(): returns the value in the box

box.set(v): replaces the value in the box with v

7.10.2 Structures

structure types can be made with plt.runtime.makeStructureType. For example,

var Color = plt.runtime.makeStructureType(

    'color',    // name

    false,      // parent structure type

    3,          // required number of arguments

    0,          // number of automatically-filled fields

    false,      // OPTIONAL: the auto-v value

    false       // OPTIONAL: a guard procedure

    );

makeStructuretype is meant to mimic the make-struct-type function in Racket. It produces a structure type value with the following methods:
  • constructor: create an instance of a structure type.

    For example,

    var aColor = Color.constructor(3, 4, 5);

    creates an instance of the Color structure type.

  • predicate: test if a value is of the given structure type.

    For example,

    Color.predicate(aColor) --> true

    Color.predicate("red") --> false

  • accessor: access a field of a structure.

    For example,

    var colorRed = function(x) { return Color.accessor(x, 0); };

    var colorGreen = function(x) { return Color.accessor(x, 1); };

    var colorBlue = function(x) { return Color.accessor(x, 2); };

  • mutator: mutate a field of a structure.

    For example,

    var setColorRed = function(x, v) { return Color.mutator(x, 0, v); };

In addition, it has a type whose prototype can be changed in order to add methods to an instance of a structure type. For example,

Color.type.prototype.toString = function() {

    return "rgb(" + colorRed(this) + ", "

                  + colorGreen(this) + ", "

                  + colorBlue(this) + ")";

};

should add a toString method for instances of the Color structure.

7.11 Tests

The test suite in "tests/test-all.rkt" runs the test suite. You’ll need to run this on a system with a web browser, as the suite will evaluate JavaScript and make sure it is producing values. A bridge module in "tests/browser-evaluate.rkt" brings up a temporary web server that allows us to pass values between Racket and the JavaScript evaluator on the browser for testing output.

7.12 What’s in js-vm that’s missing from Whalesong?

(This section should describe what needs to get done next.)

The only types that are mapped so far are
  • immutable strings

  • numbers

  • pairs

  • null

  • void

  • vectors

We need to bring around the following types previously defined in js-vm: (This list will shrink as I get to the work!)
  • immutable vectors

  • regexp

  • byteRegexp

  • character

  • placeholder

  • path

  • bytes

  • immutable bytes

  • keywords

  • hash

  • hasheq

  • struct types

  • exceptions

  • thread cells

  • big bang info

  • worldConfig

  • effectType

  • renderEffectType

  • readerGraph

)

What are the list of primitives in "js-vm-primitives.js" that we haven’t yet exposed in whalesong? We’re missing 221:
  • abort-current-continuation

  • andmap

  • append

  • apply

  • argmax

  • argmin

  • arity-at-least-value

  • arity-at-least?

  • assoc

  • assq

  • assv

  • boolean=?

  • boolean?

  • box-immutable

  • box?

  • build-list

  • build-string

  • build-vector

  • byte?

  • bytes

  • bytes->immutable-bytes

  • bytes->list

  • bytes-append

  • bytes-copy

  • bytes-fill!

  • bytes-length

  • bytes-ref

  • bytes-set!

  • bytes<?

  • bytes=?

  • bytes>?

  • bytes?

  • caaar

  • caadr

  • caar

  • cadar

  • cadddr

  • caddr

  • cadr

  • call-with-continuation-prompt

  • call-with-current-continuation

  • call-with-values

  • call/cc

  • cdaar

  • cdadr

  • cdar

  • cddar

  • cdddr

  • cddr

  • char->integer

  • char-alphabetic?

  • char-ci<=?

  • char-ci<?

  • char-ci=?

  • char-ci>=?

  • char-ci>?

  • char-downcase

  • char-lower-case?

  • char-numeric?

  • char-upcase

  • char-upper-case?

  • char-whitespace?

  • char<=?

  • char<?

  • char=?

  • char>=?

  • char>?

  • char?

  • complex?

  • compose

  • cons?

  • continuation-mark-set->list

  • continuation-mark-set?

  • continuation-prompt-tag?

  • current-continuation-marks

  • current-inexact-milliseconds

  • current-seconds

  • default-continuation-prompt-tag

  • e

  • eighth

  • empty

  • empty?

  • eof

  • eof-object?

  • equal~?

  • eqv?

  • error

  • even?

  • exact->inexact

  • exn-continuation-marks

  • exn-message

  • exn:fail:contract:arity?

  • exn:fail:contract:divide-by-zero?

  • exn:fail:contract:variable?

  • exn:fail:contract?

  • exn:fail?

  • exn?

  • explode

  • false

  • false?

  • fifth

  • filter

  • first

  • foldl

  • foldr

  • for-each

  • fourth

  • gensym

  • hash-for-each

  • hash-map

  • hash-ref

  • hash-remove!

  • hash-set!

  • hash?

  • identity

  • immutable?

  • implode

  • inexact->exact

  • inexact?

  • int->string

  • integer->char

  • js-function?

  • js-object?

  • js-value?

  • length

  • list*

  • list->bytes

  • list->string

  • list-tail

  • list?

  • make-arity-at-least

  • make-bytes

  • make-continuation-prompt-tag

  • make-exn

  • make-exn:fail

  • make-exn:fail:contract

  • make-exn:fail:contract:arity

  • make-exn:fail:contract:divide-by-zero

  • make-exn:fail:contract:variable

  • make-hash

  • make-hasheq

  • make-placeholder

  • make-reader-graph

  • make-string

  • make-struct-type

  • make-thread-cell

  • map

  • max

  • memf

  • memq

  • memv

  • min

  • negative?

  • null

  • number?

  • odd?

  • ormap

  • pi

  • placeholder-get

  • placeholder-set!

  • positive?

  • posn?

  • print-values

  • procedure-arity

  • procedure-arity-includes?

  • procedure?

  • quicksort

  • raise

  • rational?

  • real?

  • remove

  • replicate

  • rest

  • second

  • seventh

  • sixth

  • sleep

  • sort

  • string

  • string->immutable-string

  • string->int

  • string->list

  • string-alphabetic?

  • string-ci<=?

  • string-ci<?

  • string-ci=?

  • string-ci>=?

  • string-ci>?

  • string-copy

  • string-fill!

  • string-ith

  • string-lower-case?

  • string-numeric?

  • string-ref

  • string-set!

  • string-upper-case?

  • string-whitespace?

  • string<=?

  • string<?

  • string>=?

  • string>?

  • string?

  • struct-accessor-procedure?

  • struct-constructor-procedure?

  • struct-mutator-procedure?

  • struct-predicate-procedure?

  • struct-type?

  • struct?

  • subbytes

  • substring

  • symbol=?

  • third

  • throw-cond-exhausted-error

  • true

  • undefined?

  • values

  • vector?

  • verify-boolean-branch-value

  • void?

  • write

  • xml->s-exp

(I should catalog the bug list in GitHub, as well as the feature list, so I have a better idea of what’s needed to complete the project.)

(We also need a list of the primitives missing that prevent us from running racket/base; it’s actually a short list that I’ll be attacking once things stabilize.)

8 Acknowledgements

Whalesong uses code and utilities from the following external projects:

The following folks have helped tremendously in the implementation of Whalesong by implementing libraries, giving guidence, and suggesting improvements: