8 Scribble’s Extensibility
Scribble’s foundation on PLT Scheme empowers programmers to implement a number of features as libraries that ordinarily must be built into a documentation tool. More importantly, users can experiment with new and more interesting ways to write documentation without having to modify Scribble’s implementation.
In this section, we describe several libraries that we have already built atop Scribble: for stand-alone API documentation, for automatically running examples when building documentation, for combining code with documentation in the style of JavaDoc, and for literate programing.
8.1 API Specification
Targets for code hyperlinks are defined by defproc (for functions), defform (for syntactic forms), defstruct (for structure types), defclass (for classes in the object system), and other such forms – one for each form of binding. When a library defines a new form of binding, an associated documentation library can define a new form for documenting the bindings.
(louder str) → string? str : string? Adds “!” to the end of str.
For the above defproc, the for-label binding of louder partly determines the library binding that is documented by this defproc form. A single binding, however, can be re-exported by many modules. On the reference side, the scheme and schemeblock forms follow re-export chains to discover the first exporting module for which a binding is documented; on the definition side, defproc needs a declaration of the module that is being documented. The module declaration is no extra burden on the document author, because the reader of the document needs some indication of which module is being documented.
@(require scribble/manual |
(for-label scheme/base |
comics/string)) |
|
@title{String Manipulations} |
|
@defmodule[comics/string] |
|
Adds “!” to the end of @scheme[str]. |
} |
The defproc form is implemented by a scribble/manual layer of Scribble, which provides many functions and forms for typesetting PLT Scheme documentation. The scribble/manual layer is separate from the core Scribble engine, however, and other libraries can build up defproc-like abstractions on top of the core typesetting and cross-referencing capabilities described in Core Scribble Datatypes.
8.2 Examples and Tests
In the documentation for a function or syntactic form, concrete examples help a reader understand how a function works, but only if the examples are reliable. Ensuring that examples are correct is a significant burden in a conventional approach to documentation, because the example expressions must be carefully checked against the implementation (often by manual cut and paste), and a small edit can easily introduce a bug.
Examples:
> (/ 1 2) 1/2
> (/ 1 2.0) 0.5
> (/ 1 +inf.0) 0.0
Example:
> (/ 1 +infinity.0) reference to undefined identifier: +infinity.0
Evaluation of example code mingles two phases that we have otherwise worked to keep separate: the time at which a library is executed, and the time at which its documentation is produced. For simple functional expressions, such as (/ 1 2), the separation does not matter, and examples could simply duplicate its argument in both an expression position and a typeset position. More generally, however, examples involve temporary definitions and side-effects. To prevent examples from interfering with each other while building a large document, examples uses a sandboxed environment, for which PLT Scheme provides extensive support (Flatt et al. 1999; Flatt and PLT Scheme 2009, §13).
8.3 In-Code Documentation
For some libraries, the programmer may want to write documentation with the source instead of in a separate document. To support such documentation, we have created a Scheme/Scribble extension that is used to document some libraries in the PLT Scheme distribution.
(require scheme/contract |
scribble/srcdoc) |
(require/doc scheme/base |
scribble/manual) |
|
(define (louder s) |
(string-append s "!")) |
|
[louder |
@{Adds “!” to the end of @scheme[str].}]) |
The #lang at-exp scheme/base line declares that the module uses scheme/base language extended with @-notation. The imported scribble/srcdoc library binds require/doc and provide/doc. The require/doc form imports bindings into a “documentation” phase, such as the scheme form that is used in the description of louder. The provide/doc form exports louder, annotates it with a contract for run-time checking, and records the contract and description for inclusion in documentation. The description is an expression in the documentation phase; it is dropped by normal compilation of the module, but combined with the require/doc imports and inferred (require (for-label ...)) imports to generate the module’s documentation.
@(require scribble/manual |
scribble/extract |
(for-label comics/string)) |
|
@title{Strings} |
|
@defmodule[comics/string] |
|
The @schememodname[comics/string] library |
provides functions for creating punchlines. |
|
@include-extracted[comics/string] |
An advantage of using scribble/srcdoc and scribble/extract is that the description of the function is with the implementation, and the function contract need not be duplicated in the source and documentation. Similarly, the fact that string? in the contract gets its binding from scheme/base is specified once in the code and inferred for the documentation. At the same time, a phase separation prevents document-generating expressions from polluting the library’s run-time execution, and vice versa.
8.4 Literate Programming
The techniques used for in-source documentation extend to the creation of WEB-like literate programming tools. Figure 3 shows an example use of our literate-programming library; the left-hand side shows a screenshot of DrScheme editing the source code for a short, literate discussion of the Collatz conjecture, while the right-hand side shows the rendered output.
Unlike a normal Scribble program, running a scribble/lp program ignores the prose exposition and instead evaluates the program in the chunks. In literate programming terminology, this process is called tangling the program. Thus, to a client module, a literate program behaves just like its illiterate variant. The compiled form of a literate program does not contain any of the documentation, nor does it depend on the runtime support for Scribble, just as if an extra-linguistic tangler had been used. Consequently, the literate implementation suffers no overhead due to the prose.
@lp-include[filename] |
Both weaving and tangling with scribble/lp work at the level of syntactic extensions, and not in terms of manipulating source text. As a result, the language for writing prose is extensible, because Scribble libraries such as scribble/manual can be imported into the document. The language for implementing the program is also obviously extensible, because a chunk can include imports from other PLT Scheme libraries. Finally, even the bridge between the prose and the implementation is extensible, because the document author can create new syntactic forms that expand to a mixture of prose, implementation, and uses of chunk.
Tangling via syntactic extension also enables many tools for Scheme programs to automatically apply to literate Scheme programs. The arrows in Figure 3’s screenshot demonstrate how DrScheme can draw arrows from chunk bindings to chunk references, and from the binding occurrence of an identifier to its bound occurrences, even across chunks. These latter arrows are particularly helpful with literate programs, where lexical scope is sometimes obscured by the way that textually disparate fragments of a program are eventually tangled into the same scope. DrScheme’s interactive REPL, test-case coverage support, module browser, executable generation, and other tools also work on literate programs.
To gain some experience with non-trivial literate programming in Scribble, we have written a 34-page literate program that describes our implementation of the Chat Noir game, which is distributed with PLT Scheme. The source is included in the distribution as "chat-noir-literate.ss", and the rendered output is in the help system and online at http://docs.plt-scheme.org/games/chat-noir.html.
Consider a function that, starting from (collatz n), recurs with<even> ::=(collatz (/ n 2))if n is even and recurs with<odd> ::=(collatz (+ (* 3 n) 1))if n is odd.We can package that up into the collatz function:The Collatz conjecture is true if this function terminates for every input.Thanks to the flexibility of literate programming, we can package up the code to compute orbits of Collatz numbers too:Finally, we put the whole thing together, after establishing different scopes for the two functions.<*> ::=
(require scheme/local) (local [<collatz-sequence>] (collatz 18)) (local [<collatz>] (collatz 18))