1 Documentation as Code

Most existing documentation tools fall into one of three categories: LaTeX-like tools that know nothing about source code, JavaDoc-like tools that extract documentation from annotations in source code, and WEB-like literate-programming tools where source code is organized around a prose presentation.

Scribble is a new documentation infrastructure for PLT Scheme that can support and integrate all three kinds of tools. Like the LaTeX category, Scribble is suitable for producing stand-alone documents. Like the other two categories, Scribble creates a connection between documentation and the program that it describes – but without restricting the form of the documentation like JavaDoc-style tools, and with a well-defined connection to the language’s scoping that is lacking in WEB-like tools. Specifically, Scribble leverages lexical scoping as supplied by the underlying programming language, instead of ad hoc textual manipulation, to connect documentation and code. This connection supports abstractions across the prose and code layers, and it enables a precise and consistent association (e.g. via hyperlinks) of references in code fragments to specifications elsewhere in the documentation.

Figure 1: DrScheme with binding arrows and documentation links on Scribble code

For example, @scheme[circle] in a document source generates the output text circle. If the source form appears within a lexical context that imports the slideshow library, then the rendered circle is hyperlinked to the documentation for the slideshow library – and not to the documentation of, say, the htdp/image library, which exports a circle binding for a different GUI library. Moreover, the hyperlink will be correct even if @scheme[circle] resides in a function that is used to generate documentation, and even if the lexical context of the call does not otherwise mention slideshow. Such lexically scoped fragments of documentation are built on the same technology as Scheme’s lexically scoped macros, and they provide the same benefits for documentation abstraction and composition as for ordinary programs.

To support documentation in the style of JavaDoc, a Scribble program can “include” a source library and extract its documentation. Bindings in the source are reflected naturally as cross-references in the documentation. Similarly, a source program can use module-level imports to introduce and compose literate-programming forms; in other words, the module system acts as the language that Ramsey has suggested to organize the composition of noweb extensions (Ramsey 1994).

Scribble’s capacity to span documentation-tool categories is a consequence of PLT Scheme’s extensibility. Extensibility is an obstacle for JavaDoc-style tools, which parse a program’s source text and would have to be extended in parallel to the language. Scribble, in contrast, plugs into the main language’s extensibility machinery, so it both understands language extensions and is itself extensible. Similarly, Scheme macros accommodate a WEB-like organization of a library’s implementation, and the same macros can simultaneously organize the associated documentation.

Indeed, Scribble documents are themselves Scheme programs, which means that PLT Scheme tools can work on Scribble sources. Figure 1 shows this paper’s source opened in DrScheme. After clicking Check Syntax, then a right-click on a use of emph directly accesses the documentation of the emph function, even though the surface syntax of the document source does not look like Scheme. Such documentation links are based on the same lexical information and program-expansion process that the compiler uses, so the links point precisely to the right documentation.

We developed Scribble primarily for stand-alone documentation, but we have also developed a library for JavaDoc-style extraction of API documentation, and we have created a WEB-style tool for literate programming. In all forms, Scribble’s connection between documentation and source plays a crucial role in cross-referencing, in writing examples within the documentation, and in searching the documentation from within the programming environment. These capabilities point the way toward even more sophisticated extensions, and they illustrate the advantages of treating documentation as code.