#lang scribble/doc
@(require "common.ss")

@title{Scribble Design Overview}

The Scribble documentation framework addresses three parts in the
design of a programmable documentation system: the data structures
needed to represent document fragments, the concrete syntax used to
express such fragments and their manipulations, and the connection
between run-time entities and the documentation that describes them or
that references them in typeset code fragments.

As an example, consider the beginning of the PLT Scheme overview
documentation:

@nested{
 @fake-section{Welcome to PLT Scheme}

 Depending on how you look at it, @bold{PLT Scheme} is

 @itemize[

   @item{a @emph{programming language} --- a descendant of Scheme,
   which is a dialect of Lisp;}

   @item{a @emph{family} of programming languages --- variants of
   Scheme, and more; or}

   @item{a set of @emph{tools} for using a family of programming
   languages.}

 ]

 Where there is no room for confusion, we use simply ``Scheme''
 to refer to any of these facets of PLT Scheme.
}

Within in a programmable documentation system, this documentation
fragment could be represented as follows:

@code-block{
#lang scheme

(make-part
 (list "Welcome to PLT Scheme")
 (list
  (make-paragraph
   (list "Depending on how you look at it, "
         (make-bold "PLT Scheme") " is"))
  (make-itemize
   (list
    (list
     (make-paragraph
      (list "a " (make-emph "programming language") 
            "\u2015 a descendant of Scheme, "
            "which is a dialect of Lisp;")))
    (list
     (make-paragraph
      (list "a " (make-emph "family") " of "
            "programming languages \u2015 "
            "variants of Scheme, and more; or")))
    (list
     (make-paragraph
      (list "a set of " (make-emph "tools") " "
            "for using a family of programming "
            "languages.")))))
  (make-paragraph
   (list "Where there is no room for confusion, "
         "we use simply \u201CScheme\u201D to "
         "refer to any of these facets of PLT "
         "Scheme."))))
}

The above representation illustrates several facets of the document
structure. A part contains a list of elements for its title, as well
as a list of paragraph-like entities for its content. An itemization
is like a paragraph, except that it contains a list of bullets, each
of which can be a sequence of paragraph-like entities. A paragraph,
meanwhile, contains a list of textual elements, which might be either
strings (for plain text) or instances of datatypes for styled text
(bold, emphasized, etc.).

The sample representation is approximate in some ways. For example, a
``part'' representing a section needs more attributes, such as a list
of parts for sub-sections and a key to identify the section for
cross-referencing. One task in designing Scribble is identifying a
suitable representation of documents, for which we draw heavily on
Skribe@~cite["Skribe"]. We describe the data structures in more detail
in @secref["struct"], but the above approximation shows the basic
idea.

In terms of concrete syntax, even the approximation of the datatype
above is far too unwieldy to write directly. We could use functions
that fill in various defaults (instead of using raw data
constructors), but any S-expression format will be only slightly
more readable than the above approximation. Scribble's @|latex|-like
surface syntax for the document fragment is more readable and
maintainable:

@code-block|{
 #lang scribble/doc
 @(require scribble/manual)

 @section{Welcome to PLT Scheme}

 Depending on how you look at it, @bold{PLT Scheme} 
 is

 @itemize[
   @item{a @emph{programming language} --- a 
         descendant of Scheme, which is a dialect 
         of Lisp;}

   @item{a @emph{family} of programming languages
         --- variants of Scheme, and more; or}

   @item{a set of @emph{tools} for using a family 
         of programming languages.}
 ]

 Where there is no room for confusion, we use
 simply ``Scheme'' to refer to any of these facets
 of PLT Scheme.
}|

The initial @code-elem{#lang scribble/doc} line declares that the
module uses Scribble's documentation syntax, and it imports all of the
usual PLT Scheme functions and syntax. The @scr:code-elem|{@(require
scribble/manual)}| form imports more functions specific to typesetting
a user manual. The remainder of the module represents the document
content.

This syntax works better than nested S-expressions, because it more
closely resembles the resulting layout. First, although all of the
text belongs in a section, it is implicitly grouped by the section
title, instead of explicitly group into a section with
brackets. Second, the default parsing mode is ``text'' instead of
``expression.'' Third, in text mode, various automatic rules convert
ASCII to more sophisticated typeset forms, such as the conversion of
@tt{---} to an em-dash and @tt{``...''} to curly quotes.

The most obvious difference between @|latex| and Scribble is the use
of an at-sign escape to switch into expression mode instead of a
backslash escape. More significant differences relate to programming
with functions and datatypes, as opposed to macros that manipulate raw
text. For example, @scheme[itemize] is a function that accepts
document fragments created by the @scheme[item] function, instead of a
text-parsing macro like @|latex|'s @tt{itemize} environment. The
square brackets after @scheme[itemize] in the document source reflect
that it accepts @scheme[item] values, whereas @scheme[item] and many
other functions are followed by curly braces that indicate raw text
arguments. We describe Scribble's concrete syntax in more detail in
@secref["at-sign"] and @secref["decode"].

To connect documentation with source code, Scribble builds on Scheme's
support for lexically-scoped macros, where literal fragments of syntax
can be packaged with information about the enclosing lexical
environment. In the case of documentation, the lexical environment of
a program fragment determines the meaning of each identifier in the
fragment, so that the identifier can be hyperlinked to its
definition. For example, to typeset an example definition

@(define-syntax-rule (typeset-louder)
   ;; This macro introduces lang/htdp-beginner
   ;; bindings for just the example that is generated
   ;; by the same macro expansion --- which is the way
   ;; that bindings always work with macros.
   (begin
    (require (for-label lang/htdp-beginner))
    @schemeblock[
      (define (louder s)
        (string-append s "!"))
    ]))
@(typeset-louder)

the Scribble document source is

@code-block|{
 #lang scribble/doc
 @(require scribble/manual
           (for-label lang/htdp-beginner))
 ....
 @schemeblock[
   (define (louder s)
     (string-append s "!"))
 ]
}|

The @scheme[require] form in this module imports
@schememodname[scribble/manual] as before. The additional
@scheme[(for-label lang/htdp-beginner)] imports bindings into the
module's ``label'' phase, which binds identifiers in typeset code
examples, but does not cause the imported module to be
executed. Specifically, the for-label import above causes the
@scheme[define] and @scheme[string-append] identifiers to be typeset
as a keyword and pre-defined function, respectively. In addition, the
identifiers are automatically hyperlinked (in HTML output) to the
documentation for the @scheme[define] form and @scheme[string-append]
function as exported by the @schememodname[lang/htdp-beginner]
module. This typesetting and hyperlinking is determined by the
@scheme[schemeblock] macro in much the same way that other Scheme
macros manipulate syntactic program fragments. We describe the
mechanisms for connecting documentation and source more in
@secref["scheme"].

The @schemeidfont{define} identifier in the above module is bound in
two different ways in two different phases. The initial
@code-elem{#lang scribble/doc} binds the usual Scheme @scheme[define]
for the ``run time'' phase, so that it can be used when implementing
the document content. The @scheme[(for-label lang/htdp-beginner)]
import binds @scheme[define] so that code examples though
@scheme[schemeblock] refer to the restricted teaching-language form,
instead of the normal Scheme form. In this way, the language used to
implement a document need not be constrained by the language that is
documented, nor vice-versa. The separation of binding phases is a
straightforward generalization of our work on macro-expansion
phases@~cite["macromod"], and we describe more of the implications for
writing examples and extracting documentation from source in
@secref["srcdoc"].