If you find that this library lacks some feature you need, or you have a suggestion for improving it, please don’t hesitate to get in touch with me!
This library adds three features to Racket:
library support for bit strings, a generalization of byte vectors;
syntactic support for extracting integers, floats and sub-bit-strings from bit strings; and
syntactic support for constructing bit strings from integers, floats and other bit strings.
It is heavily inspired by Erlang’s binaries, bitstrings, and binary pattern-matching. The Erlang documentation provides a good introduction to these features:
A bit string is either
a byte vector, as returned by bytes and friends;
a bit-resolution slice of a byte vector, as returned by ; or
a splicing-together of two bit strings, as returned by .
The routines in this library are written, except where specified, to handle any of these three representations for bit strings.
If you need to flatten a bit string into a contiguous sequence of whole bytes, use or .
All the functionality below can be accessed with a single require:
|(require (planet tonyg/bitsyntax:1:0))|
|( value-expr clause ...)|
Each clause is then tried in turn. The first succeeding clause determines the result of the whole expression. A clause matches successfully if all its bit-string-segment-patterns match some portion of the input, there is no unused input left over at the end, and the guard-expr (if there is one) evaluates to a true value. If a clause succeeds, then (begin body-expr ...) is evaluated, and its result becaomes the result of the whole expression.
Each bit-string-segment-pattern matches zero or more bits of the input bit string. The given type, signedness, endianness and width are used to extract a value from the bit string, at which point it is either compared to some other value (if (= expr) was used in the segment-pattern), bound to a pattern variable (if (? id) was used), or discarded (if :discard was used) before matching continues with the next bit-string-segment-pattern.
The supported segment types are
:integer – this is the default. A signed or unsigned, big- or little-endian integer of the given width in bits is read out of the bit string. Unless otherwise specified, integers default to big-endian, unsigned, and eight bits wide. Any width, not just multiples of eight, is supported.
:float – A 32- or 64-bit float in either big- or little-endian byte order is read out of the bit string using floating-point-bytes->real. Unless otherwise specified, floats default to big-endian and 64 bits wide. Widths other than 32 or 64 bits are unsupported.
:binary – A sub-bit-string is read out of the bit string. The bit string can be an arbitrary number of bits long, not just a multiple of eight. Unless otherwise specified, the entire rest of the input will be consumed and returned.
Each type has a default signedness, endianness, and width in bits, as described above. These can all be overridden individually:
:unsigned and :signed specify that integers should be decoded in an unsigned or signed manner, respectively.
:big-endian, :little-endian and :native-endian specify the endianness to use in decoding integers or floats. Specifying :native-endian causes Racket to use whatever is the native endianness of the platform the program is currently running on (discovered using system-big-endian?).
:default causes the decoder to use whatever the default width is for the type specified.
:bytes integer causes the decoder to try to consume integer bytes of input for this segment-pattern.
:bits integer causes the decoder to try to consume integer bits of input for this segment-pattern.
( some-input-value ([(= 0 :bytes 2)] 'a) ([(? f :bits 10) (:discard :binary)] (when (and (< f 123) (>= f 100))) 'between-100-and-123) ([(? f :bits 10) (:discard :bits 6)] f) ([(? f :bits 10) (:discard :bits 6) (? rest :binary)] (list f rest)))
This expression analyses some-input-value, which must be a (). It may contain:
16 zero bits, in which case the result is 'a; or
a ten-bit big-endian unsigned integer followed by 6 bits which are ignored, where the integer is between 100 (inclusive) and 123 (exclusive), in which case the result is 'between-100-and-123; or
the same as the previous clause, but without the guard; if this succeeds, the result is the ten-bit integer itself; or
the same as the previous clause, but with an arbitrary number of bits following the six discarded bits. The result here is a list containing the ten-bit integer and the trailing bit string.
The following code block parses a Pascal-style byte string and decodes it using a UTF-8 codec:
( input-bit-string ([(? len) (? body :binary :bytes len)] (bytes->string/utf-8 ( body))))
Notice how the len value, which came from the input bit string itself, is used to decide how much of the remaining input to consume.
Each spec can specify an integer or floating-point number to encode, or a bit string to copy into the output. If a type is not specified, :integer is assumed. If an endianness is (relevant but) not specified, :big-endian is assumed. If a width is not given, :integers are encoded as 8-bit quantities, :floats are encoded as 64-bit quantities, and :binary objects are copied into the output in their entirety.
If a width is specified, integers will be truncated or sign-extended to fit, and binaries will be truncated. If a binary is shorter than a specified width, an error is signalled. Floating-point encoding can only be done using 32- or 64-bit widths.
(define (string->pascal/utf-8 str) (let ((bs (string->bytes/utf-8 str))) ( [(bytes-length bs)] [bs :binary])))
This subroutine encodes its string argument using a UTF-8 codec, and then assembles it into a Pascal-style string with a prefix length byte. If the encoded string is longer than 255 bytes, note that the length byte will be truncated and so the encoding will be incorrect. A better encoder would ensure that bs was not longer than 255 bytes before encoding it as a Pascal string.
Note that if you wish to leave all the modifiers at their defaults (that is, :integer :bits 8), and the expression you want to encode is held in a variable, you can use the second form of spec given above: that is, you can simply mention the variable. For example, the above subroutine could also have been written as follows:
(define (string->pascal/utf-8 str) (let* ((bs (string->bytes/utf-8 str)) (len (bytes-length bs))) ( len [bs :binary])))
|( x) → boolean?|
|offset : integer?|
|( x) → bytes?|
|target-offset : integer?|
|source-offset : integer?|
|count : integer?|
These procedures may be useful for debugging, but should not be relied upon otherwise.
|( x) → bytes?|