2 From Syntactic Abstraction to Language Construction
The examples in the previous section (Example Racket Languages) demonstrate several languages that are relatively distant from the more primitive racket/base language. The point of the examples was to illustrate Racket’s reach. In practice, however, many programming problems require a much more modest variation of an existing language than building an entirely new language.
Racket’s support for language-oriented programming provides a smooth path from simple syntactic abstractions (to avoid boilerplate, even within a single module) to syntactic extension (to provide new language constructs, such as classes and objects) to full new languages (such as typed/racket or Datalog). The key parts of Racket that enable that spectrum are
syntax objects as a semi-structured, binding-aware representation of program fragments;
macro expansion to trigger compile-time computations that manipulate syntax objects, transforming source programs to more primitive terms; and
modules for managing bindings so that they can bridge compile-time transformations and run-time support in a convenient and scalable way.
We sometimes use the word macros as a shorthand for this combination of technologies in Racket, because writing macro transformations is often the central task in implementing a language with Racket.
2.1 Syntactic Abstraction
Suppose that you find yourself writing repetitive code when setting up a bridge between Racket and a C-implemented library using Racket’s FFI:
(define insstr (get-ffi-obj 'insstr ncurses-lib (_fun #:lock-name "ncurses" _string -> _int))) (define insnstr (get-ffi-obj 'insnstr ncurses-lib (_fun #:lock-name "ncurses" _string _int -> _int))) (define winsstr (get-ffi-obj 'winsstr ncurses-lib (_fun #:lock-name "ncurses" _pointer _string -> _int))) (define winsnstr (get-ffi-obj 'winsnstr ncurses-lib (_fun #:lock-name "ncurses" _pointer _string _int -> _int))) ....
Functional abstraction is generally the best way to avoid repetition. In this case, however, we need to abstract over definitions and syntactic patterns of _fun.
This is repetition is easily avoided through a pattern-based macro, because a macro can generate a definition as easily as an expression.
(define-simple-macro (defncurses name arg-type ... -> result-type) (define name (get-ffi-obj ncurses-lib 'name (_fun #:lock ncurses-lock arg-type ... -> result-type)))) (define-ncurses insstr _string -> _int) (define-ncurses insnstr _string _int -> _int) (define-ncurses winsstr _pointer _string -> _int) (define-ncurses winsnstr _pointer _string _int -> _int) ....
Note that the four-dot .... is intended to represent code not shown here, but the three-dot ... in define-simple-macro is part of the syntax of pattern-based macros. If pattern-based macros seem a little mysterious right now, that’s ok; in a little while, we’ll break things down to a more primitive level, and then we’ll build back up to patterns and templates.
This use of macros to avoid keystrokes is relatively shallow. The define-ncurses form is unlikely to be useful in other programs, so it is unlikely to be separately documented, and (as we will see) it is missing some integrity checks that would be needed to guard against misuse, such as insisting that name is matched to an identifier as opposed to a number. This macro illustrates syntactic abstraction, but it’s abstraction only in putting common code in one place. A reader of the module that uses define-ncurses is probably going to just look at its definition to understand what it means (so, not abstract in that sense).
2.2 Language Extension
Another task that may lead to repetitive code is in measuring the time for an expression to evaluate. That measurement can be implemented in Racket using current-inexact-milliseconds:
(let* ([before (current-inexact-milliseconds)] [answer (fib 30)] [after (current-inexact-milliseconds)]) (printf "It took ~a ms to compute.\n" (- after before)) answer)
Clearly, there’s a lot of boilerplate here around the expression to measure, which is just (fib 30). In this case, we can abstract the boilerplate into a function:
(define (time-thunk thunk) (let* ([before (current-inexact-milliseconds)] [answer (thunk)] [after (current-inexact-milliseconds)]) (printf "It took ~a ms to compute.\n" (- after before)) answer)) (time-thunk (lambda () (fib 30)))
This will be better if we have many expressions to time, since we can wrap just (time-thunk (lambda () ....)) around each expression. Still, it’s not great as an abstraction of timing, because nothing about the problem of timing an expression says that it should involve thunks. We can create a better abstraction by using a macro to create the thunk:
(define-simple-macro (time-it expr) (time-thunk (lambda () expr))) (time-it (fib 30))
The time-it syntactic form is a good enough abstraction to put into a library somewhere. (Actually, a form like this already exists in racket/base as time, but this is how time is implemented.)
When you decide that a function belongs in a library, then you simply take its implementation along with any private helper function, put it in the new library, export the function from the new library, and then import the function in other libraries to use it. The same strategy works with Racket macros, and that’s part of the smooth path from simple cases to more ambitious cases. That is, the library to provide time-it contains the time-it macro and time-thunk function:
#lang racket/base (require syntax/parse/define) ; for define-simple-macro (provide time-it) (define-simple-macro (time-it expr) (time-thunk (lambda () expr))) (define (time-thunk thunk) (let* ([before (current-inexact-milliseconds)] [answer (thunk)] [after (current-inexact-milliseconds)]) (printf "It took ~a ms to compute.\n" (- after before)) answer))
Only time-it is exported. When time-it is used in another module, its expansion introduces a call to time-thunk. The macro system makes that work right, even though time-thunk is not directly available in a module that uses time-it.
The time-it macro is an example of language extension. A module in the racket/base language that imports time-it gets an extension to the language that behaves as much a part of the language as any built-in form.
2.3 Language Construction
A module that starts #lang X refers to another module X in much thte same way that (require X) refers to a module X. The difference is that #lang X gives X the chance to determine bindings from scratch, instead of strictly adding to the current of bindings as (require X) does. Since a module can re-provide bindings from another module, X can extend racket/base or any other module, if it chooses. Alternatively, a language X can withold require so that it has complete control over bindings in a module that uses #lang X .
There’s more to the story for languages, including the protocol that gives X control over character-level parsing. But the idea that a language is just another module is the key to turning language extension into language construction.