History of Metaprogramming

Brian Harvey

Teaching Professor Emeritus

University of California, Berkeley

What’s “Metaprogramming”?

Metaprogramming is a programming technique in which computer programs have the ability to treat other programs as their data. It means that a program can be designed to read, generate, analyze or transform other programs, and even modify itself while running. In some cases, this allows programmers to minimize the number of lines of code to express a solution, in turn reducing development time. It also allows programs greater flexibility to efficiently handle new situations without recompilation.

Metaprogramming can be used to move computations from run-time to compile-time, to generate code using compile time computations, and to enable self-modifying code. The ability of a programming language to be its own metalanguage is called reflection. Reflection is a valuable language feature to facilitate metaprogramming.

—Wikipedia

Are higher order functions metaprogramming?

• Well, yes and no.
• Yes, map is using × as data, and “program as data” is part of the definition.
• But with higher order functions, our emphasis is on the abstract, mathematical input-output behavior and what it means. In a programming language we can represent only procedures, not really functions, but it’s the function that matters. So in

​

• what matters is that we’re interested in the squares of some numbers, not that to express that we use a round green block inside a gray ring.

“the ability to treat other programs as data.”

“to generate code”

Metaprogramming out in the real world: text macros.

#define foo(x) #x)

• Foo is a macro in the C programming language.

​

#include <stdio.h>

#define foo(x) #x)

prints 87

main(){

printf("%s\n",foo(87);

}

• The macro processor doesn’t know anything about the language syntax; the macro would result in a compiler error if there didn’t happen to be an unmatched open parenthesis in the printf call.
• That’s good because it lets you create your own syntax, but bad because it’s too easy to make undebuggable mistakes.

Lisp 1.5 FEXPRs

• Lisp 1.5 was the first Lisp that anyone outside the MIT Artificial Intelligence Lab ever used. It had four kinds of procedure:
• SUBR: Ordinary Lisp primitive procedure (applicative order evaluation)
• FSUBR: Lisp built-in special form (special syntax rules)
• EXPR: User-defined ordinary procedure (applicative order)
• FEXPR: User-defined special form (special syntax rules)
• FEXPRs were the first Lisp macros. They were special only in that Lisp didn’t evaluate their inputs prior to calling the procedure (normal order evaluation) and were variadic. This was enough to allow you to create pretty much any syntax, as long as it looked like a Lisp list. E.g., you could have title text between your real inputs, and your macro would check to make sure the title text is correct, or distinguish two different functions based on interior title text, or use infix order arithmetic expressions (x + 3) instead of the usual Lisp (+ x 3) prefix form.

Macro evaluation environments

• Normal order was the only needed exception to the usual evaluation rules in Lisp 1.5 because it was dynamically scoped.
• Dynamic scope: Variables that are used but not defined within a procedure are found by looking in the variable bindings (the environment) of the calling procedure.
• Under dynamic scope, a macro can use both its own variables and any variables that are visible to the procedure that calls the macro.
• … which can lead to errors because the same name appears in both environments. That’s one reason why computer scientists almost all hate dynamic scope these days.
• But when Scheme and other more recent Lisp dialects introduced lexical scope to Lisp, macrology became more complicated. Macros now have to use their own variables (such as formal parameters) but also have to be able to access their callers’ variables.

Eval-twice macros

• One solution was to introduce macros that were evaluated twice, first in the macro’s own environment as for any procedure, then the value reported from that evaluation was evaluated again in the caller’s environment.

(define-macro (cons-stream . args)

`(cons ,(car args) (delay ,(cadr args))))

​

• This is how Berkeley Logo macros work.
• But, as already mentioned, this solution allows for confusion between the macro’s own variable names and those of the caller.
• The solution: hygienic macros, introduced by Scheme, in which the two environments are kept separate. Snap! 8.0 doesn’t quite do that yet.

Snap! isn’t self-reflective.

• In Lisp, code is just list structure. The same procedures that allow visiting every node of a (data) tree work on procedure code.
• The solution: translation procedures code -> syntax tree and syntax tree -> code. Snap! 8.0 overloads text processing functions for this purpose: and .

​

​

• Logo, from its earliest days, had TEXT and DEFINE.

Metaprogramming in Snap! 8.0

• Conversion between code and syntax trees

​

​

​

​

• Examine and modify block definitions

​