boo metaprogramming facilities I - the ast

The boo compiler follows the common design of internally representing code using what’s usually called a heterogeneous abstract syntax tree, ast for short. Code constructs such as class and method definitions, if and while statements, references and all sorts of expressions are represented by specific concrete types such as ClassDefinition, Method, IfStatement, WhileStatement, ReferenceExpression, etc.

In other words, given an arbitrary fragment of boo code:

    def ltuae():
        return 42

Its internal representation can be obtained through the chained instantiation of the right ast types:

    import Boo.Lang.Compiler.Ast
    fragment = Method(Name: "ltuae", Body: Block(ReturnStatement(IntegerLiteralExpression(42))))
    print fragment.ToCodeString()

Such an approach to ast construction however fundamental might lead very rapidly to unintelligible code so boo provides a more natural way of constructing and manipulating ast fragments through the use of code literals:

    fragment = [|
        def ltuae():
            return 42
    |]
    print fragment.ToCodeString()

[| and |] are the quasi-quotation markers and they instruct the compiler to produce the proper ast instantiation chain for the delimited code.

They can be used with expressions:

    expression = [| question is ltuae |]
    print expression.Operator         # ReferenceEquality
    print expression.Left.GetType()   # ReferenceExpression
    print expression.Right.GetType()  # ReferenceExpression

They can be used with statements:

    statement = [| 
        if question is ltuae:
            return 42
    |]
    print statement.Condition # (question is ltuae)

As well as with complete type definitions:

    type = [|
         class BigThinker:
             def ltuae():
                 return 42
    |]
    for member in type.Members:
        print member.Name

Code literals also provide for a simple way of assembling larger code structures out of smaller fragments:

    def bigThinkerOf(thought as Boo.Lang.Compiler.Ast.Expression):
        return [|
            class BigThinker:
                def ltuae():
                    return $thought
        |]
        
    for member in bigThinkerOf([| 42 |]).Members:
        print member.ToCodeString()

$ is the splicing operator and it instructs the compiler to inject the marked expression into the ast instantiation chain.

Primitive values can also be spliced in:

    primitive = 42
    fragment = [| $primitive |]
    print fragment.GetType() # IntegerLiteralExpression

Getting an ast fragment out of a primitive value is called lifting and can be more directly achieved through one of the many Expression.Lift, Statement.Lift and TypeReference.Lift overloads:

    fragment = Boo.Lang.Compiler.Ast.Expression.Lift(42)
    print fragment.GetType() # IntegerLiteralExpression

Complex expressions can be spliced in but must be parenthesized for disambiguation:

    primitive = 42
    fragment = [| $(primitive.ToString()) |] # in contrast to [| $primitive.ToString() |] 
    print fragment.GetType() # StringLiteralExpression

And finally an ast fragment can be compiled into runnable code using one of the compile primitives available in the Boo.Lang.Compiler.MetaProgramming namespace:

    import Boo.Lang.Compiler.MetaProgramming
    
    ast = [|
        class BigThinker:
    	    def ltuae():
                return 42
    |]
    type = compile(ast)
    instance as duck = type() # duck typing to the rescue
    print instance.ltuae()

Runtime code generation is a useful technique but most really interesting metaprogramming opportunities manifest themselves at compile time through one of the extension points provided by boo: attributes, macros, metafunctions and the compilation pipeline which shall all be covered next.