Writing HTML using Racket and X-expressions
What if I wanted to make HTML programmable - treat it as data, and add my own constructs and syntax to it? Let's make something that would generate a list for any value I pass it. Let's write it in Python.
The cool thing is that
genlist can take itself as an argument:
genlist(genlist('item', 3), 5) will return a nested list of HTML elements. We've augemented HTML with new “syntax”.
But this approach is a little limited. We end up manipulating strings around, instead of a proper data representation for HTML. Instead of having a tree structure, we have a stream that can be only added to. There's also a lot of busywork, like opening and closing tags that could be done away with. Let's try again with templates:
With templates, we're writing HTML directly. At any moment we can drop down to Python to generate HTML. Unfortunately, by using this new templating language, we need to keep track of both Python and the templating engine.
We've lost a lot of expressiveness: what if we wanted to create a nested list again? We could make it a template macro, but we have to do extra work to call it inside itself, and suddently we have an extra language to learn.
That leaves us with a choice:
- use our favorite programming language to generate HTML, allowing to be more dynamic.
- use templates at a loss of expressiveness.
What if we could have both?
Racket X-expressions: a data structure for HTML
The elements with a quote
' before them are symbols, so
'li represents an list element.
(list 'li "item") becomes an li element containing
"item". Elements can contain other elements too:
(list 'li (list 'p "item")) contains an li that contains a p, and so on. We can convert this X-expression into HTML with
(xexpr->string), giving us a way to go back to HTML.
This form is a little unwieldy, with
list everywhere. Racket offers a construct, named
quote and written
' that can make writing these expressions easier:
quote takes any thing that looks like a list - anything with parantheses, creates a list and quotes each value. putting a
' before each of them. If the thing to be quoted is a value, it leaves it as is. Quoting is recursive: if it finds a list, it will quote each element of the list.
With this form
'(li "item") becomes
(list 'li '"item"). On nested lists,
'(li (p "item")) becomes
(list 'li (list 'p "item")).
Sometimes we may want to quote something, but keep some expressions from being quoted. We can use the
quasiquote form, written
` (backtick). Within the quasiquoted list, we can use
, to declare our non-quoted expressions.
For example, take
(string-append), which puts two strings together. If we want a list with a call to
(string-append) in it, we run into issues quickly:
'(li (p "hello") (string-append "hi" "world")) evaluates to
(list ('p "hello") ('string-append "hi" "world")). With quasiquoting, we can tell
(string-append) is a function call:
`(li (p "hello") ,(string-append "hi" "world")) evaluates to
(list ('li ('p "hello") "hiworld")).
We could have something that generates a list, like a variable with a list of several values, or a function that returns a list of values. To get its elements instead of the list directly, we can use
,@, to flatten the list:
Code as data: adding syntax to HTML
We have a data structure that represents HTML - and we can define
The bulk of the work is done by
recur-li, a recursive function that constructs a list of
(genlist "hi" 5) will build
'(ul ((li "hi") (li "hi") (li "hi") (li "hi") (li "hi"))).
Finally, we get back the power we had with the naive Python implementation. We can use
genlist within itself:
(genlist (genlist "item" 3) 5) will create a list, where each element is a
(genlist) call which is then interpreted. We've also kept a nice structure around, where we deal with a representation of HTML in our code.
Let's make a page with this in mind:
We can use
(xexpr->string our-page) to convert it to a string. It's neat, but it doens't have an
head element. It would also be nice to set the title of the page to something. Surely we can pass the title to
And, as expected:
It looks a lot like a template, where you pass data and that data goes somewhere within the template, and that's because it is one. With a little bit of ingeniuity and quoting, we've managed to make HTML programmable in a much more natural way.
There is no distinction between code and data. Data can go anywhere code usually goes: as seen, we use the usual HTML elements like
body along with our own constructs like
genlist and it works. This is the principle behind S-expressions: code is data. With X-expressions, which are expressions as well, we can transform a fixed language like HTML and add our own constructs to it, instead of treating it as data to pass around.
Racket is particularly skilled at this feat: it can create its own languages. We could imagine a new language based on HTML but with new, more convenient syntaxic forms. If you are interested in this idea, check out Beautiful Racket.
Conclusion: making a blog in Racket
This blog is written in Racket, and most of the HTML here is generated with X-expressions. The static site generator I use is called polyglot, which allows to write HTML in any Racket-created language. It takes this idea and adds a lot of cool stuff to make it usable, including:
- a webpack-like system in Racket to link assets together
- writing in markdown, with the ability to drop down to a Racket script that generates HTML like shown here, to have full control over prose and markup
- the ability to write HTML with any Racket language desired, not just Racket itself
If nothing else, making sites in Racket is fun. It's not tiring to use like many web frameworks, and I think the end result looks nice. Racket can be practical, too!