Declarative Text Kit: Token-Based Adapters

I’ve now finished adding an example adapter to get a structural representation of a Markdown code block to my app.

It bridges the abstract syntax tree (or “token tree”) of libMultiMarkdown to NSTextStorage-compatible UTF-16 substring ranges – which DeclarativeTextKit works with, reducing code size and potential for errors even further:

if let codeBlock = try textStorage.block(FencedCodeBlock.self, atPoint: insertionPointLocation) {
    try buffer.evaluate {
        Select(LineRange(codeBlock.blockRange)) { selectedRange in
            Modifying(selectedRange) { blockRange in

The codeBlock value exposes its own blockRange, and the (line) ranges of the opening and closing fences.

Previously, I had to spend a couple of procedural lines of code on finding the subtree ranges for the triple backticks and clean things up. No more!

This is, for all intents and purposes, a one-liner now. If I can grab a fitting value that represents the code block, then I perform a declarative change directly, and that’s it.

This is the ideal that’s worth striving for.

The value type itself is quite simple:

public struct FencedCodeBlock: Equatable {
    public let blockRange: UTF16Range
    public let openingFencedLineRange: UTF16Range
    public let closingFencedLineRange: UTF16Range
    public let language: String?

That is the information I will want to work with when it comes to fenced code blocks. The language string is not used for anything at the moment, but when inspecting fenced code blocks later, that metadata is interesting to e.g. perform syntax highlighting.

That value type was the goalpost: I sketched this type about two weeks ago and needed to work my way towards it. – An example of “wishful programming” in practice!

But to reach it, I first needed to

  • build facilities to find fenced code block tokens at a location in the document,
  • walk the subtree to find the opening/closing fenced lines,
  • read the language metadata, if present, and bridge the UTF-8 C string of the token tree to a Swift string.

With that in place to get from libMMD token tree pointers to a safely walkable tree in Swift to read structured document information on one side, and with DeclarativeTextKit to write changes to the document in a convenient manner on the other, toggling fenced code blocks on or off is now really straight-forward.

How You Can Use DeclarativeTextKit

Since DeclarativeTextKit is used to write changes, you can use any reader that fits your needs:

I’ve sketched a way to express an interesting piece of a Markdown document above, called FencedCodeBlock. I get there via libMMD’s token tree, but you won’t have that in your app most likely.

But you can get to a FencedCodeBlock in a Markdown-documen based on different data. You could use NSLayoutManager’s (temporary) attributes to store semantic information like “this range is a code block” in the Text Kit document.

Or if you use tree-sitter, you can ask its AST for a fitting range.

Same with Apple’s own swift-markdown package: you can use a MarkupWalker to find code blocks.