OpenMoji Support in emacs-emojify

Some OpenMoji emoji with modus-operandi and modus-vivendi themes

If you have no dignity (like me), you might enjoy displaying emoji in Emacs. I actually really like that when composing/reading email, and when I want to add colorful stuff like stars in my to-do lists.

There’s this package, emacs-emojify, you might want to check out. When you install the package, it will download EmojiOne images by default. You can set up your own, and I found the OpenMoji set to look very nice. Clear lines, crisp shapes, very recognizable expressions in big and small sizes. Love it.

OpenMoji is open source, and I took their 72px color PNG exports from their v13.0 release from GitHub and imported them in emacs-emojify. The file name pattern matches (it’s unicode code points + .png), so I only had to point emacs-emojify to the new folder, and that was it.

By now, my pull requests went through and OpenMoji support will ship with an upcoming release of emacs-emojify.

Decorate NSGlyphStorage to Replace Glyphs On-the-Fly

There are many ways to affect the glyphs that are used to show text on screen. Since macOS 10.11 El Capitan (released in 2015), possibly the simplest override points are in NSLayoutManagerDelegate. Another, arguably more ancient way to replace glyphs on-the-fly is to override and customize NSGlyphGenerator. This predates the existence of NSLayoutManagerDelegate, and it’s not even available on iOS, so that’s how arcane the approach outlines here will be.

I was revisiting WWDC 2010’s session 114, Advanced Cocoa Text Tips and Tricks by Aki Inoue and Dan Schimpf, because I had notes on that session from 2017 in my Zettelkasten when I learned the basics of customizing a text editor. Schimpf/Inoue show how you can implement folding paragraphs in their talk, and they use NSGlyphGenerator to effectively hide all the folded-away glyphs on screen.

In general, NSGlyphGenerator has only one method: generateGlyphs(for:desiredNumberOfCharacters:glyphIndex:characterIndex:). In our subclasses of NSGlyphGenerator, we are supposed to call NSGlyphGenerator.shared which actually provides a useful implementation of mapping characters to glyphs.

In case you’re confused by the terminology: think of characters are the string representation without any info about the looks, and glyphs as the digital, movable type from a font family. Or, for programmers: so actually draw a character on screen you take the character code, map it with a font, then get a glyph that you can draw. More or less.

To actually customize what happens in NSGlyphGenerator, you can decorate the NSGlyphStorage target that is used in the process. This is usually the NSLayoutManager, but the rest of the layout manager is not important for this job. By “decorate” I mean in the “Decorator” or “Wrapper” Design Pattern sense: you implement all required methods, forward all of them to a base implementation, and adjust the parameters you forward and/or the results produced by the base implementation.

In the WWDC session from 2010, Schimpf/Inoue subclassed NSGlyphGenerator and made the subclass itself conform to NSGlyphStorage, so their subclass was actually the decorator. Their delegation to NSGlyphGenerator.shared then passed self – I don’t like that mixing of concepts for didactical reasons.

So even though a bare NSGlyphGenerator subclass is virtually useless, here’s an extracted decorator:

class GlyphGenerator: NSGlyphGenerator {
    private let glyphStorageDecorator = GlyphStorageDecorator()

    override func generateGlyphs(for glyphStorage: NSGlyphStorage,
            desiredNumberOfCharacters nChars: Int,
            glyphIndex: UnsafeMutablePointer<Int>?,
            characterIndex charIndex: UnsafeMutablePointer<Int>?) {
        // Update decorator; the actual NSLayoutManager will probably stay
        // the same and this is a no-op.
        glyphStorageDecorator.base = glyphStorage

        // Call default class-cluster implementation using the decorator
        NSGlyphGenerator.shared.generateGlyphs(
            for: glyphStorageDecorator,
            desiredNumberOfCharacters: nChars,
            glyphIndex: glyphIndex,
            characterIndex: charIndex)
    }
}

final class GlyphStorageDecorator: NSGlyphStorage {
    var base: NSGlyphStorage?

    init(base: NSGlyphStorage? = nil) {
        self.base = base
    }

    func insertGlyphs(_ glyphs: UnsafePointer<NSGlyph>, length: Int,
            forStartingGlyphAt glyphIndex: Int, characterIndex charIndex: Int) {
        guard let base = base else { assertionFailure(); return }
        // TODO: Hide glyphs here :)
        base.insertGlyphs(pointer, length: len,
            forStartingGlyphAt: glyphIndex, characterIndex: charIndex)
    }

    func setIntAttribute(_ attributeTag: Int, value val: Int,
            forGlyphAt glyphIndex: Int) {
        guard let base = base else { assertionFailure(); return }
        base.setIntAttribute(attributeTag, value: val, forGlyphAt: glyphIndex)
    }

    func attributedString() -> NSAttributedString {
        guard let base = base
        else { assertionFailure(); return NSAttributedString() }
        return base.attributedString()
    }

    func layoutOptions() -> Int {
        guard let base = base else { assertionFailure(); return 0 }
        return base.layoutOptions()
    }
}

This is just a very basic scaffolding. The actual glyph replacements have to be done in insertGlyphs.

To hide every 2nd glyph, this would do the trick:

func insertGlyphs(_ glyphs: UnsafePointer<NSGlyph>, length: Int,
        forStartingGlyphAt glyphIndex: Int, characterIndex charIndex: Int) {
    guard let base = base else { assertionFailure(); return }

    var mutatedGlyphs: [NSGlyph] = []
    // Extra fancy: avoid reallocation by preparing the array size
    mutatedGlyphs.reserveCapacity(length)
    for i in 0...length {
        let glyph = (i % 2 == 0)
            ? NSGlyph(NSNullGlyph)
            : glyphs[i]
        mutatedGlyphs.append(glyph)
    }

    mutatedGlyphs.withUnsafeBufferPointer { bufferPointer in
        guard let pointer = bufferPointer.baseAddress else { fatalError() }
        base.insertGlyphs(pointer, length: length,
            forStartingGlyphAt: glyphIndex, characterIndex: charIndex)
    }
}

The glyphs: UnsafePointer<NSGlyph> parameter actually denotes a C-style array using a pointer. We are guaranteed that it has length elements, so we can use Swift’s pointer subscript to get an element from the array at any index using glyphs[i].

You can probably (!) get by when you make use of this pointer to replace elements at particular indexes, but at this point, we don’t own the glyphs array. The calling context does. And who knows what happens when you mess around with the array. That’s why we operate on a copy.

As with many things that predate Swift for 10+ years, the bridging between NSNullGlyph (type Int) and NSGlyph (typealias for UInt32) is a bit awkward. (Why didn’t they declare let NSNullGlyph: NSGlyph = 0?)

Please also note that in the decades since introducing NSNullGlyph, according to the header files NSNullGlyph is soft-deprecated (aka there’s no compiler warning) and we’re supposed to use NSLayoutManager.GlyphProperty.null instead. That is wrapped in a Swift type, and that’s also a strong indicator that NSLayoutManager is the place to go to override glyphs on-the-fly.

All in all, take this piece as a Swift recreation of a historic document that taught Mac programmers over 10 years ago how to affect the stuff you see on screen. And then brace yourselves for what came afterwards. I’ll keep you posted.

TableFlip v1.3.0 Released

TableFlip v1.3.0 just passed App Store Review. Direct customers had access to the update since yesterday evening. Here’s what’s new:

  • Fixed: Tab bar scrolling. I received reports that you couldn’t scroll horizontally through the list of tabs. This should now be fixed.
  • Fixed: Opening file via URL scheme now requests access permission when needed to comply with App Sandboxing. Previously, the app tried to make a NSDocument from the URL you passed in, but wasn’t allowed to access the file contents in its Sandbox, and thus errored-out. Asking for confirmation once per file is the best we get, it seems.
  • New: New app and preferences icons to match our new Silicon overlords. I also slightly changed the colors inside the app to look better and solve a background color fill glitch or two on the way.

Credits got to Alex Käßner for iterating on the “classic” TableFlip icon and bringing it to the Big Sur era of superellipses, soft gradients, and smooth arrows!

Upgrading to Sparkle 2.x Branch

TL;DR: The Sparkle 2.x release branch is working fine for production when you switch from the ui-separation-and-xpc branch and enable DSA signing of updates.

While updating TableFlip for macOS Big Sur, I figured I might just as well update my dependency on Sparkle to whatever they came up with in the past 18 months or so.

I’m all for the XPC-based approach. In summer of 2019, I wrote a couple of posts about how to migrate to the XPC branch. I am still using the same setup, with custom XPC service bundle identifiers. That change is the only commit on top of the actual code from the Sparkle branch.

Changing the dependency from the ui-separation-and-xpc to the 2.x tracking branch produced problems when I tried to test the updates, though.

Which reminds me that I don’t have a very robust testing approach for updates. It’s all very manual and involved building a version of the app with the build set to something far back in the past to trigger an update download and installation.

During this step, I found that Sparkle couldn’t even download the update, though, and exited like this:

com.apple.xpc.launchd[1] (com.apple.xpc.launchd.user.domain.502.100006.Aqua): de.christiantietze.TableFlip-sparkle-updater (lint): Assuming EnablePressuredExit=false given EnableTransactions=false.
com.apple.xpc.launchd[1] (com.apple.xpc.launchd.user.domain.502.100006.Aqua): de.christiantietze.TableFlip-sparkle-progress (lint): Assuming EnablePressuredExit=false given EnableTransactions=false.
com.apple.xpc.launchd[1] (de.christiantietze.TableFlip-sparkle-updater[96616]): Service exited with abnormal code: 1
com.apple.xpc.launchd[1] (de.christiantietze.TableFlip-sparkle-progress[96617]): Service exited with abnormal code: 1

The error “Service exited with abnormal code: 1” is the least helpful that Sparkle ever produced for me.

Reverting the Sparkle settings and switching branches for a while to figure out what might be going on – I found out I had a typo in the file name. A . was missing to separate file extension from file name:

   <enclosure url=".../MyApp-v1.2.3zip">

Aye, and that’s how you get a Service exited with abnormal code: 1.

On the bright side, I can confirm that the Sparkle 2.x branch works as a drop-in for the ui-separation-and-xpc branch, provided you are DSA-signing your updates. It’s now a hard requirement.

Convert org-babel Markdown Blocks to HTML

During app development, I track the tasks in an org-mode task list. And I track the stuff I finished and want to highlight in the release notes in a Markdown block right there.

A list of tasks for the latest update, and the running list of relese note changes

When I release an update, I’ll copy & paste the Markdown part to the “Release Notes” of the app and push the changes to the server online.

I also convert the Markdown to HTML and paste the HTML in the app’s updater feed. The result will be rendered by the app when people are notified of a new update.

Until now, I opened my update.xml feed in TextMate, pasted the Markdown, then converted it via a macOS system service into HTML. Today, it occured to me that since I do everything in Emacs up until this point, why not also convert the Markdown to HTML? That should be 100% possible and I don’t need to switch the app for that.

The Markdown block in my task list is wrapped in #+begin_src#+end_src. This is an org-babel block. The name’s “Babel” because you can get syntax highlighting and execution of code in various (programming) languages. You can put the cursor anywhere inside the block, hit the idiomatic “execute” shortcut (C-c C-c), and confirm the execution of the code block.

By default, a Markdown source block does nothing. You need to install functions for org-babel to understand Markdown. For some reason, nobody pushed this as a downloadable package to the common package repositories, so all I found is Takahiro Noda’s ob-markdown.el code. You have to save the file and include it manually during the Emacs startup sequence.

The default output format is kind of disappointingly useless

The default output is not that helpful, though: each line is prepended by a colon (:). So copying the result is a pain. This is influenced by the variable org-babel-min-lines-for-block-output – the default value is 10, and this is less than 10 lines of output, so it doesn’t get wrapped in a #+begin_src-block but instead each line is marked as a result output line. (This behavior is the default to make replacing of the output possible by default.)

So you could lower org-babel-min-lines-for-block-output to, say, 3 and that’s great.

But you also can customize the code block’s header arguments to affect the output format. There’s a ton of options. But when you add :results output html, the output is always wrapped in a #+begin_src html block. That makes sense for Markdown conversion.

You can also make this the default for all Markdown code blocks so you don’t have to type these settings all the time.

This can be set as the default for the Markdown format, too:

;; By default, the header arguments for Markdown are empty:
;; (setq org-babel-default-header-args:markdown '())
(add-to-list 'org-babel-default-header-args:markdown
             '(:results . "output verbatim html"))

When you evaluate a Markdown code block now, these settings are applied by default and you get much nicer output.

The Markdown code is converted to an HTML block with beautiful (?) syntax highlighting!

Now it’s ready for copying and pasting into the Update feed.

Making Emacs Autosaving of Files Ignore Email Drafts

I found these fellas, and many more, all of a sudden.

This week I noticed that a ton of files like *message*-20201029-134012 piled up in my home folder. These are email drafts from message-mode, and the numbers are a date-time-stamp.

These didn’t appear ever before. So I figured it might have something to do with autosaving after looking at the change history of my configuration file – and it sure does.

I had enabled auto-save-visited-mode in emacs some time last month. It saves the files you open (“visit”) in Emacs automatically to disk. That’s a fairly new feature since version 26.1: before, you had auto-save-mode, but that would auto-save to a temporary copy of the file you looked at.

Normally, I’d deactivate the autosave feature for email drafts, and that’d be it. But auto-save-visited-mode is a global mode. You can turn it on or off, but the setting affects all buffers for all files. Meh.

The real-auto-save package is a bit more clever. It has the same effect, but can be activated on a per-buffer (or per-file) basis.

I just created a PR on GitHub to create a global mode so that I don’t have to selectively enable real-auto-save. I want to have it on by default, and deactivate for e.g. message-mode.

I’m no Emacs Lisp wiz, but this was pretty simple (after looking at how existing packages do such things).

I basically executed this code to test if a global mode would work, and then submitted a PR with the relevant changes:

;; Load the package
(require 'real-auto-save)
(setq real-auto-save-interval 10) ;; in seconds

;; Helpers to enable/disable the note per-buffer
;; (so I don't have to use lambdas)
(defun turn-on-real-auto-save () (real-auto-save-mode 1))
(defun turn-off-real-auto-save () (real-auto-save-mode -1))

;; Here's the kicker: add a global mode!
(define-globalized-minor-mode global-real-auto-save-mode
  real-auto-save-mode turn-on-real-auto-save)

;; Enable globally
(global-real-auto-save-mode 1)

;; Disable for message-mode
(add-hook 'message-mode-hook #'turn-off-real-auto-save)

In my layman understanding, define-globalized-minor-mode takes 3 arguments:

  1. The global mode’s name
  2. The corresponding local mode’s name (which has no effect on it’s own apart from the name association?)
  3. The global mode execution function (which actually turns this on)

The global minor modes are enabled first, then the hook for having an email buffer open is executed, and I don’t get the autosaving files anymore. Nice!

I have to figure out why the drafts (1) appeared in these locations, since they don’t match any of my configuration, and (2) why they didn’t auto-delete when sending the email.

Zettelkasten Method Introduction

Sascha wrote an introduction to the Zettelkasten Method, and after months of editing and polishing, we finally published in online!

What’s a Zettelkasten, you ask?

Think personal wiki of knowledge. But instead of merely cartographing what is in the world, you create new stuff from the things you collect, building up layers upon layers of ideas by connecting what you already have, and then making a big hypertext from it.

At first you have atomic ideas, then you begin to link them and think and write about their connection.

In other words, it’s a way to add layers of abstractions to your thinking and writing.

I use something like it for 11+ years now and the amount of cross-connections is very cool. Also, by using my Zettelkasten properly, I “accidentally” prepare blog posts for this site: every note is self-contained and written in a semi-publishable manner, so I could just copy and paste notes together to make an article. That’s the sweetest part of it: how it accelerates my writing.

Check out the intro and see how simple the actual implementation is.

Hide Traffic Light Buttons in NSWindow Without Removing Resize Functionality

I noticed that in macOS’s dark mode, a window without a visible title bar won’t draw a light border. It will draw a dark/black border, unlike all the other windows, and thus be a lot less visible than it needs to be.

So for a floating helper window, I had to make the title bar visible, and then hide all traffic light buttons in the window’s top-left corner:

class WindowController: NSWindowController {
    override func windowDidLoad() {
        super.windowDidLoad()

        // Hide the textual window title.
        self.window?.titleVisibility = .hidden
        self.window?.styleMask = []

        // We need a textured window to get the correct border color.
        self.window?.styleMask.insert(.texturedBackground)

        // Making the window resizable will show the traffic lights automatically ...
        self.window?.styleMask.insert(.resizable)
        // ... so hide all traffic light buttons manually:
        self.window?.standardWindowButton(.closeButton)?.isHidden = true
        self.window?.standardWindowButton(.miniaturizeButton)?.isHidden = true
        self.window?.standardWindowButton(.zoomButton)?.isHidden = true
        self.window?.standardWindowButton(.fullScreenButton)?.isHidden = true
    }
}

Until today, I didn’t know that NSWindow even has standardWindowButton(_:) that you can use to get to the traffic lights!

With this setup, you get resizability of the window, which would usually result in at least 1 traffic light to show, and a full-sized content view without any visible title bar at all.

Shorten Emacs Yes-or-No Confirmation Dialogs

Press ‘y’ to confirm!

Some commands in Emacs and its various packages are destructive. They require a confirmation by the user. These usually use yes-or-no-p, which won’t complete the command until the user replies by writing “yes” and then hits enter, or “no”, or aborts the command with C-g.

Some of the commands that require confirmation in this way are overly protective, I find. Like projectile-kill-buffers (C-p k) which closes all open buffers for a project at once. I use this heavily when e.g. editing my website: jump into the project, edit a file or two, commit, then leave. I don’t want to type “yes” and hit enter just for that. (Please note that killing a buffer this way will still ask me if I want to save or discard changes, so the kill command alone is not destructive.)

There’s a shorter variant of the confirmation, y-or-n-p, where you only have to hit the “y” or “n” key to select an answer. It’s more like a dialog prompt and less like a conversational chat UI.

Ingenious people on the interwebs figured out that you can temporarily, or rather locally, override the yes-or-no-p with the shorter y-or-n-p by using what’s called an “advice”. It’s basically a function decorator. If all you know is Objective-C, think “method swizzling”.

(defun yes-or-no-p->-y-or-n-p (orig-fun &rest r)
  (cl-letf (((symbol-function 'yes-or-no-p) #'y-or-n-p))
    (apply orig-fun r)))

(advice-add 'projectile-kill-buffers :around #'yes-or-no-p->-y-or-n-p)

With this code, when I run projectile-kill-buffers, the actual call ((apply orig-fun r)) is wrapped in a context where all yes-or-no-p calls will actually invoke y-or-n-p. I love it.

Follow Link at Insertion Point in NSTextView

More power for keyboard shortcut lovers is always good!

If you ever wondered how to programmatically trigger a click on a link in a NSTextView, here’s one way to do so.

This assumes that clickable links are not stored as temporary attributes in the NSLayoutManager, but permanently as part of the “model” in your NSTextStorage. You can then ask the storage for the attribute at the cursor/insertion point location:

extension NSTextView {
    func followLinkAtInsertionPoint(_ sender: Any?) {
        guard let textStorage = textStorage else { return }

        let location = selectedRange().location
        let attributes = textStorage.attributes(at: location, effectiveRange: nil)

        guard let url = attributes[.link] as? URL else { return }

        self.clicked(onLink: url, at: location)
    }
}

To further refine this and make it work with selected ranges properly, you’d need to find all links in the selected range. Call NSTextStorage.attributes(at:longestEffectiveRange:in:) repeatedly for this.

For example, if your selection spans let wholeRange = NSRange(location: 100, length: 50), you call attributes(at: 100, longestEffectiveRange: &effectiveRange, in: wholeRange). The storage produces an effectiveRange clipped to wholeRange. It might be smaller, e.g. equal to NSRange(location: 100, length: 10), so the range from 11 to 50 is still uncovered. If effectiveRange is fully contained within wholeRange like this, you can call the method again with the location parameter advanced by effectiveRange.length + 1 to 111. Then you get the next piece. Rinse and repeat until wholeRange is exhausted.

This is how a rich text document with nested styles, e.g. underlining a portion of an italic text, is linearized. Users think of “nested” styles, but the computer would return a range of “italics only”, followed by “italics + underlines”, followed by “italics only” again. That’s how you would iterate over the attributes, too.


→ Blog Archive