Instead of writing view model types that expose the input and/or output ports as properties, consider writing free functions that are the transformation.

This reminds me of Ken Scambler’s proposal to get rid of test doubles through side-effect free function composition. One of his examples, simplified, goes like this: Instead of testing what customer.pay(amount, wallet) does in your tests by mocking the wallet object, you rewrite Wallet as a value object and change pay to return the effect.

Before:

class Wallet {
    private var cash: Int

    func takeOut(amount: Int) {
        self.cash -= amount
    }
}

extension Customer {
    func pay(amount: Int, wallet: Wallet) {
        // Need to mock `wallet` reference type in your tests
        wallet.takeOut(amount: amount)
    }
}

After:

struct Wallet {
    var cash: Int
}

func pay(amount: Int, wallet: Wallet) -> Wallet {
    var result = wallet
    result.cash -= amount
    return result
}

This way, the original Wallet isn’t affected and you can test the payment action in isolation. Ken’s examples are more sophisticated than that, but it should get the idea across.

Now DJ Mitchell proposes a similar approach to RxSwift view models: make them mere transformations of one set of observables into another, e.g. of this form: (Observable<T1>, Observable<T2>, ...) -> (Observable<U1>, Observable<U2> ...)

Usually, in your view code, you end up with PublishSubject/PublishRelay as to generate output events that you expose as Observable. This dance is a well-established pattern to bridge between the non-reactive and reactive world. It’s useful to expose an OOP message input (e.g. accept changeUserName(_:) messages) and expose a reactive observable sequence output (e.g. userName: Observable<String>).

But when you write reactive code, you can end up with input events as observable event sequences anyway. There’s no need to use non-reactive messages in some cases.

This is where direct transformations kick in.

Look at the example by DJ:

func loginViewModel(
    usernameChanged: Observable<String>,
    passwordChanged: Observable<String>,
    loginTapped: Observable<Void>
) -> (
    loginButtonEnabled: Observable<Bool>,
    showSuccessMessage: Observable<String>
) {
    let loginButtonEnabled = Observable.combineLatest(
        usernameChanged,
        passwordChanged
    ) { username, password in
        !username.isEmpty && !password.isEmpty
    }

    let showSuccessMessage = loginTapped
        .map { "Login Successful!" }

    return (
        loginButtonEnabled,
        showSuccessMessage
    )
}

You have 3 input streams and return a tuple of 2 output streams. That pretty easy to test thorougly thanks to RxTest and its timed event recording. And it reduces the actual amount of ugly RxSwift binding code.

You can get from a lot of input/output bindings like this:

disposeBag.insert(
    // Inputs
    loginButton.rx.tap.asObservable()
        .subscribe(onNext: viewModel.inputs.loginTapped),

    usernameTextField.rx.text.orEmpty
        .subscribe(onNext: viewModel.inputs.usernameChanged),

    passwordTextField.rx.text.orEmpty
        .subscribe(onNext: viewModel.inputs.passwordChanged),

    // Outputs
    viewModel.outputs.loginButtonEnabled
        .bind(to: loginButton.rx.isEnabled),

    viewModel.outputs.showSuccessMessage
        .subscribe(onNext: { message in
            print(message)
            // Show some alert here
        })
)

To a transformation call and a lot less bindings here:

let (
    loginButtonEnabled,
    showSuccessMessage
) = loginViewModel(
    usernameChanged: usernameTextField.rx.text.orEmpty.asObservable(),
    passwordChanged: passwordTextField.rx.text.orEmpty.asObservable(),
    loginTapped: loginButton.rx.tap.asObservable()
)

disposeBag.insert(
    loginButtonEnabled
        .bind(to: loginButton.rx.isEnabled),

    showSuccessMessage
        .subscribe(onNext: { message in
            print(message)
            // Show some alert here
        })
)

I’m constantly refactoring my RxSwift-based code as I learn new things. I found view model types to be pretty useless in the long run. Writing tests for them is cumbersome because they are the transformations and expose outputs as properties, accepting inputs in the initializer. I end up with lots of object setup boilerplate. They don’t need to be actual objects since they expose to actual behavior. They are already mere value transformations, only wrapped in a type.

I guess this is a natural step when you start with OOP in mind and think about sensible encapsulations in types. But sometimes, like in DJ’s example, free functions work just as well to express your value transformation intent.

The Cocoa/AppKit documentation is very sparse when it comes to tabbing. You can make use of the native window tabbing introduced in macOS Sierra with a few simple method calls, though.

Conceptually, this is what you will need to do:

• Set NSWindow.tabbingMode to .preferred so the tab bar will always be visible.
• It suffices to call NSWindow.addTabbedWindow(_:ordered:) to add a window to the native tab bar and get everything tabs do for free.
• Once you put NSResponder.newWindowForTab(_:) into the responder chain of the main window, the “+” button in the tab bar will be visible.

However, there are some caveats when implementing these methods naively. The plus button may stop working (no new tabs are added when you click it) and all default shortcuts are broken, their main menu items greyed out.

How to Implement newWindowForTab

First, where to add @IBAction override func newWindowForTab(_ sender: Any?)? That’ll be the event handler to create new tabs.

• If you use Storyboards, then put this into a NSWindowController subclass you own. That’s the simplest way to get to an NSWindow to call addTabbedWindow for.
• If you use Xibs, the AppDelegate will have a reference to the main window. You can put the method here.
• If you use a programmatic setup, put it wherever you know the main NSWindow instance.

We’ll stick to NSWindowController for the rest of this post, no matter how you create it:

class WindowController: NSWindowController {
    @IBAction override func newWindowForTab(_ sender: Any?) {
        // Implementing this will display the button already
    }
}

How to Call addTabbedWindow

Once you have newWindowForTab(_:) in place, add functionality to it: create a new NSWindow and add it to the tab bar.

• If you use Storyboards, grab the instance via NSWindowController.storyboard; then instantiate a new window controller instance, for example using self.storyboard!.instantiateInitialController() as! WindowController.
• If you use Xibs with a NSWindowController as the File’s Owner, create an identical window controller using NSWindowController.init(windowNibName:). Use its window property, discard the controller.
• If you use Xibs with a NSWindow only and no controller, get the window from there.
• If you use programmatic setup, well, instantiate a new object of the same window type as usual.

When you have the new window object, you can call addTabbedWindow. Using the Storyboard approach, for example, turns the implementation into this:

class WindowController: NSWindowController {
    @IBAction override func newWindowForTab(_ sender: Any?) {
        let newWindowController = self.storyboard!.instantiateInitialController() as! WindowController
        let newWindow = newWindowController.window!
        self.window!.addTabbedWindow(newWindow, ordered: .above)
    }
}

Fix the “+” Button and Main Menu

TL;DR: When you initialize a new window, set window.windowController = self to make sure the new tab forwards the responder chain messages just like the initial window.

Take into account how events are dispatched. Main Menu messages are sent down the responder chain, and so is newWindowForTab. NSApp.sendAction will fail for standard events if the source of the call doesn’t connect up all the way – that means, at least up to your NSWindowController, maybe even up to your AppDelegate.

You have to make sure any additional window you add is, in fact, part of the same responder chain as the original window, or else the menu items will stop working (and be greyed-out/disabled). Similarly, the “+” button stops to work when you click on it.

If you forget to do this and run the code from above, it seems you can’t create more than two tabs. That’s the observation, but it’s not an explanation. You can always create more tabs, but only from the original window/tab, not the new one; that’s because the other tab is not responding to newWindowForTab.

Remember: “The other tab” itself is just an NSWindow. Your newWindowForTab implementation resides in the controller, though. That’s up one level.

class WindowController: NSWindowController {
    @IBAction override func newWindowForTab(_ sender: Any?) {
        let newWindowController = self.storyboard!.instantiateInitialController() as! WindowController
        let newWindow = newWindowController.window!

        // Add this line:
        newWindow.windowController = self

        self.window!.addTabbedWindow(newWindow, ordered: .above)
    }
}

Now newWindow will have a nextResponder. This will fix message forwarding.

Using Multiple Window Controllers

The solution above shows how to add multiple windows of the same kind, reusing a single window controller for all of them.

You can move up newWindowForTab one level to another service object, say the AppDelegate. Then you could manage instances of NSWindowController instead of instances of NSWindow. I don’t see why you would want to do that if you can share a single controller object.

I haven’t tried to do anything fancy besides, but you should be able to use different windows and different window controllers and group them in the tab bar of a single host window. You will then need to keep the window controller instances around, too.

Last year, I had a full-time job from May until November and haven’t had much time to prepare for Mojave. Then it hit hard, and I am still tweaking my apps to look properly in Mojave’s Dark Mode. I welcome that macOS offers two modes now, .aqua and .darkAqua in programmer parlance. Because then I don’t have to implement a homebrew solution in all my writing applications.

One troublesome change is NSColors adaptability: system colors like NSColor.textBackground (white in light mode, jet black in dark mode) or NSColor.gridColor (an 80%ish grey color in light mode, a transparent 30%ish gray in dark mode) are very useful, but some custom interface elements need different colors to stand out visually. But you cannot create adaptive NSColor objects programmatically, only via Asset Catalogs. And these don’t even work on macOS 10.11. So you limit yourself to macOS 10.13 High Sierra and newer if you use Asset Catalogs at all, even though only the light variant will be exposed there. Increasing the deployment target from 10.10 to 10.13 just for the sake of easy color changes for 10.14’s dark mode is lazy. (If you have other reasons, go for it! It’s always nice to drop backwards compatibility and use the new and cool stuff where possible.)

So what options do you have?

First, check out Daniel Jalkut’s excellent series on dark mode changes. He covers a lot of ground. Chances are you don’t need to know more than what he explains in his posts.

But how do you react to changes to the appearance? Daniel uses KVO on NSApp.effectiveAppearance, and I had mixed results with observing changes to this attribute. Also, it’s macOS 10.14+ only.

Still I want a NSNotification. Since macOS 10.9, you have NSAppearanceCustomization.effectiveAppearance, which NSView implements. So even if you cannot ask your NSApplication instance for its effective appearance, you can ask your window’s main contentView. That’s what we’ll do.

If you want to see all code, take a look at the accomanying GitHub Gist.

NSAppearance Convenience Code

First, a simple boolean indicator isDarkMode would be nice:

// Adapted from https://indiestack.com/2018/10/supporting-dark-mode-checking-appearances/
extension NSAppearance {
    public var isDarkMode: Bool {
        if #available(macOS 10.14, *) {
            if self.bestMatch(from: [.darkAqua, .aqua]) == .darkAqua {
                return true
            } else {
                return false
            }
        } else {
            return false
        }
    }
}

Now either use KVO on your main view, or expose a reactive stream of value changes.

My latest app, The Archive, uses RxSwift and other reactive patterns wherever possible. I found this to be a very nice way to decouple parts of my program, so I wanted to stick to that approach. RxSwift adds “Reactive eXtensions” to types in the .rx namespace of objects. That means I want to end up using window.contentView.rx.isDarkMode and get an Observable<Bool> instead of the momentary value. Notice the “rx” in the middle: the idea is that your own extensions use the same attribute names but inside the reactive extension to wrap changes in an Observable sequence.

Here, look at this. You write your own reactive extension for NSView in extension Reactive where Base: NSView { ... }, like that:

import Cocoa
import RxSwift
import RxCocoa

extension Reactive where Base: NSView {

    /// Exposes NSView.effectiveAppearance value changes as a
    /// never-ending stream on the main queue.
    var effectiveAppearance: ControlEvent<NSAppearance> {
        let source = base.rx
            // KVO observation on the "effectiveAppearance" key-path:
            .observe(NSAppearance.self, "effectiveAppearance", options: [.new])
            // Get rid of the optionality by supplementing a fallback value:
            .map { $0 ?? NSAppearance.current }
        return ControlEvent(event: source)
    }

    /// An observable sequence of changes to the view's appearance's dark mode.
    var isDarkMode: ControlEvent<Bool> {
        return base.rx.effectiveAppearance
            .map { $0.isDarkMode }
            .distinctUntilChanged()
    }
}

Now I can subscribe to any of the window’s view changing.

Post Notifications from Main Window Changes

As I said, I want to have notifications, e.g. in places where there is no associated view or view controller. That’s where the settings are wired up and the current editor theme switches from light to dark or vice-versa in accordance with the app’s appearance.

First, I define some convenience constants for the notification sending:

extension NSView {
    // Used as Notification.name
    static var darkModeChangeNotification: Notification.Name { return .init(rawValue: "NSAppearance did change") }
}

extension NSAppearance {
    // Used in Notification.userInfo
    static var isDarkModeKey: String { return "isDarkMode" }
}

To post the notification, I have to subscribe to any view in the view hierarchy and ask for it’s observable stream of dark mode changes:

window.contentView.rx.isDarkMode
    .startWith(NSAppearance.current.isDarkMode)
    .subscribe { isDarkMode in
        NotificationCenter.default.post(
            name: NSView.darkModeChangeNotification,
            object: nil,
            userInfo: [NSAppearance.isDarkModeKey : isDarkMode])
    }.disposed(by: disposeBag) // Assuming you have one around

It’s important to note that the view has to be in the view hierarchy (and, I think, visible) to be updated at all. I tried observing a stand-alone NSView() in a service object that had no knowledge about the app’s UI, but to no avail. Doesn’t work. So I resort to the main window’s contentView which will be around while the app is running.

Subscribe to the notification as usual.

Or employ RxSwift’s NotificationCenter extensions: “`swift

let darkModeChange = NotificationCenter.default.rx .notification(NSView.darkModeChangeNotification) .map { $0.userInfo?[NSAppearance.isDarkModeKey] as? Bool } // Equivalent to a filterNil(): ignore incomplete notifications .filter { $0 != nil }.map { $0! }

darkModeChange .subscribe(onNext: { print(“Dark mode change to”, $0) } .disposed(by: disposeBag) “`

So that’s what I use and I’m content with the solution for now. I can switch colors manually where needed and reload the editor theme for the app’s current appearance.

Thanks again to Daniel Jalkut and his research!

To see all the code in one place, have a look at the GitHub Gist that goes with this post.

I just published an update to the WordCounter for Mac that modernizes the UI and typography a bit to look rad on Mojave in both dark and light modes. It also fixes pertinent issues with “Launch at Login” not doing what it’s supposed to do, and fixes a couple of small bugs.

But probably most noteworthy, this is the first public release since 2016! Publishing updates stalled because of … managerial problems. I introduced the Swift programming language to this Objective-C project a couple years back, and didn’t keep up with language updates. So I had to take a couple of days to update the Swift code base. And then fix crashing tests, key–value-observation issues with Swift objects – in short, I failed to maintain this project on a regular basis, and once it became harder to get back to speed, I deferred touching the app even longer, making matters worse.

And all that while a new feature lay dormant, 3/4th done.

This updates means serious business. I’m back to make the app even cooler. Expect nice things to follow early 2019.

Also expect the latest update not to work on macOS 10.9, 10.10, or 10.11 anymore. You will need macOS 10.12 Sierra or newer from this day forward.

Check out the release notes for a breakdown of the changes.

When you write a Swift type, you should prefer to write its tests in Swift, too.

In a mixed-language code base, you will run into the situation where your Objective-C tests need to reference Swift types to initialize and pass to the object under test. That’s a different story.

But you cannot import the generated -Swift.h out-of-the-box:

#import "MyProject-Swift.h"

Problem is, this generated code doesn’t reside in the project directory, but in the build directory. More specifically, it resides in

$CONFIGURATION_TEMP_DIR/MyProject.build/DerivedSources

Once you add this line (and replace “MyProject” with the appropriate name) to your unit test target’s Build Settings > Header Search Paths, importing your app’s generated Swift header file will work.

Let’s hope neither you nor I will ever forget how this works again.

Two years ago, I wrote about how I implemented a toolbar with NSSegmentedControl, much like Apple’s own apps have them since macOS 10.11 Yosemite. Last week I discovered my implementation was buggy: it did not work at all when you customize the toolbar to show “Icon Only”, i.e. hide the label text.

1. Original Approach
2. Enabling/Disabling Segments
3. Fixing the action dispatching bug (this post)

NSToolbarItems are not activated without labels

My interpretation of the situation is this: when you show labels, the NSToolbarItem is responsible for providing the label below the segments of the control. It also is responsible for dispatching an action on click. This works with “Icon and Text” and “Text Only”. But once you hide the labels, the NSToolbarItems do not do anything at all – neither display, nor action dispatching.

The nice thing about splitting a single NSSegmentedControl into multiple NSToolbarItems was that you were able to bind each item to its own action, which magically corresponded to the segments of your control. Unlike other NSView-based controls, NSSegmentedControl’s segments are not NSView based themselves, and thus cannot be NSControl types which handle their own action dispatching. Instead, they are NSCell subclasses that mostly handle drawing. NSSegmentedControl is the only one in the setup that has an action property that takes a selector. It’s the only one responding to events.

Change the setup to make the NSSegmentedControl respond to events

So let’s change everything and let the NSSegmentedControl do all the work.

Why transfer ownership of action dispatch instead of mix both approaches? – Because then you have two approaches to maintain. And I don’t want to manually test all buttons in all toolbar configurations.

The course of action is thus:

1. Make the universally functional NSSegmentedControl.action the main click handler.
2. Leave NSToolbarItem.action = nil so the control doesn’t get overridden.
3. From the NSSegmentedControl.action handler, dispatch the real segment’s action, depending on which the user did click.

If you’re lazy, you’d be implementing the toolbar actions in your NSWindowController since that’s guaranteed to be somewhere in the responder chain.

But we ain’t no lazy folk in this here web space!

Also, the app I’m using this for, TableFlip, was creating the old setup in a factory outside of any view or window controller. And I wasn’t going to sacrifice this one good design choice by pasting everything back into the window controller.

Questions raised:

• Where to put the general-purpose action?
• How to delegate to a single segment’s real action?

There was no meaningful name I could give a general-purpose action that was going to respond to clicks to any segment. segmentSelected(_:) was the best name I could come up with: it is not an expression of a user intent, it’s an implementation detail. The user intent, like ”addRow(_:) to this table”, is bound to the segments; the NSSegmentedControl grouping does not carry any meaning except the grouping in the UI. Again, an implementation detail. That’s a good reason to stick to a method name that would otherwise be considered code smell.

Which object shall become the dispatcher? In my opinion, this setup is a deficit of AppKit’s current API. So I think the best place to handle this is an NSSegmentedControl itself:

class DispatchingSegmentedControl: NSSegmentedControl {
    func wireActionToSelf() {
        self.target = self
        self.action = #selector(segmentSelected(_:))
    }

    @IBAction func segmentSelected(_ sender: Any) {
        // Dispatch according to `self.selectedSegment`
    }
}

The implementation for this could instead be part of a controller object. But this is not an event I want to respond to; it’s an event the view component should transform in accordance with its internals, with its setup of segments, into a more specific event.

You could have a Array<(target: Any?, action: Selector)> that corresponds to the segments, and get to it using selectedSegment inside of segmentSelected(_:). Or you whip up a more elaborate configuration. I did the latter, and will show you my stuff in the remainder of this post.

My approach to define the segments

Note: I am still using Interface Builder to configure the NSSegmentedControl. So I do start with instances of these, with icons and segment widths pre-configured.

I do have to overlay the target–action-mechanism, though, similar to the overlay by NSToolbarItem used before.

Representation of the segment configurations

The view model type I introduces is called ToolbarSegmentedControlSegment. It represents the configuration of a segment:

struct ToolbarSegmentedControlSegment {

    var toolbarItemIdentifier: NSToolbarItem.Identifier
    var label: String
    var action: Selector
    var menuTitle: String?
    var menuImage: NSImage?

     init(toolbarItemIdentifier: NSToolbarItem.Identifier,
          label: String,
          action: Selector,
          menuTitle: String? = nil,
          menuImage: NSImage? = nil) {
        self.toolbarItemIdentifier = toolbarItemIdentifier
        self.label = label
        self.action = action
        self.menuTitle = menuTitle
        self.menuImage = menuImage
    }
}

It offers a factory method to get to a NSToolbarItem representation. This is still needed to display the labels in the toolbar, and to handle the toolbar overflow menu. This is what I was doing in the original NSToolbarItem setup, now moved here:

extension ToolbarSegmentedControlSegment {
    func toolbarItem() -> NSToolbarItem {
        // Do not set the item's action and rely on the ToolbarSegmentedControl instead.
        // This makes it easier to always be running into the same bugs, if any,
        // and not have 2 paths :)
        let item = NSToolbarItem(itemIdentifier: toolbarItemIdentifier)
        item.label = label
        item.updateMenuFormRepresentation()
        item.menuTitle = menuTitle
        item.menuImage = menuImage
        return item
    }
}

It can also execute the action dispatch:

extension ToolbarSegmentedControlSegment {
    func dispatchAction() {
        NSApp.sendAction(action, to: nil, from: nil)
    }
}

And that’s it for that!

Wiring the segment configurations into the NSSegmentedControl

Lastly, the changes to the view component. Remember that I am using Interface Builder, so I rely on configuration after initialization. If you create your control programmatically, you can change things up a bit, like wire the action to self in the initializer, and take an array of segment configurations as initializer parameter as well.

I’m storing the segment configurations in an array and mutate it using the addSegment method, which acts as a factory.

class ToolbarSegmentedControl: NSSegmentedControl {

    var segments: [ToolbarSegmentedControlSegment] = []

    func addSegment(toolbarItemIdentifier: NSToolbarItem.Identifier,
                    label: String,
                    action: Selector,
                    menuTitle: String? = nil,
                    menuImage: NSImage? = nil) {
        guard !segments.contains(where: { $0.toolbarItemIdentifier == toolbarItemIdentifier }) else { return }

        let segment = ToolbarSegmentedControlSegment(
            toolbarItemIdentifier: toolbarItemIdentifier,
            label: label,
            action: action,
            menuTitle: menuTitle,
            menuImage: menuImage)
        segments.append(segment)
    }

    func toolbarItems() -> [NSToolbarItem] {
        return segments.map { $0.toolbarItem() }
    }

    func wireActionToSelf() {
        self.target = self
        self.action = #selector(segmentSelected(_:))
    }

    @IBAction func segmentSelected(_ sender: Any) {
        segments[selectedSegment].dispatchAction()
    }
}

With these convenient wrappers in place, I can setup a whole NSToolbarItemGroup:

// Get the view component from Interface Builder
let toolbarSegmentedControl: ToolbarSegmentedControl! = ...

// Configure segments
toolbarSegmentedControl.addSegment(
    toolbarItemIdentifier: .init(rawValue: "alignLeft"),
    label: "Left",
    action: #selector(TableInteractions.alignColumnLeft(_:)),
    menuTitle: "Align Left",
    menuImage: Alignment.left.image)
toolbarSegmentedControl.addSegment(
    toolbarItemIdentifier: .init(rawValue: "addCenter"),
    label: "Center",
    action: #selector(TableInteractions.alignColumnCenter(_:)),
    menuTitle: "Center",
    menuImage: Alignment.center.image)
toolbarSegmentedControl.addSegment(
    toolbarItemIdentifier: .init(rawValue: "alignRight"),
    label: "Right",
    action: #selector(TableInteractions.alignColumnRight(_:)),
    menuTitle: "Align Right",
    menuImage: Alignment.right.image)

toolbarSegmentedControl.wireActionToSelf()

let itemGroup = NSToolbarItemGroup(itemIdentifier: .init("alignmentGroup"))
itemGroup.view = toolbarSegmentedControl
itemGroup.subitems = toolbarSegmentedControl.toolbarItems()

// return `itemGroup` from  `toolbar(_:itemForItemIdentifier:willBeInsertedIntoToolbar:)`

Conclusion

Even though I hesitated at first, putting the NSSegmentedControl.action handler into a subclass itself and call it segmentSelected(_:) was the right choice. It’s an implementation detail on the code level, and I want to encapsulate forwarding segment selection to in that type.

Configuring a segment and thus preparing a NSToolbarItem is quite a bit of ceremony – there are just so many properties you have to set. But I prefer this over the standard AppKit approach any day, which would have you write the property settings in a procedural fashion. I like that these configuration object initializers group all the options, and that I get a couple of methods in for free which I don’t have to write elsewhere.

All in all, it’s still surprisingly cumbersome to set up a NSToolbarItemGroup around a NSSegmentedControl in AppKit. But it’s manageable, and now that you know the individual subitems’s action dispatching cannot be trusted, you’re better off with a self-made solution.

Ever wanted to implement a full-text search in your app? Didn’t find a boolean search expression parser that works? Look no further!

For the initial release of my note-taking app The Archive I had to create a couple of open source libraries that I have yet to talk about. Today, I want to show you my search expression parser. It powers the app’s Omnibar to find notes quickly using simple boolean expressions.

Searching in a Pile of Notes

A prerequisite for me was to have an interface that’d work well with C Strings. strstr is just the fastest way to look for a needle in a haystack. It’s crazy. To utilize the full potential of full-text C String search, though, you should keep an all-caps or all-lowercase version of the note around so you can do a case-insensitive search. While case-insensitive searches are slower when you perform them on-the-fly, the results are stunning on a bre-built index that doesn’t distinguish upcase and downcase anymore.

The actual objects in my app have a bit more information, but this is a good-enough approximation of what a note representation in the in-memory index looks like:

struct IndexableNote {
    private let cString: [CChar]

    init(text: String) {
        self.cString = text
            // Favor simple over grapheme cluster characters
            .precomposedStringWithCanonicalMapping
            .cString(using: .utf8)!
    }
}

The original String doesn’t matter for the purpose of indexing. All we want to do is perform fast searches.

My first naive implementation of a full-text search separated the search string at every space. So foo bar baz would search for foo, bar, and baz in every note. To make the parts easier to search for, convert them to C Strings as well.

Here’s the implementation that served me well through the beta and into the v1.0 release:

struct IndexableSearchString {
    private let cStringWords: [[CChar]]

    init(_ string: String) {
        self.cStringWords = string
            // Favor simple over grapheme cluster characters
            .precomposedStringWithCanonicalMapping
            .lowercased()
            .split(separator: " ")
            .flatMap { $0.cString(using: .utf8) }
    }

    func matchesAll(in haystack: [CChar]) -> Bool {
        for needle in cStringWords {
            if strstr(haystack, needle) == nil { return false }
        }

        return true
    }
}

extension IndexableNote {
    func matches(searchString: IndexableSearchString) -> Bool {
        searchString.matchesAll(in self.cString)
    }
}

You prepare the search once, then match it against each note in the index as a filter. That’s it. It’s super simple, but it works well for 99% of all use cases. This is pretty close to a Google search already: You enter a list of terms and want search results that contain all of them.

Then there are tech-savvy people, though, who want to exclude search terms and use boolean OR to search for variants, like “banana OR apple OR fruit”. To cater to their needs, I wrote a search expression parser that does just that, and which provides the C String matching that proved to be so useful.

The Boolean Search Expression Parser

I wrote the SearchExpressionParser library with note-taking apps in mind. Search terms had to be human-readable enough for a layperson to understand what’s going on. That’s why operators are all caps: AND, OR, and NOT/!.

The library behaves as follows:

• foo bar baz is equivalent to foo AND bar AND baz
• NOT b equals !b
• ! b (note the space) is ! AND b
• "!b" is a phrase search for “!b”, matching the literal exclamation mark
• Escaping works in addition to phrase search, too: \!b also searches for ”!b”.
• Escaping inside phrase searches also works: hello "you \"lovely\" specimen"
• Escaping operator keywords treats them literal: \AND.

Note that a lowercase “and” will not be treated as an operator, only all-caps ”AND” will. So there’s no need to escape a lowercase \and, for example.

You can parenthesize expressions:

!(foo OR (baz AND !bar))

That evaluates to an equivalent of:

!foo OR !baz AND !foo OR !bar

As of yet, there is no real operator precedence implementation. I didn’t need that, and I discovered not every full-text search implements this correctly at all. So instead of operator precedence that satisfies math-nerds, logicians, and programmers, I roll with a strict left-to-right approach.

The Expression object tree of the nested term above looks like this, by the way:

// !(foo OR (baz AND !bar))
NotNode(
    OrNode(lhs: ContainsNode("foo"), 
           rhs: AndNode(lhs: ContainsNode("baz"), 
                        Rhs: NotNode(ContainsNode("bar")))))

That’s what you’ll get from the parser. It’s a self-evaluating expression node tree.

Expressions are not optimizing themselves to abort quickly; instead, the whole expression tree will be traversed and checked if the operators permit. Since an AndNode simply combines the result of the left-hand side with the right-hand side using Swift’s && operator, the right-hand side could be skipped if the left-hand side evaluates to false already. In the best-case scenario, this is just as efficient as regular boolean expressions in Swift.

This also means that !(a OR b) will result in:

NotNode(OrNode(lhs: ContainsNode("a"), rhs: ContainsNode("b")))

Since the underlying || operator always evaluates both sides, this is less efficient than the equivalent term !a AND !b.

But does it matter in your case?

If so, pull requests with boolean expression normalization are welcome, of course!

It didn’t matter to me. When people compose intricate expressions, well, then I think they’re using their note archive in a very peculiar way. I don’t see the benefit of spending extra work on a normalizer when I could be adding features that benefit a much wider audience.

Using the SearchExpressionParser API

The SearchExpressionParser API exposes Parser.parse(searchString:) that you’ll be using:

import SearchExpressionParser
guard let expr = try? Parser.parse(searchString: "Hello") else { fatalError() }
expr.isSatisfied(by: "Hello World!") // true
expr.isSatisfied(by: "hello world!") // false

The parser can potentially throw an error, but all errors you’ll get are programmer errors on my side. There are no regular error conditions. When an error gets thrown here, please report it, because it’s a bug.

The library provides a CStringExpressionSatisfiable protocol to perform my beloved strstr search instead of the more literal and much slower String.contains. It will also make the search case-insensitive.

To implement this, take the IndexableNote from above and modify it to meet the API criteria:

struct IndexableNote {
    private let cString: [CChar]

    init(text: String) {
        self.cString = text
            // Favor simple over grapheme cluster characters
            .precomposedStringWithCanonicalMapping
            .cString(using: .utf8)!
    }
}

import SearchExpressionParser

extension IndexableNote: CStringExpressionSatisfiable {
    func matches(needle: [CChar]) -> Bool {
        return strstr(self.cString, needle) != nil
    }
}

That’s all you need to get lightning-fast case-insensitive search:

guard let expr = try? Parser.parse(searchString: "Hello") else { fatalError() }
expr.isSatisfied(by: IndexableNote(text: "Hello World!")) // true
expr.isSatisfied(by: IndexableNote(text: "hello world!")) // true

Let the boolean expression parser do its magic for you:

let warAndPeace = IndexableNote(text: String(contentsOf: "books/Tolstoy/War-and-Peace.txt"))
let protagonist = try! Parser.parse(searchString: "\"Pierre Bezukhov\" OR \"Pyotr Kirillovich\"")
protagonist.isSatisfied(by: warAndPeace) // true

That’s all there is to the power of The Archive’s search expressions! They were pretty fun to implement and make searching for relevant notes in your note archive much easier, e.g. hyperlink (#zettelkasten OR #note-taking). With The Archive’s saved search feature, you can compose boolean queries once and then get back to an accurately reduced subset of your thousands of notes in a split-second.

Find SearchExpressionParser on GitHub and feel free to open issues, pull requests, and ask questions anytime!

A little side-project of mine is a role-playing game written in Ruby that runs in the terminal and uses Unicode/ASCII characters instead of bitmap pixel graphics. In my personal tradition of these kinds of side projects, this is called TermQuickRPG. It’s a work-in progress, so there’s not a lot to do in the sample game at the moment.

How I found out why special characters wouldn’t draw under Linux

After I finished a little scenario with some custom scripting on the maps, I wanted to share the game with friends. Some have Linux machines running, and since I use the curses gem, I thought I was good to go. But no such luck: on macOS, it behaves totally different. On Linux, the Unicode Box Drawing characters cannot be printed. I get garbage output instead.

So the unicode characters don’t get printed. Bummer. I am using Linux Mint 19 (a Ubuntu derivate) in a VM to test this, by the way. Here are my initial test results:

• The font was capable of showing these characters.
• The Terminal was capable of showing these characters.
• Python 3 was capable of displaying these characters in a curses window.

The last one made me curious. The Python 3 docs say:

Note: Since version 5.4, the ncurses library decides how to interpret non-ASCII data using the nl_langinfo function. That means that you have to call locale.setlocale() in the application and encode Unicode strings using one of the system’s available encodings. This example uses the system’s default encoding:

import locale locale.setlocale(locale.LC_ALL, ”) code = locale.getpreferredencoding()

Then use code as the encoding for str.encode() calls.

And sure enough! When I call setlocale(locale.LC_ALL, ''), the Python sample did display the box drawing characters; without, it didn’t. There was no such setting in Ruby, though, and it seems no tinkering with the LC_ALL environment variable and file encodings did help.

That’s when I tried a quick sample in plain C.

• Plain C was not capable of displaying these characters.

Wait, what? Python is, C isn’t? (Not even with setlocale called from C.)

So I dug into the Python code. The Python implementation of addstr, the curses function that will eventually print a string on screen, reveals that for some environments, mvwaddwstr is used. That’s part of ncursesw.

Once I installed ncursesw sudo apt install libncursesw5-dev and compiled the C code with the -lncursesw option and called mvwprintw (note the trailing “w”, which makes this part of ncursesw, not ncurses!) – sure enough, it did output the characters just fine.

Curses’s internal representation of the string contents I was giving it did work with the ncursesw library, not with the curses or ncurses library.

There’s a ncursesw ruby gem, too, and it does work just as fine once you change the code to use that gem’s API.

Well, the ruby/curses gem says it in the README, too, once I looked a second time:

Requires ncurses or ncursesw (with wide character support).

Wide character support is what I was looking for all the time. I just didn’t pay attention to this stuff after I settled for the ruby/curses gem because its API was so nice. Sheesh!

Adjusting the game to ncursesw

The ruby/curses gem supports ncursesw, actually. It just loads the older stuff first, if possible. It comes 3rd since 2016. Switching the order of the #if defined compile-time macros to load ncursesw first, instead, instantly made the nice ruby/curses gem’s API do the job just as well as the ncursesw gem I mentioned above. No need to adjust even a single line of code!

Naturally, I created a pull request to incorporate the changes after local testing.

It even works on Linux! Sort of. My linux terminal displays the box drawing characters as wide unicode characters (which was my problem in the first place), so each box drawing character takes up the same screen space as 2 regular characters.

Up next, I’ll have to figure out if I can enforce double character width on macOS (why doesn’t macOS have to use wide-character support at all?) and adjust everything to these new width constraints. Or the other way around, get Linux terminals to display more narrow characters instead.

Or maybe I’ll switch to either a graphics-based renderer or PDCurses, which can use SDL to draw characters, it seems. We’ll see about that.

I am upgrading the code base of the Word Counter to Swift 3. Yeah, you read that right. I didn’t touch the Swift code base for almost 2 years now. Horrible, I know – and I’m punished for deferring this so long in every module I try to convert and build.

One very interesting problem was runtime crashes in a submodule I build where URLs were nil all of a sudden. This code from 2015 (!!) used to work:

public struct LocalURL {
    public let URL: NSURL

    public init(URL: NSURL) {
        assert(URL.fileURL)

        if URL.isFileReferenceURL() {
            self.URL = URL
        } else {
            self.URL = URL.fileReferenceURL()!
        }
    }
    // ...
}

Yeah, some unicorns are dying from my code thanks to force unwrapping. This is not the only place I did that. I was a terrible Swift citizen in 2015, it turns out.

Apart from my low Swift !-standards, this did use to work. Now it doesn’t. Even for an existing file path, URL.fileReferenceURL() just returns the same URL as you put in; this is clearly not what I was aiming for, since fileReferenceURL() is supposed to convert existing file URLs to a path-independent pointer to a file, aka a “file reference”. These look something like file:///.file/id=6571367.437879/ instead of file:///tmp/test.txt.

After casting from URL to NSURL and back for a while, I discovered Swift Bug SR-2728: apparently this is a known problem since Swift 3. Seems to be related to the bridging between NSURL, which is an NSObject subclass, and URL, which is a Swift struct.

An annoyingly verbose workaround by Charles Srstka for Swift 3.1 is to perform all the work in the Objective-C runtime:

if let refURL = (url as NSURL).perform(#selector(NSURL.fileReferenceURL))?.takeUnretainedValue() as? NSURL {
    print(refURL) // will print something along the lines of 'file:///.file/id=01234546.789012345'
}

That does indeed work! So if you have to work on a Swift 3.1 codebase and encounter this, there you go.

Swift 4.1 has a simpler mechanism to ensure you get a NSURL instead of a URL – the only type that supports file reference URLs as of September 2018, still.

if let fileRefURL = (url as NSURL).fileReferenceURL() as NSURL? { 
    print(fileRefURL)
}

Try that in the Swift 4.1 REPL to see that it works. Whew.

I do understand that there may be reasons to remove fileReferenceURL from URL and leave it on NSURL, but when you do invoke it on NSURL, I think it should at least return another NSURL object that works as expected instead of bridging to Swift’s URL struct that, for some reason, won’t work.

Interestingly enough, if you do know the file reference, e.g. file:///.file/id=6571367.437879/, you can work with Swift’s plain URL just fine:

let url = URL(string: "file:///.file/id=6571367.437879/")
print(url?.path)
// Output: 
//   /private/etc/tmp/test.txt

So when Swift’s Foundation URL type supports getting a path from a file reference URL, why does the fileReferenceURL() stuff not work?

Beats me!

If you happen to know something more, I’d be happy to know about the secret in the comments.