Earlier this month, I wrote about validating temporary models for forms. The validation returned .complete or .incomplete, which doesn’t help much when you want to show what did go wrong.

So I came up with a richer validation syntax.

You can see the code as a whole without my comments as a Gist.

Example Model, its Partial, and Validation

If this is our model:

struct User {
    let firstName: String
    let lastName: String
    let age: Int?
}

extension User: PartialInitializable {
    init(from partial: Partial<User>) throws {
        self.firstName = try partial.value(for: \.firstName)
        self.lastName = try partial.value(for: \.lastName)
        self.age = partial.value(for: \.age)
    }
}

Then I want validations to look like this:

// Specify a non-empty list of validation requirements
let validation = Partial<User>.Validation(
    .required(\User.firstName),
    .valueValidation(keyPath: \User.firstName, { !$0.isEmpty })
    .required(\User.lastName),
    .valueValidation(keyPath: \User.lastName, { $0.count > 5 }),
    .valueValidation(keyPath: \User.age, { $0 >= 18 })

Given this validation and a “partial” which contains data provided by the user, executing and evaluating the validation will look like this:

var partial = Partial<User>()
partial.update(\.firstName, to: "foo")
partial.update(\.lastName, to: "bar")
partial.update(\.age, to: 12) // <- too young!

switch validation.validate(partial) {
case .valid(let user): 
    print("Is valid: \(user)")

case .invalid(let reasons):
    for reason in reasons {
        switch reason {
        case .missing(let keyPath):
            if keyPath == \User.firstName { print("Missing first name") }
            if keyPath == \User.lastName  { print("Missing last name") }
        case .invalidValue(let keyPath):
            if keyPath == \User.firstName { print("Invalid first name value") }
            if keyPath == \User.lastName  { print("Invalid last name value") }
            if keyPath == \User.age       { print("User is too young") }
        }
    }
}

Thanks to nested generic types inside generic types, this is pretty easy to accomplish!

The Code, Explained in Much Detail

protocol PartialInitializable {
    init(from partial: Partial<Self>) throws
}

This is new: it is a type restriction so that successful Partial<T>.Validation can attempt to create an instance right away.

Except for the T: PartialInitializable constraint, this is pretty much Ian’s original Partial<T>:

struct Partial<T> where T: PartialInitializable {
    enum Error: Swift.Error {
        case valueNotFound
    }

    private var data: [PartialKeyPath<T>: Any] = [:]

    mutating func update<U>(_ keyPath: KeyPath<T, U>, to newValue: U?) {
        data[keyPath] = newValue
    }

    func value<U>(for keyPath: KeyPath<T, U>) throws -> U {
        guard let value = data[keyPath] as? U else { throw Error.valueNotFound }
        return value
    }

    func value<U>(for keyPath: KeyPath<T, U?>) -> U? {
        return data[keyPath] as? U
    }
}

Here comes the validation extension:

extension Partial {
    struct Validation {
        enum Strategy {
            case required(PartialKeyPath<T>)
            case value(AnyValueValidation)

            static func valueValidation<V>(keyPath: KeyPath<T, V>, _ block: @escaping (V) -> Bool) -> Strategy {
                let validation = ValueValidation(keyPath: keyPath, block)
                return .value(AnyValueValidation(validation))
            }

            struct AnyValueValidation {
                let keyPath: PartialKeyPath<T>
                private let _isValid: (Any) -> Bool

                init<V>(_ base: ValueValidation<V>) {
                    keyPath = base.keyPath
                    _isValid = {
                        guard let value = $0 as? V else { return false }
                        return base.isValid(value)
                    }
                }

                func isValid(partial: Partial<T>) -> Bool {
                    guard let value = partial.data[keyPath] else { return false }
                    return _isValid(value)
                }
            }

            struct ValueValidation<V> {
                let keyPath: KeyPath<T, V>
                let isValid: (V) -> Bool

                init(keyPath: KeyPath<T, V>, _ isValid: @escaping (V) -> Bool) {
                    self.keyPath = keyPath
                    self.isValid = isValid
                }
            }

        }

Partial<T>.Validation.Strategy offers two modes to validate key paths at the moment:

1. required, which just checks for mere presence of any value, and
2. value, which performs a custom check via a closure. Use this to validate that a string is non-empty, for example.

The ValueValidation<V> type accepts a key path of type KeyPath<T,V>. Part of the key path’s generic constraints are satisfied by Partial<T>, so the subject is given; the value you point to specifies which type ValueValidation will work on.

Example: The key path \User.name has the type KeyPath<User, String>. If you pass this in, you’ll get a Partial<User>.Validation.ValueValidation<String>. The second parameter in its initializer is the actual boolean validity check. Thanks to the power of generics and nested types within generic types, we end up with a very specialized ValueValidation. The compiler will help us with these in place: we cannot accidentally treat string values as numbers, unlike a dictionary of type [String : Any], where the cast from Any may fail for all the wrong reasons.

While creating ValueValidation objects with a key path and a fitting closure is then very straight-forward, you cannot lump different value type validations together. [ValueValidation<Int>(...), ValueValidation<String>(...)] will be an Array<Any> since the specified types have nothing in common although you and I know they’re supposed to do a similar thing. The same-ness has to be expressed in code, though. In other words, we have to erase the generic information and thus make all generic specializations the same. There’s no communism(_:) function in the Swift standard library that does this for us. We need to provide another type on our own and type-erased the generic V from the ValueValidation<V> in order to do that. I went with the AnyValueValidation approach that hides the generic constraint from the outside and wraps the isValid check accordingly.

That’s why the Strategy case is called value(AnyValueValidation) and not value(ValueValidation) – because the latter won’t compile.

Initializing the type-erased validation sucks big time, though: AnyValueValidation(ValueValidation(keyPath: \User.name, { !$0.isEmpty })).

That’s why I added a static factory called valueValidation. When you call it, it looks like an enum case, but really isn’t. You’ll see how we can call it as .valueValidation(keyPath: \User.firstName, { $0.count > 5 })) in a second.

Now that the Strategy type is declared, here’s how it’s used when computing a Partial<T>.Validation.Result:

        let validations: [Strategy]

        // Prevent creating an empty validation collection by making 1 parameter
        // required, and then add a variadic list afterwards.
        init(_ first: Strategy, _ rest: Strategy...) {
            var all = [first]
            all.append(contentsOf: rest)
            self.validations = all
        }

        enum Result {
            case valid(T)
            case invalid([Reason])
            enum Reason {
                case missing(PartialKeyPath<T>)
                case invalidValue(PartialKeyPath<T>)
            }
        }

        func validate(_ partial: Partial<T>) -> Result {
            var failureReasons: [Result.Reason] = []
            for validation in validations {
                switch validation {
                case .required(let keyPath):
                    if !partial.data.keys.contains(keyPath) {
                        failureReasons.append(.missing(keyPath))
                    }

                case .value(let valueValidation):
                    if !valueValidation.isValid(partial: partial) {
                        failureReasons.append(.invalidValue(valueValidation.keyPath))
                    }
                }
            }

            guard failureReasons.isEmpty else { return .invalid(failureReasons) }

            return .valid(try! T.init(from: partial))
        }
    }
}

The validate method exercises all instances of the validation strategy. The failure reasons match the strategy cases: you get missing for required, and invalidValue for value. Both pass the PartialKeyPath<T> along.

It’d be nice to have the fully qualified KeyPath<T, V> here, but again you cannot lump these together. PartialKeyPath<T> is a type-erased variant already – but using a different approach, namely being the parent class instead of boxing the specialized type in.

After validation, you get a Result; if nothing failed, you will get a fully initialized object. Hopefully. The initializer could throw an error for reasons not expressed in the validation constraints. That’d be bad. And I’d argue that the person responsible for throwing an error that’s not covered by the validation mechanism didn’t adhere to the contract of PartialInitializable.

Improvements

There’s room for improvement. For example, I think I’ll rename Strategy to Constraint and then provide the constraint’s user-facing message upon initialization. That way you can have multiple value validations for the same property like “age is above 18” and “age is below 30” for cheap student insurance rates in Germany with different explanations for the validation failure.

Also, I don’t like the try! a lot. I still think it’s a programmer error if a validation passes but object initialization isn’t possible, because that’s the whole point of all this. But there could be a better way.

Maybe there’s room for more Constraints? The value validation is very powerful already, but maybe it’s too generic and someone could use more specialized cases instead.

Again, if you want to have a look at the whole code, it’s available as a Gist on GitHub. Feedback is very welcome!

I wanted to handle a file loading error in a consumer of an RxSwift Observable sequence. But if the sequence itself produces an .error event, it completes. After fiddling around for a while, I decided to simply use the Result enum and ended up with Observable<Result<Theme, ThemeError>>.

But someone told me about another way. A way without Result. A way of pure RxSwift.

I was intrigued. And puzzled.

Adam Borek explains how you can use materialize() on Observable sequences to get access to the underlying event that you can use instead of the Result enum. You use the .next and .error event cases instead. Great!

So my sequence turned into Observable<Event<Theme>>. What next?

You can use RxSwiftExt’s convenience methods to the mix and split this sequence into two using .elements() and .errors(). That’s exactly what I needed: in most cases, I map errors to a fallback value – in this case, a fallback theme the app will use. And in one instance I consume the errors to display a warning with additional info.

There’s one caveat with materialize, though: the sequence will still complete after error. That means you will want to create a new sequence for every request, be it a network request or a file reading request like in my case, and then flatMap or flatMapLatest into it. That’s what Adam Borek does in his code, too. If you just mapped the original observable sequence and materialize that instead, it’ll complete on error.

The request produces a single value, like Observable.just, but there’s no API for a throwing factory closure. I came up with the following extension:

extension Observable {
    static func attemptJust(_ f: @escaping () throws -> E) -> Observable<E> {
        return Observable.create { observer -> Disposable in
            do {
                observer.on(.next(try f()))
                observer.on(.completed)
            } catch {
                observer.on(.error(error))
            }
            return Disposables.create()
        }
    }
}

The code I use to load my themes thus becomes:

let themeURL: Observable<URL?> = // ... the user loads a file ...
let themeFromURL: Observable<Event<Theme>> = themeURL
    .filterNil()
    .flatMapLatest { url in
        return Observable
            .attemptJust { try Theme(fromURL: url) }
            .materialize()
    }

The inner, materialized sequence can error-out and I can consume the error events in the main sequence. Works like a charm.

I don’t know if I like that Event has a .completed case, too. Result only knows about success and failure. It was closer to my intention. But ditching another dependency in this case is nice, too, and it taught me a thing about RxSwift, so I’ll keep it and see if I understand what’s going on a year from now.

Ian Keen posted an article about type-safe temporary models. You would use them like scratch pad contexts in Core Data: in forms, you collect information into these temporary models and then generate your real objects from them.

Update 2018-06-16: I wrote a post about a nicer validation sub-type.

What’s that good for? Imagine a form to edit a user’s name. If you pass in the real entity, which might be a reference type aka class, then the form would perform changes on the real object immediately. But what if you want to cancel? How do you get the original values back? That’s why you don’t use the real entity but a placeholder instead. That’s why I proposed a separate read model and a write model a couple of years ago.

Usually, I write these temporary models myself for ever form. Thanks to Swift, this is super easy to do, and you can tie the temporary models closely to the forms with primitive String and Int properties, much like good view models, but for output. (“Data Transfer Object” would be another classical term.) But I like how the Partial<T> Ian came up with works. It uses Swift key paths, so unlike a String-based dictionary, this is pretty type-safe already. And it is super flexible: you don’t have to add or remove properties in the temporary model when you change the target model.

Here’s part of the example from Ian so you get an impression:

struct User {
    let firstName: String
    let lastName: String
}

var partial = Partial<User>()
partial.update(\.firstName, to: "Ian")

Afterwards, the partial contains information on firstName, but not on lastName. You can create a failable initializer like this:

extension User {
    init(from partial: Partial<User>) throws {
        self.firstName = try partial.value(for: \.firstName)
        self.lastName = try partial.value(for: \.lastName)
    }
}

That would throw an error because there’s no value for the key path to lastName. Nice!

I figured you’d want to validate the form input before trying to initialize the model object. So I played around creating a validator.

let validator = Partial<User>.Validation(requiredKeyPaths: \User.firstName, \User.lastName)

switch validator.validate(partial)
case .valid: 
    print("Is valid!")
    do {
        let user = try User(from: partial)
    } catch {
        print("Totally unexpected problem")
    }

case .incomplete(let missingKeyPaths):
    if missingKeyPaths.contains(\.firstName) { print("Missing first name") }
    if missingKeyPaths.contains(\.lastName)  { print("Missing last name") }
}

Instead of printing, you would scroll to and highlight the form fields that have to be completed, of course.

The Validation type is a pretty simple extension:

extension Partial {
    struct Validation {
        let requiredKeyPaths: Set<PartialKeyPath<T>>

        init(requiredKeyPaths first: PartialKeyPath<T>, _ rest: PartialKeyPath<T>...) {
            var all = [first]
            all.append(contentsOf: rest)
            requiredKeyPaths = Set(all)
        }

        func validate(_ partial: Partial<T>) -> Result {
            let intersection = requiredKeyPaths.intersection(Set(partial.data.keys))
            if intersection == requiredKeyPaths { return .valid }
            let difference = requiredKeyPaths.subtracting(intersection)
            return .missing(Array(difference))
        }

        enum Result {
            case valid
            case incomplete([PartialKeyPath<T>])
        }
    }
}

And for posterity, here’s Ian’s Partial<T> code itself so you have the complete picture:

struct Partial<T> {
    enum Error: Swift.Error {
        case valueNotFound
    }

    private var data: [PartialKeyPath<T>: Any] = [:]

    mutating func update<U>(_ keyPath: KeyPath<T, U>, to newValue: U?) {
        data[keyPath] = newValue
    }

    func value<U>(for keyPath: KeyPath<T, U>) throws -> U {
        guard let value = data[keyPath] as? U else { throw Error.valueNotFound }
        return value
    }

    func value<U>(for keyPath: KeyPath<T, U?>) -> U? {
        return data[keyPath] as? U
    }
}

WWDC people noticed that Panic Inc. are coming back to the Mac App Store with their beloved file transfer app, Transmit. This puzzled a lot of people because they moved away from the MAS starting with Coda 2.5 in 2014. Sandboxing was just too restrictive. But now, it seems, the new Mac App Store’s Sandboxing rules will be different enough for Transmit to work. See Panic’s tweets on the topic. The details:

• The Mac App Store version will be a subscription; this way they can offer a free trial on the Mac App Store. When you need the app only for a short while, subscribing for a single month will be cheaper than buying a full license.
• Their web store will feature a classic purchase option where you buy the app (at a higher price than the subscription) once and get free updates.

The combo is an interesting approach to cater to 2 different needs, but I think it’s also not that easy to communicate. Pro users will understand what’s going on, but less tech-savvy people will probably never know about the non-App Store version.

So if you’re a fan and want to support Panic Inc. with your money, you can opt-in to using the subscription, even in addition to your non-App Store purchase. It’s like donating, only without all the tax ambiguities.

In my quest of total immersion into Emacs, I am trying to write blog posts in Emacs instead of TextMate. That means my TextMate macros are gone for good, including insertion of YAML header information. On this very blog and on Zettelkasten.de, I used to type cthead and zkhead respectively, then expanded the YAML header with tab. TextMate’s great template feature even allows you to specify multiple points inside the templates to jump to by tabbing forward. Emacs works a bit differently, so I adapted the templates to request information up front instead and fill in the values.

What I needed for the Zettelkasten blog was:

• title
• creation date (auto-generated)
• author (Sascha or I)
• tags
• preview image, if any

I found a way to specify template code in Emacs Org mode using Babel: Babel is used for literate programming, and I wanted to embed the template as a template. I could also write this in pure Emacs Lisp, but I don’t enjoy maintaining multi-line strings and favor heredocs whereever possible. So with Babel’s named BEGIN_EXPORT, I exported the template with the name ct/blog-header-template:

#+NAME: ct/blog-header-template
#+BEGIN_EXPORT txt
---
title: "%s"
created_at: %s
kind: article
author: %s
tags: [  ]
image:
---
#+END_EXPORT

The %s designates a printf-style placeholder. So the template would allow to be formatted with 3 strings. I didn’t add the image because I usually add, rename, and manage these much later in the process.

Now I can reference this named template block in other Babel blocks using source blocks’s header arguments. This time I want to have access to the template string as a variable, and Babel allows you to pass this in as context. This is much like any other template language, be it Ruby’s Erb, Handlebars.js or whatever, only it works on a per-block basis, not per-file.

#+BEGIN_SRC emacs-lisp :var template=ct/blog-header-template
(eval `(defun ct/blog-insert-header ()
         (interactive)
         (insert
           (format ,template 
             (read-string "Title: ") 
             (format-time-string "%Y-%m-%d %H:%M:%S +0100") 
             (ido-completing-read "Author: " '("christian" "sascha"))))))
#+END_SRC

I use eval to inline the template string into the function definition. I cannot define the function (defun) directly and reference the template string directly. The template is stored in the variable template according to the block definition (:var template=...). I am no expert in Emacs Lisp and Babel, but I think this means: the variable is not part of the Lisp function when it is executed. This BEGIN_SRC block is run once, and it behaves much like a template language itself, only with Lisp syntax. The template variable will be available when the source block “template” is executed; when you execute the function that’s produces by running the block, though, the variable isn’t visible there anymore. That’s why I use eval to literally put the YAML header inside the function body.

• format behaves like printf: it takes a template string and a variadic list of arguments to pass into the template.
• read-string prompts the user for a title in the minibuffer interactively
• format-time-string produces an ISO-formatted string that complies to YAML’s date serialization
• ido-completing-read is a variant of completing-read (which you could also use) to show a set of options, much like a dropdown or picker of author names

The template for this blog is similar but lacks the author picker, which I found most interesting.

I put the header template definition and the resulting function in the Org file that’s being executed during startup. It behaves like the usual .emacs.d/init.el file, but you can organize the file with Org outlines and embed Emacs Lisp code like the one shown above to be executed. This is called literate Emacs initialization. I prefer this over separating my initialization routines and macros into multiple init files because the Org outlines amaze me.

Some day I might add a tag picker, too, aggregating all tags I use on the blog from a local cache file or something. While I’d love that for the sake of technical prowess, it’s not important or even that useful.

Happy GDPR day!

With the recent EU law stuff taking effect, I reconsidered my use of tracking information from you. So I just removed Google Analytics tracking on this website, on our productivity blog at zettelkasten.de, and on all my app websites completely. I find the numbers interesting, but not actionable anyway. I don’t do A/B testing. I just write for fun and with the intent to help you do stuff. There’s no benefit in tracking how you use my site. Your visit is much appreciated, and that’s it!

I’m a bid sad about having to get on people’s nerves with regard to the newsletter. I removed the signup forms for now until I figure out how many opt-in checkboxes I really need to provide to collect consent for sending a newsletter. Meh.

I noticed that my Emacs didn’t maintain the current line’s indentation when editing code blocks in Markdown (markdown-mode). I indent the first line with 4 spaces like any sane person would. When I hit enter to continue in the next line, thanks to the markdown-mode defaults, I am presented with a new line that’s properly indented visually. Only when committing to git did I notice that Emacs inserted tabs instead of spaces. Gasp!

Turns out you’re supposed to turn this off for text-mode (a parent mode of Markdown). See the great Emacs indentation tutorial for a good reference.

Using hooks, here’s what I am now using to indent by 4 spaces instead of adhering to tab stops:

(add-hook 'text-mode-hook
          '(lambda ()
             (setq indent-tabs-mode nil)
             (setq tab-width 4)))

This will also affect the “press tab to indent the current line” behavior that’s active even when your in the middle of text. I needed to get used to this at first, because I expected hitting tab to insert a tabulator at the current cursor position. By default, this indent-line-function is configured to indent the current line up to the previous line’s indentation, at least for a lot of cases. In some modes, it is even more aware of the context and does clever things. You can change this to be dump and always indent by 4 spaces, too. Just add this to the hook’s lambda list of expressions:

             (setq indent-line-function (quote insert-tab))