(2021-10-30) Matuschak Doing-centric Explanatory Mediums: Board Game Instruction Manuals And An Unusual Figma Document

Andy Matuschak on Doing-centric explanatory mediums: board game instruction manuals and an unusual Figma document. ...another problem with books, and specifically with books meant to help you learn a skill: the medium makes it difficult to collapse the distance between prose and action. Books rarely involve doing what they’re about. (learning by doing)

If you ask people about the highest-growth periods of their life, you’ll notice that the most enabling environments tend to involve doing

but skill-building books have many practical advantages

So: how might we create a mass medium which possesses the book’s advantages, but which is situated in doing?

The role of the dynamic medium

One reason it’s hard to create books which are situated in doing: the book is static, fixed.

Authors can advise how to reflect on an exercise and generate your own feedback, but these are scripts you must execute in your own mind. Video doesn’t appreciably change this situation.

But the promise of computers, and the dynamic mediums they enable, is representations which behave and respond.

Where it’s possible, I’m much more excited about what happens when a computational environment becomes the authentic—not simulated, not “educational”—environment for doing. A non-linear video editing interface isn’t a “toy” way to edit a film; they’re how professional filmmakers actually edit films.

In the last decade, authors and programmers have created dozens of interactive articles which might inform our answer (see Communicating with Interactive Articles for a good overview).

They’re integrated with simulated petri dishes, not an actual lab bench.

If these interactive articles are often structured as wide seas of prose which contain islands of interactivity, one path to integrating authentic environments might be to invert this structure. How might we create “articles” which are primarily interactive environments, but with embedded islands of prose?

What about computational notebooks like Mathematica and Jupyter? I don’t think these qualify; I don’t know of any strong examples which are “doing-centric”, where the code listings interspersed throughout the text are really about doing the topic in question

Could we move the YouTube lesson on 3D modeling with Blender into Blender?

Video games excel at this. Sometimes tutorials appear in non-interactive cutscenes, sharply delimited from ordinary play, but better examples (e.g. Portal) present instruction and narrative as a seamless element of the interactive environment, never “stealing” the camera or the controls away from the player.

Video games also improve upon another problem of interactive articles: the challenge of separating prose and dynamic representation

Video games take advantage of audio to layer instruction onto what the player’s seeing, but even when using only text, they can position that text right next to the relevant part of the action

Oddly enough, the proximal cause of this letter was an extremely unusual Figma document.

Figma changed the way copy and paste works in their interface. They made a document to explain the change to their users.

The document initially reads like a hypertext slide deck. It shows visually how the new clipboard functions behave. It’s cute that it’s a document about using Figma, both created and consumed in Figma, but that’s not so special. Interleaved with explanatory preamble, the document turns control over to you:

This could have been a blog post, with little interactive “demo” areas interspersed between paragraphs. But instead, you’re interacting with this document in the full-blown Figma environment.

It’s important to recognize that there’s a lot of text in this document. That’s part of what makes it so interesting. “Worksheets” aren’t so unusual—there are lots of Figma documents like this which give you an exercise and some context to play in.

the copy/paste Figma document is unusual because it’s roughly a thousand words long. There’s a lot of explanatory material here, and there could be even more. The possibility of expansive, in-depth documents opens the door to canonical works in the medium

it’s not a stretch to imagine a much more significant variation: a “textbook” about interface design, written in Figma

Of course, the Figma “meta-document” medium can itself be pushed much further.

The fine-grained exchange between explanation and action makes it easier to help readers generate their own feedback, but the medium could go further by reacting to the reader’s actions. Earth, a Primer demonstrates one simple approach, “checking off” suggestions as readers complete them:

Another interesting direction—and one which fits well with Figma’s multiplayer primitives—would be to integrate opportunities for collaborative learning into the text. A standard “move” in collaborative learning is: introduce a problem which can be solved with several different approaches; juxtapose several contrasting students’ solutions and help students learn from each others’ ideas.

Extensions to other environments

Documents must offer some way for authors to communicate explanatory content, potentially at length. Figma has text elements; programming environments have comments. In other environments—like an audio production tool—we may have to add such affordances to the document model.

Authors must be able to establish relationships between passages of their content and corresponding elements in the reader’s interactive environment. In Figma, we can position text immediately above each element you’re meant to manipulate.

Authors need ways to linearize and structure their explanation; readers need affordances for “navigation.” In Figma, artboards are arranged in a sequence which can be easily navigated by scrolling or through keyboard shortcuts. Hyperlinks and a “table of contents” artboard support navigation.

A key implicit point here is that more "normal" people can learn to create Figma documents than write video games. And even if you know both, it's a lot easier/faster to do the former. Then again, maybe using a game engine for hypermedia rather than a pure/fun "game" is in-between. Playing with Godot?