Eleven years ago, I wrote Collaborative Editing in ProseMirror , my first stab at doing collaboration in a rich text editor. Then six years ago I wrote Collaborative Editing in CodeMirror , describing a more elegant approach for a text editor. Now, after releasing Wordgard last week, I've spent a lot of time thinking about how to apply those techniques to a rich text editor. This post is a report…
I am happy to announce that my latest project, which I've been talking about for years, is now out with a first release. The project is called Wordgard . It is a new iteration of a ProseMirror -style rich text editor system, integrating the things I've learned since stabilizing ProseMirror, nine years ago. The architecture also takes a lot of inspiration from the version 6 redesign of the…
Type systems have a tendency of sneaking up on you. You start just trying to enforce some obvious invariants like “I shouldn't be able to assign a string value to a Boolean-typed variable”, and before you know what's happened you're reasoning about subtyping relations and type parameters. One thing that I keep running into, but for a long time refused to properly get to the bottom of, is that…
Every text editor system, whether it works with plain or rich text, needs some way to refer to specific positions in the document. The very first reason one runs into this, when implementing such a system, is for representing the cursor and selection. But in a mature editor, you'll be doing a lot more things that refer to document positions—representing changes, displaying user interface elements…
The concept of an object, record, structure, or product type occurs in almost every programming language. Being able to declare a type of value that groups a number of other values together is a fundamentally useful thing. As similar as the various record-ish constructs are in their basic functionality, the way such values are actually looked at and handled can be rather sharply divided in two…
CodeMirror 6 is a new code editor library for the web, a from-scratch implementation based on the experience of building and maintaining versions 1 to 5 for the past 13 years. It aims to be more extensible and accessible than previous versions. As of today, version 6.0 is stable. Going forward, probably at least several years, all new releases will be under the 6 major version, and be backwards…
An extensible system, at its base, is a system that allows people to add additional functionality that was not anticipated by the core system. A good extensible system also makes sure multiple extensions that don't know anything about each other can be combined, and compose in ways that don't cause problems. The problem has several aspects. Composition : Multiple extensions attaching to a given…
I'm happy to announce that with version 0.8.0 the CodeMirror 6 project is entering its beta phase, and you're very much invited to start trying it out and poking at it. Roughly, this “beta” status means: I actually like the current programming interface. Apart from the move from a mono-package to a group of separate packages (removing the next/ in the package names), no significant breaking…
This post describes the considerations that came up in designing the document-change data structure and built-in collaborative editing feature in the upcoming version of CodeMirror (a code editor system). It is something of a followup to the Collaborative Editing in ProseMirror post. I won't introduce anything new or exciting here—the design I ended up with is a very boring non-distributed…
Development of CodeMirror 6 this year has been generously supported by Mozilla through their MOSS program. The MOSS program asks for a closing retrospective blog post describing the progress made during the grant period. I've written about the progress in the first 9 months of the year in my status update post from August. To summarize, in that period we: Implemented solid text composition…
I keep coming across people who consider parser technology a forbidding, scary field of programming. This is nonsense—a small parser can be very, very simple , and provide a wholesome exercise in recursive thinking. At the same time, it is true that you can make parsing extremely complicated. Mostly, this tends to happen when you generalize parsing techniques to work for different grammars. Since…
It has been almost a year since we officially announced the CodeMirror 6 project, which aims to rewrite CodeMirror (a code editor for in the browser) to align its design with the technological realities and fashions of the late 2010s. This post is for you if, in the course of that year, you've occasionally wondered what the hell Marijn is doing and if he's getting anywhere at all. I have been…
Most code editors have some concept of automatic indentation. For each language, they have some logic that can compute an appropriate indentation for a given line. They may immediately indent when you press Enter, or require some interaction to reindent a line or selection. Even with the cleverest tools, there isn't always a single definite indentation for a given line. In a language like Python,…
(The details at the bottom of this post no longer reflect the way things work in CodeMirror 6. See this post for an updated description.) It has become fashionable to structure big systems as a number of separate packages. The driving idea is that it is better to, instead of locking people into your implementation of a feature, provide the feature as a separate package that they can load alongside…
Two years ago, I started the ProseMirror project because I wanted to take a stab at a better approach to WYSIWYG-style editing. Today, I'm releasing version 1.0 of the library . The architecture and scope of the project have changed quite a bit during its lifetime, but I feel that the original goal has been met. ProseMirror is a Web interface component, and though some of the challenges it tackles…
I am looking forward to the time where my node.js and browser can natively run ECMAScript 6 code. Build steps are a pain, and one of the reasons I like JavaScript in the first place is the (traditional) absence of “waiting for the compiler” from my work-flow. You can question whether ECMAScript 6 is worth incurring the inconvenience of a build step. I think it is. The various syntactic niceties…
I spent one week being a resident at the Recurse Center in New York this November. The Recurse Center is a somewhat unusual institution. They call themselves a “programmers retreat”. That more or less means they are a space where people who want to improve their programming skills (whether beginners or experienced programmers) hang around for a few months, doing self-directed learning,…
This post describes the algorithm used to make collaborative editing work in ProseMirror . For an introduction to ProseMirror, see another post here. The Problem A real-time collaborative editing system is one where multiple people may work on a document at the same time. The system ensures that the documents stay synchronized—changes made by individual users are sent to other users, and show up…
Sometimes I lie awake at night, feverishly searching for new ways to load myself down with more poorly-paying responsibilities. And then it comes to me: I should start another open-source project! Well, that's not really what happens. But the effect is the same. I keep building complex, demanding pieces of code and then giving them away. The actual mechanism is usually that I think of some…
As a programmer I create artifacts that, unlike classical commodities such as loaves of bread or toothbrushes, can be copied at zero cost. Fitting copyable things into the capitalist market economy is something of an unsolved problem. The standard model of commerce is to sell our products by the unit. But here, everyone who gets their hands on a single unit can create an infinite number of units,…
This post is part of the series about the internals of CodeMirror . This time, we'll discuss the way CodeMirror schedules updates to the DOM. The problem is this: CodeMirror has an internal model of the document that is being edited, and is displaying a representation of this model on the screen, using the browser's DOM. These have to be kept synchronized, for obvious reasons. However, updating…
A text is not just a string of Unicode. It has structure, and a certain internal coherence. People might want to read it in different formats, and if I am going to hand-edit it, I want its source to be in an editing-friendly format. For the first edition of my book (Eloquent JavaScript), I wrote my own markup format style and a Haskell program to parse it. That was, of course, a big waste of time…
Four out of five stars. Would read again. ★★★★☆ When Angus Croll first told me that he was working on a book that solves simple programming exercises in the style of various famous literary authors, I couldn't help but suspect that such a book would be completely pointless, and he wouldn't sell ten copies. After reading the result of his efforts, I still firmly stand by the latter prediction, but…
This is a brief post to explain why my JavaScript code is full of == null comparison expressions, even though linter software tends to disapprove. JavaScript defines both undefined and null as separate values. The first comes up in many constructs in the core language—an uninitialized variable, a parameter that wasn't passed, a missing field in an object, or a call to a function that doesn't…
I spend a rather large fraction of my days inside Emacs, writing and editing JavaScript code. Of this time, a significant amount is spent doing things that follow patterns. Pattern which, with a little machine intelligence, could easily be automated. Years ago, before accidentally rolling into this JavaScript career, I mostly programmed Common Lisp. Emacs integration for Common Lisp is divine. It…
From the very start, CodeMirror was set up as a system with zero unused abstractions. This is a doctrine that I've come to esteem highly: write code that solves the current problem you have, and not a bunch of other, similar problems that you can imagine someone may have in the future. And such a system will have to grow, almost without exception, as new use cases come up. But I argue that a…
I am writing a tool that tries to enhance a JavaScript code editor by doing some halfway serious static analysis on code, as it is being written. To analyze code, you'll want to parse it, because—believe me—running casual regexps over programs in order to determine their structure is not a healthy direction to go in. But unfortunately , code that's in the process of being written will, most of the…
Postmodern is a Common Lisp library for communicating with a PostgreSQL database. Sabra Crolleton recently published a nice collection of examples for the library. That reminded me that it's been over a year since the last release. There have been several major improvements, including support for notifications and bulk copying , so a release was in order, if only to prevent the impression that the…
When you've lied about something, it tends to take more and more lies to cover up the discrepancies created by the first lie. A very similar thing happened when CodeMirror needed to fake its scrollbars. CodeMirror's viewport First, some background. Why would we want to fake a scrollbar to begin with? In order to remain responsive when huge documents are loaded in, CodeMirror does not render the…
One common feature request that CodeMirror version 1 was fundamentally unable to support (due to its reliance on contentEditable ), and which was thus built into version 2 from the start, is programmatically styling stretches of text. In version 2, you can call instance.markText(from, to, className) , and it'll style that stretch of text with the given CSS class. By version 2.16, it was actually…
Acorn is a JavaScript parser written in JavaScript. Another one. Just like: The original UglifyJS parser The new UglifyJS parser ZeParser The Narcissus project's parser Esprima Acorn produces a well-documented, widely used AST format . The same as the last two parsers in that list. Acorn is really fast. Just like the last one in the list: Esprima . Acorn is tiny. About half as big as Esprima, in…
A CodeMirror mode is a module that helps CodeMirror , a code editor, highlight and optionally smart-indent text in a specific programming language or other kind of structured text format. Code editors take widely different approaches to the way syntax highlighting styles are defined. An elegantly simple approach is outlined in Patrick Walton's Common JavaScript Syntax Highlighting Specification ,…
I've just marked the current state of the master branch as version 2.34. The main changes are: New mode: Common Lisp Fix right-click select-all on most browsers. Change the way highlighting happens: Saves memory and CPU cycles. compareStates is no longer needed. onHighlightComplete no longer works. Integrate mode (Markdown, XQuery, CSS, sTex) tests in the central testsuite. Add a…
This is a post in the cm-internals series, describing the internals of the CodeMirror editor. The problem it tackles is this: you are writing a JavaScript control that needs to act as a text input field—it must be focusable, support copy and paste, receive typed input—but really isn't. I.e. you want to draw it yourself, and have full control over its content. In this post, I won't talk about…
Note: the question asked in this post, "why aren't closures super-fast?", was thorougly answered by Vyacheslav Egorov in his followup . Reading that is probably more informative than reading the text below. I originally structured CodeMirror instances as one huge closure that contained all the internal variables. The constructor would create local variables for all internal state, and local…
This post is part of an ongoing series of articles that aim to document the internals of the CodeMirror code editor. I will use the cm-internals tag to distinguish these posts—if you intend to hack on CodeMirror, it might be worthwhile to see what else is there. The problem The initial implementation of CodeMirror 2 represented the document as a flat array of line objects. This worked quite…
Last week, the need for a platform to publish my bi-directional text story on forced me to think about blogging software once more. I had heard that all the cool kids are now using Jekyll on their Github pages to publish their blogs. I am not keen on depending on Github for yet another aspect of my online life, but the idea of generating a static site from a git repository does sound appealing.…
"Unicode is hard" is a commonplace among developers. And I guess it is hard. Witness the amount of systems that get things like string en- and decoding wrong. And that is the easy part—the real fun starts when you need to actually display those strings. Fortunately, toolkits and libraries are able to hide the horrors of combining characters, directionality, and word breaking most of the time.…
Read this post at http://marijnhaverbeke.nl/turtle/ .
I've just put my Common Lisp Tcl/Tk bindings online. They differ from the existing LTK library in that they... Support for both FFI bindings and talking to a wish shell. Have hardly any 'wrapper' functionality — you're directly driving a Tcl interpreter from Lisp. They've only been used in one medium-sized project so far, but they are so simple that I'm rather confident they work as intended.…