Modern JavaScript
My presentation the jQuery Divide (video here) has been making the rounds on the internet again, six months after I delivered it at JSConf.eu in Berlin, and this time around, a colleague on IRC shared a link with me that drew from it: Is JavaScript the New Perl?
Perl has a special place in my heart; it’s the first language I used to solve a real-world problem, and I still have the second edition Learning Perl that my good friend Marcus got for me at the time. These days I struggle for it not to look mostly like a lot of gibberish, but in the late 1990s it was funtimes.
Anyway. The post that linked to my presentation asked if JavaScript might be going through some of the same pains that Perl has gone through, and linked to an eerily relevant presentation about Modern Perl, a movement “actively seeks to both to teach how to write good code and change perceptions of Perl that still linger from the dot.com 90s.” It talks about the void that Perl sought to fill way back in 1987, and then steps through the highs and lows of the intervening 23 years.
One thing that struck me, reading the slides, is that Perl — like other open-source, server-side languages — has the distinct benefit of being community-driven. While, yes, JavaScript has a wonderful and vibrant community, the language itself is held hostage by browser vendors, some of whom have shown a strong inclination to not give a fuck about owning up to and fixing their egregious mistakes. Using new features of a language like Perl is, at the end of the day, a largely internal problem — given enough time and money, switching to a new version of the language that offers new features for code organization, testing, and abstraction is a thing a project can do. Indeed, Perl as a community can even make bold decisions like deciding that a new version simply won’t be back-compat with a version that came before, throwing away ideas that turned out to be duds; meanwhile, JavaScript web developers often must bend over backwards to ensure back-compat with decade-old technology, and the only way to transition away from that technology is to give up on a set of users entirely.
We’ve already seen what this means for JavaScript as a language: it was years after JavaScript’s debut before we really started seeing conversations about what a module should look like in JavaScript, and we’re still fighting over it today. Without a solid dependency management system — something you can take for granted in any 15-year-old community-driven language — dependency management often means sticking another script tag on the page, and even the most popular JavaScript library on the planet struggles with how to participate in a fledgling ecosystem. With no arbiter of common, tested, community-approved, community-vetted solutions — see Perl’s CPAN — it’s an environment that’s ripe for fragmentation, and shining examples of Not Invented Here (NIH) litter the JavaScript landscape. Lacking even an agreed-upon method of expressing dependencies, the findability of good solutions is low, and coalescence only occurs around tools with extremely low barriers to entry and extremely high near-term reward.
When Marcus was teaching me Perl, back in the dot com heyday of the late 1990s and before the world temporarily went to hell for a few years, there was great emphasis on TIMTOWTDI: there is more than one way to do it. That mantra made Perl beautiful and elegant and powerful. Too often, it also made it ridiculously hard for the next developer to build upon and maintain, especially as the problems developers were solving got more complicated than copying and pasting some code to support a contact form (sound familiar?). In the end, that mantra meant Perl’s reputation suffered, as the consequences of code written by developers with a whole lot of freedom and not so much skill became clear.
This, in a nutshell, is what I was talking about in Berlin: that the reputation of this language we love stands to suffer if we don’t get around to working together to solve these larger problems, and educating the wider world of JavaScript developers as we figure it out. Unlike with Perl, the language itself isn’t going to evolve in time to help us here — unless and until we’re willing to give up on huge swaths of users, we will, generously, be stuck with the browser technology of 2009 for a long time to come. Unlike the Modern Perl movement, the patterns and tools and practices that will form the foundation of Modern JavaScript are going to have to come from outside implementations of the language itself.
Realizing that, it becomes clear just how imperative it is that we, as a community, figure out dependency management, modularization, and intentional interoperability so that these patterns, tools, and practices can start to emerge organically. James Burke, the creator of RequireJS, is something of a hero to me, not for creating RequireJS, but for taking on the challenge of interacting calmly and level-headedly with all sorts of stakeholders to try to make AMD modules a viable reality. Tool and library developers need to stop debating whether this is a good idea and get to work on making it happen.
Tools and libraries also need to take seriously the need for modularization — though I confess I have many misgivings about the NIH aspect of Dustin Diaz’s Ender.js, and wish that the considerable effort involved had been directed toward an established project with similar features, I can’t help but hope it will pressure libraries like jQuery to make more efforts in the direction of modularization.
An equally important aspect of modularization is ensuring minimal duplication of effort. As a community, we need to agree on a core set of functionality that ought to be provided by the language but isn’t, and implement that itself as an interchangeable module. A page with both Underscore.js and jQuery on it has tremendous duplication of functionality, for example. Interchangeability will allow end users to roll exactly the tool they need, no more and no less. Eventually, standard toolkits could emerge that draw on the best of all worlds, rather than one-size-fits-all tools that exist in isolation.
While I agree with what Tom Dale wrote in his oddly controversial post — that “unless it is designed to work well together, it usually won’t” — the more I think about it, the more I realize that the problem lies in our current inability to reliably isolate functionality and express dependencies across tools. It’s not that large tools like Dojo are the One True Way — it’s that large tools like Dojo are incredibly powerful precisely because they take seriously the need for a lightweight core leveraged by components that deliver specific, isolated functionality. JavaScript as a whole will become more powerful by embracing the pattern.
The political problems here are obvious and several: such modularization will, by definition, lead to winners and losers; the identities of libraries as we know them stand to be diluted if it becomes trivial to use only parts of them. The emphasis will shift to curated toolkits that assemble best-of-breed solutions, and NIH efforts will compete on merit, not marketing. At the same time, though, trying new things will no longer involve learning a whole new set of tools, and developers won’t be as stuck with a solution that made sense once upon a time but not anymore.
A final and important piece of the puzzle is actually educating people about the patterns that are enabled when we embrace these tools and practices. The wider community of everyday devs who are just trying to get their job done has hopefully graduated from copying and pasting scripts, but there’s a long path ahead, and part of the work of Modern JavaScript needs to be clearing that path for them.
I said it in my Berlin talk, and I will say it again: sharing what we know is as important as making new things, even if it’s not always quite as fun. All the script loaders, build tools, inheritance systems, array utilities, templating frameworks, and data abstractions in the world are meaningless if we don’t help people understand how and why to use them.
27 comments
I think the huge push right now HAS to be from the community especially if we want to move on from the dredges of ECMA 3-5, and evolve into the rich functionality that is supposed to be in the next version. Learning best practices and why libraries are written the way they are is just too difficult. I believe that's because a lot of people with little programming experience are expecting to understand CS and CIS theories, which isn't realistic. The amount of theory behind any major framework rooted heavily in abstract mathematical and computer science ideologies is staggering.
tl;dr It's putting people who don't have a heavy background in writing software into complex data-structures and algorithms, which is too daunting so they just revert to writing bad code snippets without regard to the best practices of the day because it's too steep of a learning curve
watch 17:38 - 28:00
http://www.youtube.com/watch?v=hs6tF-RDX4U
Crocford has a quote, I googled for it but couldn't find it quickly, it's like no new design flaws in 10 years, or something like that. That seems like something that is hard for other languages to attain.
You don't mention a timeframe in which the current landscape will change. Thoughts?
Dojo has a good start, but imho -- and this is something we seem to disagree on -- dojo needs to lose it's inter-depedencies. If all of the awesome modules, components, and widgets in dojo all depend on *several dozen* common modules (which they do), then do we really have modularization?
Regarding $ and _:
If you believe a redundant forEach() or map() implementation here and there is too many wasted bytes, then I can only envision API standardization as a way to allow us to truly achieve modularized javascript libraries down to this level. Abstraction only adds bytes, of course. Is there another way?
"A final and important piece of the puzzle is actually educating people about the patterns that are enabled when we embrace these tools and practices."
Abso-effin-lutely right. :)
Keep up the awesome posts.
-- John
However AMD is based entirely on limitations of older browsers. Given a build step for concatenating multiple scripts and minification (which everyone agrees is necessary for deployment), all advantages of AMD vs the existing CommonJS modules are eliminated. The repetitive boilerplate code AMD requires be hand written can easily be generated at deployment by the tool.
Normal CommonJS modules are already widely adopted among the most popular server side JavaScript platforms (Node, Ringo, etc). Part of the promise of SSJS is to share code between your client and server. AMD takes us a step away from goal.
In summary, AMD creates a divide between server-side and client-side JavaScript, and will tie us to legacy problems with the browser for many years.
Thank you for being so sound and so right.
If its a matter of round trips to the server, you could always easily do a dojo build that compresses everything you need for the modules you're using into one file.
in the js world, i see node.js eventually extending the language to fill obvious and significant gaps necessary for viable server-side utility. i see jquery and yui moving even further on their already well-established paths to becoming de-facto higher-level languages that simply utilize browser js as a execution target. and what of ecmascript itself? it isn't standing still, but one wonders who will care when everyone is writing to an abstraction layer above native javascript.
i see the path forward going through something like llvm and/or nacl. a lower-level compilation target will allow more interesting optimizations, retargetting options, and the potential for higher performance. a low-level compilation target will also allow the world to move on from a one-language mentality, which i don't think can persist for long...developers love to scratch that itch and try new toys.
There is one de facto standard javascript environment that *does* have synchronous module management. Node's require function and module.exports object are easy to understand and use - they take care of module and dependency management with great simplicity. With npm it also satisfies package management with the same simplicity. That's server-side - what about in the browser?
For *all* web apps you'll want at least a bootstrap piece of JS inline on the page (even if all that code does is load other code asynchronously). I'd argue that for most web apps you'll actually want to inline a fair bit more - for example, you'll want to set up a global delegated event handler that catches user input while the rest of the code loads (or else your app seems broken for the first couple of seconds). There definitely *is* a need for synchronous module management on the client, in addition to async.
The ideal situation would give you a development environment which supports loading dependent modules synchronously (in addition to async). For production, you'll then want to compile all synchronously dependent modules into a single minified snippet. If the compiler is aware of require and module.exports statements then it should be able to do this.
https://github.com/marcuswestin/require does the job. You get the synchronous require statement in the browser for development, and a compiler that turns all your code into a single minified piece of JS with dependencies included in the proper order.
If you have the time or feel strongly about getting the basic JS toolchain right from the get go, please consider this and give https://github.com/marcuswestin/require a whirl (it should take you no longer than 5 minutes to try, or else I did something wrong)
