Which web development tools are commodities?

We’re really bad at thinking about innovation.

To improve my own sense, I’ve been gradually absorbing Simon Wardley’s Value Chain Mapping since first seeing him talk about it a few years ago.

The picture to the right is an example of one of his maps. Each blob is a technology need.

As you go from left to right in the map, the technologies go from custom built, through product to commodities. (You can read more in an introduction by Simon, also see my post about the product/market space).

Anyway… The purpose of this blog post is to assess the state of play of various technologies a developer needs to make a new web application. Do we still have to make it ourselves, or is it a standardised thing?

I’m writing this while trying to build a web application (called MPCV in this post, to collect the CVs of candidates for Parliament). Since I’m doing it essentially for fun, I’ve got a very low tolerance of extra effort, so I’ve been pushing things as far right as I can.

Here goes:

Compute, Power, Web Server: These, and indeed Laptop and the whole of the rest of industrial civilisation, are at the Commodity stage. Or at least way over to the right hand side of Product.

Web hosting: You can just throw PHP scripts into a directory on a shared hosting account, or register a Flask app with a PaaS like Heroku or Google App Engine. This doesn’t feel like a commodity yet - there aren’t standard methods, the offering isn’t as clearly defined as, say, electricity. So kind of mid to right hand side of Product.

Source code: How programmers keep the product of their labour has a long history, but certainly feels like a commodity now. Pretty well everyone uses git for everything, with Github, Atlassian and Microsoft offering very similar hosting services. Left side of Commodity. The linked issue trackers don’t have strong standards yet and are hard to migrate between, so they’re over to the right of Product.

Web clients: With KHTML’s descendents in nearly every browser, IE6 pretty well actually gone, and web browsers in over a billion end user pockets, this is looking pretty good. Add to that the very mature low level libraries like jQuery and Backbone, and it is a place of dreams. Even video is nailed (I use the

Mobile apps: A warring duopoly, each with an identical feature set but you have to write your app twice. In different languages. I put the development tools for this in the Product side. Hopefully Firefox OS and/or the W3C will somehow force it into Commodity soon. For this reason I’m not worrying about this for MPCV, mobile web only.

Design templates: I’m using Bootstrap because we use it at ScraperWiki, but it has lots of competitors snapping at its heals. This is well over to the right of Product, getting on into Commodity soon.

Email sending: Firmly in Product. SendGrid and Mailgun are popular and work well. But you have to think about it, it is not like water. In some ways it is worse - on old Unix servers back in the low spam days, it was more of a Commodity.

User identity: The likes of Facebook and Google try to grab terrain here, both developers and their users are wary. There are a few products like Stormpath, none that great yet. Mozilla Persona is a tantalisingly close abandoned attempt - it at least met developers’ need to keep responsibility for their own users. In short, this area is still Custom built. Because that’s what everyone does, 2015 and still rolling email confirmations.

Developer identity: Every single one of the other products in this list requires you make a new user account, or these days one for each person working on the project, and enable and backup your 2FA codes. An unexpected frustration for me working on MPCV is that it is a throwaway project - it doesn’t have organisational boundaries yet. Sometimes I literally got blocked not knowing what to name an account. And I had severe limits on how often I wanted to add a credit card subscription for a short project. Heroku’s app store made me even more confused about this, and nobody seems to use “log in with Github”. This is at the Custom built phase.

Encryption: All good sites need to be over HTTPS these days. Getting the certificates is a rip-off Product. The Electronic Frontier Foundation are improving this with free certificates later this year.

Democracy data: For my purposes, mySociety’s Mapit (for postcode to Parliamentary constituency lookup), and DemocracyClub’s YourNextMP (to get the candidates in a constituency) are fantastic. It felt like Commodity territory.

I’m going to update this post as I come across more categories during development.

Conclusion

There are big big problems with identity. Both my identity and the identity of my users are taking up far too much of my time and attention. This feels like a core weakness - I hope Mozilla try again. I would attack here.

There are too many services scattered everywhere. I don’t think either Amazon or Heroku are doing a good job at bringing them together. They, or Google or Microsoft, will eventually.

Mobile apps are an embarrassing disaster. I hold little hope that there is an attack point against this, but who knows.