Patrick Meenan's technical blog
Robin Marx has been asking me for a Compression Dictionary Transport ( RFC 9842 ) test tool for a long time (since well before AI was actually useful for this kind of work). Then Sergey Chernyshev brought it up again recently. So, I finally stopped procrastinating and spent a few hours this weekend working with AI to build out a proper test tool called compdict . It turns out that using AI makes…
I was getting really frustrated with Apple's search implementation for the "Messages" app (both on mobile and a MacBook) when trying to find specific messages from quite a while ago. So, I did what any dev does these days with all of the capabilities that AI brings: I built a tool for searching, viewing, collecting and printing messages from iOS backups (encrypted included). You can use it from…
Maybe? That's the question I'm hoping to answer over the next ... ??. This is a (hopefully) fun experiment that I'm playing with in my spare time to explore both how far I can get with AI tools and where the rough edges or limits are in Chrome's tech stack and APIs. I'm targeting Chrome specifically because I'm mostly interested in the core capabilities that the latest web platform APIs offer and…
Ever since browser vendors started exposing advanced performance telemetry APIs directly to the page, like the Performance Timeline , Long Animation Frames (LoAF) , and the JS Self-Profiling API , I've been waiting for tooling that could capture, store, and visualize all of this data in the wild. We can gather incredibly detailed information about what is happening on a user's machine, but…
I've been wanting to build a 100% client-based waterfall tool for a long time. Something with a much more modern rendering engine and UI than WebPageTest's server-side php-based image waterfalls. It's a fairly big project though and I never had the time to invest into it but that all changed with the advance of the AI-assisted development tools (I refuse to use the term "vibe coding" given the…
Should we ship jQuery, React and other popular frameworks with browsers so sites don't have to re-download the same frameworks over and over? Some background For years, web performance advocates have casually suggested that browsers should "just ship jQuery" or other popular frameworks to avoid the need for every site to force users to re-download identical library code (there's a recent WHATWG…
I've used AI to generate a few websites over the last month or so and not too long before that I was helping my son create a website on Wix so I thought it might be interesting to see how they stack up against each other in terms of performance. I was REALLY motivated when I saw the new Stitch app had a "create react app" button (ok, petrified probably describes my reaction better than motivated).…
The actual writing of code hasn't been the bulk of development for a while, particularly in larger companies with larger codebases and existing users. The more existing users there are, the more process there tends to be around protections and procedures. These exist to ensure you aren't breaking existing behaviors, risking security (or press), or causing regressions in some random metric added to…
Wow, it has been a minute! Or, more specifically, 12 YEARS! I've been wanting to move off of blogger for the longest time and had all these ideas for what I wanted to move to to make it easier to post content. Mostly I just wanted to be able to throw together a post in markdown and have some tooling turn that into a "good" static site (responsive, fast, not ugly, etc.). I toyed around with…
It has been 3 years since the last tour and a lot of people have been asking if it is still hosted in my basement so it's time for an update. First, yes it is still hosted out of my basement. I did move it out of the utility room and into a storage room so if the water heater leaks it will no longer take out everything. Yes, Halloween has gotten a bit out of control. This is what it looked like…
We've been working to bring better support for measuring web performance on mobile for a while. Michael Klepikov started out by building out a new cross-platform test agent for WebPagetest that runs on Node.js, can run WebDriver/Selenium scripts and can talk to the Dev Tools interface for Chrome. Todd Wright extended that support to talk to mobile Chrome on android and even Safari on iOS using a…
TL;DR: WebPagetest will now expose any User Timing marks that a page records so you can use the same custom events for your synthetic test measurement as well as your Real User Measurement (and you can use WebPagetest to validate your RUM measurement points). Before kicking off an optimization effort it is important to have good measurements in place. If you haven't already read Steve Souders'…
TL;DR: Progressive JPEGs are one of the easiest improvements you can make to the user experience and the penetration is a shockingly-low 7%. WebPagetest now warns you for any JPEGs that are not progressive and provides some tools to get a lot more visibility into the image bytes you are serving. I was a bit surprised when Ann Robson measured the penetration of progressive JPEGs at 7% in her 2012…
I have the pleasure of helping select the talks for a couple of the Velocity conferences this year and after looking at several hundred proposals it is clear that there are widely varying opinions from submitters on what would make for a good talk and also a lot of cases where the topics may be good but the submitter may have the wrong focus. I'm certainly not an expert on the topic but I think…
My favorite web performance article in 2012 was this one from Kyle Rush about the work done on the Obama campaign's site during the 2012 election cycle. They did do some cool things but it wasn't the technical achievements that got my attention, it was the effort that they put into it. They re-architected the platform to serve as efficiently as possible by serving directly from the edge and ran…
I've spent the last week or so getting the IE testing in WebPagetest up to snuff for IE 10. I didn't want to launch the testing until everything was complete because there were some issues that impacted the overall timings and I didn't want people to start drawing conclusions about browser comparisons until the data was actually accurate. The good news is that all of the kinks have been ironed out…
The FCC released their eighth broadband progress report yesterday. The most interesting part for me is when you get to page 45 and they start talking about actual adoption (in the US), as in the speeds that people are actually subscribing to, not what is available or offered. Their buckets aren't all that granular and the data they used to build the report comes from June 2011 but they give you a…
Hot off of the Velocity Conference presses, iOS and Android 2.3 agents are now available from the Dulles, VA location on WebPagetest . We have been working with Akamai on their recently open-source mobitest agents and are happy to announce that the agents are now up and running and are also available for people to use for Private Instances . Testing is limited to loading URL's and capturing video…
WebPagetest makes EC2 AMI's available for people to use for running private instances and makes fairly extensive use of them for running the testing for the Page Speed Service comparisons . We have tested that the m1.small instances produce consistent results but we aren't necessarily sure if they are representative of real end-user machines so I decided to do some testing and see. This is a very…
It has been an exciting 2 days. Yesterday I discovered that over the weekend the forums at webpagetest.org had been hacked and that someone had installed a back door. I traced the source of the entry pretty quickly and locked out the exploit he had used but I wanted to make sure he hadn't done anything more damaging while he was there so I spent the last day pouring over the access logs to trace…
Steve Souders had a great blog post last year that talked about Frontend Single Points Of Failure (SPOF). Given the continuing rise in 3rd-party widgets on pages it is becoming increasingly important and I realized that there weren't any good tools for testing for it. Seemed like the perfect opportunity to piece something together so that's exactly what I did. Testing for Frontend SPOF Probably…
Every now and then the topic of Anycast comes up in the context of web performance so I thought I’d take a stab at explaining what it is and the benefits. tl;dr – DNS servers should always be Anycast (and even some of the largest CDN’s are not so don’t just assume you are covered). Anycast for the web servers/CDN is great if you can pull it off but it’s a lot less common than DNS. Anycast – the…
We recently launched experimental support for testing with Firefox on WebPagetest and I've already had some interesting questions as people started to see things on WebPagetest that they were not used to seeing when they use the Firebug Net Panel (of which I'm a HUGE fan and user). When we built the Chrome test agent for WebPagetest we made the decision to try and make it cross-browser and re-use…
It hasn't happened yet but it's a question of when , not if active monitoring of websites for availability and performance will be obsolete. My prediction is that it will happen in the next 5 years though if everything lines up it could be as soon as 2 years away. By active monitoring I am referring to testing a website on a regular interval and potentially from several locations to see if it is…
Well, I've been promising to do it for a while and it's finally ready. If you've ever wondered what the facility looks like that runs the Dulles test location for WebPagetest (and soon the Web Server), here you go... First up, here is the secure entrance to the cage securely below ground level in case of tornados or other such craziness (yes, in case you're wondering - my basement). Here are the…
I wanted to take a quick moment to dispel any fears that may be brewing about the future of WebPagetest with my move to Google (and in case this is the first time you're hearing it, surprise! I'm at Google now working on making the web faster). This is actually a HUGELY positive move for WebPagetest. Google is putting some engineering resources behind the development of WebPagetest (in addition to…
Stoyan's (always excellent) Performance Calendar went live today with an article I provided looking at the state of performance optimization in 2010 compared to 2008 (based on the tests run on WebPagetest ). I highly recommend reading the article when you get a chance - there's lots of data and graphs. One thing that struck me was how poorly even the most basic optimizations were being followed. I…
One thing I've always been concerned about is people taking a single measurement point and using that as representative of the performance of their site. Even for a well-performing platform (where the back-end is consistent) there are a lot of variations and sometimes significant outliers (I usually recommend taking at least 3 measurements and throwing out any that look obviously wrong). Recently…
One thing that has always amazed me while working on web performance is the need to push advertisers to care about the performance (speed) of their ads. The hard part is that the way things are structured in advertising, the ones paying the money aren't even usually aware that it's something they should care about and it's not something they hold the agencies (that actually create the ads) or the…
This is what open source is all about! Today we are taking the first step in combining the optimization checks done by Page Speed and WebPagetest by making the Page Speed results available from within WebPagetest (and from an IE browser for the first time). Huge thanks go out to the Page Speed team and Bryan McQuade in particular who did the bulk of the work getting it integrated into the Pagetest…
One of the things that has always bothered me about actively monitoring a site's performance (hitting it on a regular interval from an automated browser) is that you are only getting results for the specific page(s) you are monitoring from the locations and browsers you are using for the monitoring. To get better coverage you need to do more testing which increases the amount of artificial traffic…
If you've been over to WebPagetest today you may have noticed that things have changed a bit (after you double-checked to make sure you were really at the correct site). Thanks to Neustar Webmetrics (and Lenny Rachitsky in particular) for kicking in an actual designer to bring the UI out of the dark ages hopefully performance testing will be less intimidating to new users while still keeping all…
Hopefully by now it's clear that javascript in the head is bad - it blocks parallel requests from loading (for all but the newest browsers). If you can't move it all to the bottom of your document or make it async then we usually recommend combining it into a single file and putting it after the CSS so that it will load in parallel with the CSS and reduce the pain. This works because it's actually…
Europe is definitely a hotspot for interest in web performance ( WebPagetest sees almost as much traffic from there as the US). A huge "thank you" goes out to Aaron Peters who volunteered to expand our European testing footprint with a location in Amsterdam. For an inaugural run, he ran some tests of the top online merchants in The Netherlands (according to Twinkle magazine ) and from the looks of…
Last year I did some bulk analysis on the test data from WebPagetest to get a snapshot of what the distribution of results looked like. It's about time to update the data and compare how the distributions have changed over time. It will take a while to crunch the data and generate pretty charts but before going there I thought it would be interesting to see how individual pages have changed over…
This: In case you missed it, Google announced that site performance now affects your search rankings and in the article they included WebPagetest in the list of tools to use. I appreciate the props but it's been a busy day adding more testing capacity to get ready for Monday :-) Edit: As expected, Monday was pretty busy:
Sadly no, not that I'm aware of but it would make total sense and maybe if there is enough interest they would consider it. This is one of those "wouldn't it be nice if..." posts but there is also a fair bit of thought that went into the how's and why's. Google has been pushing to make the web faster on various fronts from Chrome to SPDY to Page Speed to Google DNS and have been saying that they…
A common complaint from users of WebPagetest is that I should make using a CDN (Content Distribution Network) an optional check (and it was a common complaint against YSlow as well before they did make it configurable in version 2). It's usually because their site is too small to justify the expense or complexity of integrating with one of the CDN providers. Recently I had a few different people…
For the most part, web site performance optimization has been something that an experienced developer had to be involved in and if they weren't then odds are your site doesn't perform well. Why do we have to work so hard to make them faster (and more importantly, why are they not fast automatically)? I think there are some key areas that could help significantly if they were addressed: Hosting…
Update 2: All back to fully operational now, sorry for the inconvenience. Update: I have disabled the forums and authentication integration as well as made some changes to at least bring the main testing online. The forums will be back once Dreamhost fixes the problems with the MySQL server. Sorry, Dreamhost seems to be having all sorts of problems trying to keep the server that runs WebPagetest…
This is the first in what is likely to be a multi-post series discussing how HTTP works down at the TCP packet level, opportunities for improvement and a look at how some of the big players perform. Background I'm going to assume that if you're reading this you have a fairly good idea of what HTTP and TCP are and what the roles of each are. What I want to focus on is the design goals of each and…
One of the things we like to look at is what a page looks like when it is loading, particularly if you are comparing a before and after or multiple sites to each other. This is particularly helpful when talking to the business about performance and sometimes the raw load times don't adequately represent the user experience. Up until now this has been a pretty manual process where we screen record…
One of the really important features that WebPagetest offers is the ability to test on various different connection types. This is critical when you're testing the end-user experience because the difference in performance between a fast Ethernet connection to a web site and a consumer connection is HUGE (like orders of magnitude huge). The main differences boil down to bandwidth (how fast your…
I pulled together the results of the tests that have been run on WebPagetest over the past year and did a bunch of aggregate analysis. There were over 24,000 unique urls tested in that time so the data is a pretty wide sampling across different types of sites. You can see the full details here: http://www.webpagetest.org/forums/thread-22.html One of the particularly useful things you can do is…
As I look through the tests that come through WebPagetest I've been wondering if all of the optimization checks really make sense or if we should really be focusing on just the top 3 or 4 things that have the largest impact and not worrying about the smaller things. A LOT of the pages I see come through don't even have the basics in place and they may be overwhelmed by the checklists, etc (and by…
Sort of as a follow-up to the last article, over the last couple of days there were some particularly interesting sites that are just failing horribly: Verizon's DNS error hijack - Verizon recently decided to start monetizing fat-fingered urls by taking over DNS failures and redirecting them to a search page. I'm using their default DNS servers (for now) for the online pagetest so some tests of…
Pagetest has been online for 8 months now with close to 26,000 pages tested. I generally look through the test log daily and see what the results look like and it's been pretty frustrating because a significant number of the sites being tested could easily be twice as fast without changing the content or code but I have no way to reach out to the owner and tell them. The test results are probably…
Sigh, webpagetest.org apparently got some visibility somewhere in the spam and script kiddie/vulnerability scanner communities. A week or so ago I started getting some "link spam" where a group of people had automated bots to kick off "tests" of their link farms. All of the hyperlinks out from webpagetest are "nofollow" links though so I'm still not sure what they hoped to gain from it.…
Dave Artz put together a great tutorial on using pagetest (the web version) for optimizing sites. It's fairly lengthy but well worth the time to make sure you get the most out of the tool: http://www.artzstudio.com/2008/07/optimizing-web-performance-with-aol-pagetest/
Back on June 24th Eric Goldsmith presented pagetest at the Velocity conference . It was well received and he learned some critical presentation lessons (the most important being not to put up an url for something you're going to demo on the first slide with an audience full of geeks with laptops). See if you can identify the traffic spike from the conference presentation: Normally something like…