Diffen • blog

Using Cloudflare Workers to get your Fastly cache hit rate to 95%

I love both Cloudflare and Fastly, and use both services for Diffen. Cloudflare powers the DNS and delivers the assets (images and JS) on static.diffen.com. Fastly is the CDN for serving HTML content, powering the main www subdomain. (if you’re wondering why, it’s because Fastly lets you stream your access logs to Bigquery, a feature only available to enterprise customers on Cloudflare.)

This is the story (and code) of how I improved the cache hit ratio from Fastly using Cloudflare Workers.

How a CDN works
A CDN has several POPs (points of presence) across the world. For example, Fastly’s network map is here. When a user makes a request for a web page, it gets routed to Fastly, and is usually handled by the POP that is closest to the user. If that POP has the content in its cache, it can serve it to the user immediately. But if the content is not cached at that POP, Fastly makes a request to the “origin” – the server that actually runs your web app – to fetch the content. This extra hop adds latency, and the total response time for the end user could end up being slower than having no CDN at all.

Naturally, you want your cache hit ratio to be as high as possible.

Priming the cache
Ideally, I’d like to prime the cache in every Fastly POP for my most popular content. And given that resources get booted out of Fastly’s cache depending upon their load, I’d like to re-prime periodically. Unfortunately, Fastly does not provide a mechanism to prime its cache.

One way to do it would be to lease VPSes in cities all over the world and run cron jobs to request these pages every hour. But this is… infeasible. Run AWS lambda functions from various data centers? Not enough diversity of cities. There are far more Fastly POPs than AWS data centers.

So let’s fight fire with fire. Cloudflare’s network has as many (if not more) POPs as Fastly. And lucky for us, Cloudflare allows you to run code on the “edge” i.e., in each of its data centers.

Using Cloudflare workers

A couple of weeks ago, Cloudflare announced Triggers for workers. I was excited to get cron job-like functionality for workers. However, their blog post says:

Since it doesn’t matter which city a Cron Trigger routes the Worker through, we are able to maximize Cloudflare’s distributed system and send scheduled jobs to underutilized machinery.

So we can’t specify which POP is used to run our worker. Bummer! How else can we run our crawlers from various POPs around the world where our users are? 

We use our users’ locations. Whenever a user visits a page, we invoke a Cloudflare worker. This worker will run at a Cloudflare POP closest to that user.

fetch(‘https://cache-primer.diffen.workers.dev’);

The job of the worker is to (1) make a call to an endpoint to request a list of URLs to prime the cache for, and (2) crawl those URLs.

async function handleRequest(request) {
  const resp = await fetch('https://www.diffen.com/API-that-returns-a-list-of-urls-to-crawl’);
  let urls = await resp.json();
  for(let url of urls){
     await fetch(url);
 }
 return new Response(“OK”, {
     headers: {
         'Access-Control-Allow-Origin’: 'https://www.diffen.com’
     }
 });
}

The job of the API-that-returns-a-list-of-urls-to-crawl is to (1) know the top pages we need primed in Fastly’s cache, (2) maintain the list of pages we have cached in each Fastly POP, and (3) for a given request (from a Cloudflare worker), return ~10 pages that we have not primed yet in that POP.

Let’s say the user is in Seattle so Cloudflare’s Seattle POP is where the worker runs. It makes a request to the API to get a list of URLs to crawl. This API request is routed to the origin server via Fastly’s Seattle POP. On the origin, we can see the name of the POP in the X-Fastly-City header of the request.

<?php

header(“Cache-Control: private, max-age=0”); //Make sure this response does not get cached.
header(“Content-type: application/json; charset=utf-8”);

$topUrls = getMostPopularUrls(); //This is a static list
$headers = getallheaders();
$city = $headers['X-Fastly-City’];
$primedUrls = getPrimedUrls($city);
$unprimedUrls = array_diff($topUrls, $primedUrls);
$newUrlsToPrime = array_slice($unprimedUrls, 0, 10);
primeNewUrls($city, $newUrlsToPrime); //Maintain a list of the URLs we have already primed
                                       //so we don’t prime them again for a few hours.

echo json_encode($newUrlsToPrime);

return;

That’s all there is to it.

We are now using one CDN to prime the cache at another. Cities with more users will have a higher chance of primed caches.

How much does it cost?

Cloudflare lets you make 100,000 requests per day for free. The paid plan is $5 for 10 million requests per month. Totally worth it.

Results
The cache hit ratio went from ~75% to ~95% with this change. YMMV; the strategy would work better for top-heavy sites where a small number of pages account for a large share of overall traffic. I wouldn’t want to jam thousands of pages into Fastly’s cache when there is a slim chance of a real user ever needing them.

You’re Doing Social Logins Wrong

Social logins can be great because it’s one of those features that’s good both for users and for app developers.

Users often don’t want to create yet another account with yet another username and password. They may end up re-using the same password across many sites, which leaves them vulnerable. 

App developers want to reduce any friction they can to increase user engagement. Creating an account is often a pretty big hurdle. Besides, it’s easy to screw up auth; it would be fantastic if you could simply avoid the headache of storing user credentials and outsource auth to a trusted, secure third-party like Google or Facebook.

But time and again I’ve seen app developers use it in a bad way, which turns users off from this method of login. Granted, federated login has plenty of problems but this rant is about 1 particular mistake.

The first and obvious mistake is asking for too many permissions. You don’t need to see my friend list and interests and likes for me to have an account for your web app. This is just like Android apps asking for a ton of permissions (pre-Marshmallow) — why does Pandora need to access my contacts list?

But this blog post is about a different mistake that is widespread but sadly not pointed out enough. It’s when you click on Facebook login, authorize the app and then it asks you to create a username and password.

The screenshot above is from the awesome free photo website Unsplash. I didn’t want to create yet another account there so I logged in with Facebook, authorized their permissions (public profile and email, well done there Unsplash!) and expected to be logged in. But this next step of choosing a username and password is just completely unnecessary.

If your app can’t function without each user having a username, then auto-generate a username for them and let them change it. But please don’t force them to create a username and password when they used their social login precisely to avoid that.

How a Progress bar for AJAX requests helped increase conversions by 38%

The NY Times published an interesting article this week about how progress bars can make it easier for people to wait for computers to respond. This idea is not new; usability experts have long advocated their use. Developers and designers have even tried to have some fun with progress bars.

When we launched the Get from Wikipedia feature on Diffen, we wanted to make the wait bearable for our users and hold their interest. Our first progress indicator was just a a loading GIF. Here’s how we replaced it with an actual progress bar and increased conversions by 38%.

First, for those who don’t know, Diffen is like Wikipedia for comparisons. If you compare two entities on Diffen and there is no information about either of them in our database, the system tries to get the information from Wikipedia infoboxes. Here’s a 20-second demo of how it works:

The page makes an AJAX request to get information about each entity. The back-end needs some time to respond because different variations and misspellings of the entity name may have to be tried before finding the right Wikipedia page for it. Also, the Wikipedia page must be parsed to tokenize the attribute/value pairs from the infobox. (We can do a separate blog post on how this is done if there is interest.) When both AJAX requests are completed successfully, the attributes are lined up side by side and shown to the user.

How we implemented the progress bar

First we benchmarked the time it took for a typical AJAX request for entity information and found that the time varies from 3-8 seconds. So we needed a progress bar that could last as long as 8-10 seconds. That’s a pretty long time, which means we needed lots of status messages. 

The status messages in the progress bar are based on what the server is likely doing, not what the server is actually doing at that time (more on that later). The status messages we use are:

var progressStatuses = [
    "Searching Wikipedia",
    "Scanning search results",
    "Reading the right Wikipedia page",
    "Scanning the information box",
    "Preparing the info",
    "Preparing the info",
    "Receiving the info",
    "Almost there"
];

The server doesn’t really send any progress report to the browser. The progress bar isn’t real; it is only meant to placate the user and tell her the system is working on her request.

A word about timing

The easy way to make progress is to increment to the next step at a set interval, say 1 second. But that is more likely to draw suspicion and give away our illusion of authenticity.

Another approach would be to progress through the initial 3 stages really fast to get the user hooked, and then gradually decelerate the pace of progress.

Instead we try and use intermittent variable rewards, known to be more pleasurable and addictive. The basic idea is that random reinforcement is more addictive (because it is more pleasurable) than consistent, predictable reinforcement for the same activity. We spend a random amount of time - between 500 and 1500 milliseconds - at each stage.

function makeProgress (reset){
    var div = $("#progressIndicator"),width=0;
    if(div.length==0)
        return; //Operation complete. Progress indicator is gone.
    if(reset)
        div.width(0);
    else
        width = div.width();
    if(width < 300){
        width += 45; //make the bar progress
        div.width(width);
        $("#progressStatus").html(progressStatuses[(width/45)-1]+'...');
        var delay = Math.floor(Math.random() * (1500 - 500 + 1)) + 500; //A random number between 500 and 1500
        setTimeout(makeProgress, delay, false);
    }
}

That’s pretty much all we need for the progress bar. The initialization code is somewhat like this:

    
function importFromWikipedia(){
    setTimeout(makeProgress,100,true); //start the progress bar
    $.getJSON("/call/to/backend/script/for/entity1",fnOnSuccess);
    $.getJSON("/call/to/backend/script/for/entity2",fnOnSuccess);
    function fnOnSuccess(data){ 
        //Did we get information about both entities or only one? 
        if(bothReceived){ 
            showTheComparisonChart(); 
            $('#progressIndicator').remove(); //helps us break out of the makeProgress() recursion
        }
        else {
            //Mark one entity as complete and wait for the other one         } 
    }); 
} 

First, we initialize the progress bar. Then we make the AJAX requests to the backend. When both requests are completed, we remove the progress indicator and show the comparison chart.

It is not necessary to loop through all 7 available progress status messages and “complete” the progress bar visually. If the user only sees the first 3 before the request completes, she is not going to care. In fact, if she notices it at all, she is likely to be delighted that the operation finished quicker than expected. 

It’s not very pretty, but this progress bar is quite functional. When this progress bar replaced a loading GIF we saw conversion rate (as measured by the weekly number of new entities imported from Wikipedia and saved to our database) increase 38%. 

Discuss on Hacker News

The Decline of IE and Firefox

Windows and IE are dying under the assault of iPads and other tablets from the left, and Chromebooks from the right. This is not surprising, or even news to people who spend a lot of time in the tech world. What surprised me recently was how precipitous the decline of Internet Explorer has been in the past few months.

(update: some comments on Hacker News questioned why the y-axis on this graph doesn’t go from 0 to 100. Browser shares tend to move slowly. All 4 lines will appear essentially flat if I did that. The story would be lost. This type of chart with a short range for the y-axis is the best way to depict browser share.)

Flashback to summer of 2012 when IE’s share of traffic to Diffen was a little over 26%. This was for n= ~ 1 million visitors. Fast-forward to December 2013 where IE’s share has dropped to 12.17% (n = ~ 4 million visitors). Diffen is a consumer-oriented website with over 60% traffic from the U.S. So yes, this is probably not a true reflection of worldwide market share. But boy is there a decline in Q4!

And all of it cannot be blamed on the decline of Windoze or PCs. Windows users still accounted for 47% of traffic in December. And Desktops accounted for 60% of all traffic. The share of mobile traffic jumped from 29 in October to 32% in December.

Also notable is the slow but steady decline of Firefox. They’ve been doing some really cool stuff with asm.js and PDF.js but this does not seem to be translating into any gains in mainstream market share.

The good news in all of this is – you guessed it – no need to support IE < 8 ! Here’s how IE’s 12.17% share in December breaks down into all its versions:

(update: in response to Christian Jensen’s request in the comments below, I have translated these numbers into % of total market share in this chart:)

Of course, I’d prefer to not have to deal with IE 8 either but at least right now its share is material.

Disclaimer: Once again, these numbers do not reflect the state of the web. There are many other sources of browser share statistics that are based on more data and offer more granularity for your research. If you already have a web presence, look into Google Analytics for data on your audience. If you’re working on a startup in the consumer space, expecting visitors mostly from the U.S., Canada, UK, and about a third of them from their smartphones, then these number might be more relevant to you.

Bonus tip: If you have goals set up in Google Analytics, create a custom report to see conversion rates by browser and by screen resolution. This can potentially uncover problems where certain segments of your visitors aren’t converting because the site is not working for them the way you expected. (Hat tip: @CalebWhitemore)

Discuss on Hacker News