youtube-dl, Python video download tool, on front page of Hacker News

By Vasudev Ram

youtube-dl is a video download tool written in Python.

I had blogged about youtube-dl a while ago, here:

youtube-dl, a YouTube downloader in Python [1]

and again some days later, here:

How to download PyCon US 2013 videos for offline viewing using youtube-dl

(The comments on the above post give some better / easier ways to download the videos than I gave in the post.)

Today I saw that a Hacker News thread about youtube-dl was on the front page of Hacker News for at least part of the day (up to the time of writing this). The thread is here:

youtube-dl (on Hacker News)

I scanned the thread and saw many comments saying that the tool is good, what different users are using it for, many advanced tips on how to use it, etc. The original creator of youtube-dl, Ricardo Garcia, as well as a top contributor and the current maintainer (Filippo Valsorda and Philipp Hagemeister, respectively) also participated in the HN thread, as HN users rg3, FiloSottile and phihag_, respectively. I got to know from the thread that youtube-dl has many contributors, and that its source code is updated quite frequently (for changes in video site formats and other issues), both points which I did not know earlier. (I did know that you can use it to update itself, using the -U option).

Overall, the HN thread is a worthwhile read, IMO, for people interested in downloading videos for offline viewing. The thread had over 130 comments at the time of writing this post.

(On a personal note, since I first got to know about youtube-dl and downloaded it, I've been using it a fair amount to download videos every now and then, for offline viewing, and it has worked pretty well. There were only a few times when it gave an error saying the video could not be downloaded, and I am not sure whether it was due to a problem with the tool, or with the video site.)

[1] My first post about youtube-dl also had a brief overview of its code, which may be of interest to some Pythonistas.

This other post which mentions youtube-dl may also be of interest:

The most-watched Python repositories on Github

since youtube-dl was one of those most-watched repositories, at the time of writing that post.

- Vasudev Ram - Dancing Bison Enterprises

Signup for emails about new products from me.

Contact me for Python consulting and training.

Share |

youtube-dl, a YouTube downloader in Python

I came across youtube-dl recently. As the title says, it is a YouTube downloader tool, written in Python.

(Update: I made a typo in the title of this post - wrote YouRube. Unintended :-), and fixed now. The post URL can't be fixed, though, except by deleting and re-entering the post.)

I downloaded the tool (it comes both as a binary and as source) and ran the tool on Windows, at the command line, to download a favorite video of mine from YouTube. It worked fine.

youtube-dl has various command-line options that you can set to configure its behavior.

Here is the documentation for youtube-dl.

Then I took a look at the source code. The code for the tool is in the public domain.

It is a few thousand lines long, and divided into 5 or 6 Python source files. There is one file called FileDownloader, another called InfoExtractor, and a few other supporting files.

I've just had a short look at the code so far, but could figure out a few things about the design, and am giving an overview of that below. Going to check out the code in more detail later. It is a non-trivial app and can serve as a good codebase to study.

Instances of the FileDownloader class and InfoExtractor classes register themselves with each other. Actually, a number of InfoExtractors can be registered with a FileDownloader instance. The FileDownloader then passes the input video URL (which the user wants to download) to the chain of InfoExtractors, and uses the first one which reports that it can handle the request, to extract some required info about the video. The FileDownloader itself then does the actual downloading of the file (if requested by the user; it can do other things than download the file, too). Some of its features are:

- Supports multiple YouTube formats - they link to this list of YouTube formats on Wikipedia, so I guess they must support at least some of them; I haven't tried out formats other than the default, but I may later. Some of those YouTube formats include HD video (higher resolution) and 3GP (for mobile). If youtube-dl supports mobile, it could be used to download videos for mobile to your PC, and then transfer it to your mobile, for offline viewing, if you can find an offline video player app for your mobile. Going to try that out for my Android phone.

- Supports multiple video sites, not just YouTube. Some of them are vimeo, Google Video (not sure if that is still operational), blip.tv, SoundCloud, and infoq.

- The tool youtube-dl can be used to update itself, and they recommend you do it often, with the command:

youtube-dl -U

Since YouTube videos are usually in Flash video format, you can play them on your Windows or Linux PC using VLC or MPlayer.

The video I downloaded was in MP4 format, and I was able to play it in Windows Media Player (didn't have VLC just then on that machine), and it did play, though the audio was low and the picture quailty was not that clear, as compared to playing the same video in YouTube via the web client. Windows Media Player gave a message that the MP4 format was unknown, but was able to play it anyway, though with the stated limitations.

youtube-dl can use either urllib or urllib2 from the Python standard library, can do download rate-limiting if asked to, and many other things, as part of its features.

An interesting Python tool, overall.

On a related note, a lot of YouTube itself is written in Python. I had seen a video about that by one of the YouTube engineers, some time ago.

Here's the summary of a talk at PyCon 2012 on Scaling YouTube - it mentions that most of YouTube is written in Python, and that YouTube is one of the largest web sites in the world, that uses Python. I just saw on Wikipedia in the entry about YouTube that it may be the third-most viewed site in the world.