800px-Solid_color_You_Tube_logo

So, you’ve got a fancy phone, laptop, or tablet, and an internet connection capable of pulling down entire full-length movies in 1/10 of the time it takes to watch them. But any time you attempt to watch a YouTube video in quality above 240p, you end up stuck with YouTube buffering every 18 seconds. (This doesn’t always happen to everyone, but if you’re not getting these YouTube buffering issues, congrats: this piece is not for you.)

The problem is usually not that YouTube is overloaded, nor is your internet connection horrible (unless it is, then that’s another story). Instead, the problem is that your internet service provider is pointing you to cached YouTube servers inside their network. What this means is that, inside your internet service provider’s scope of control, there sits a route which costs your provider less to stream YouTube through.

If you’re really interested in how this works, search for ISP peering agreements and you can read all about it. The short of it is that someone upstream is attempting to save money, and your YouTube stream of The Chica Show in HD is suffering.

What I’m about to discuss involves modification of the hosts file to redirect requests to YouTube’s cached servers directly to YouTube, bypassing your ISP’s servers. On Windows systems it’s located at:

C:\Windows\System32\drivers\etc

…and is a plain text file that has to be edited with administrative access (Start > Notepad > right click on notepad.exe > run as administrator).

On Android, you can locate it generally in /system/etc/hosts, but if for some reason your ROM moved it, you can find it with the command find / -name hosts. Macs and Chrome OS have similar setups – pretty much any OS has this file somewhere and the format is pretty much universal. However, you should also note that there is currently no one-shot solution that exists out there that works for all ISPs and all geographic regions that I can find to address the issue of YouTube buffering.

What the hosts file does

The hosts file is looked at before the DNS is checked. DNS translates the name Pocketables.com in to an IP address of 209.132.196.105, which is how your browser connects to us. However, if there’s an entry in the hosts file for pocketables.com, the local information will be trusted, and whatever you’ve put there will be used instead.

The short and easy is that hosts file overrides whatever the internet has to say, and even if your ISP swears YouTube is at 74.125.137.136, you can force it to go to 173.194.41.128.

Where things get tricky

The YouTube caching servers, and where you’re sent, are changed frequently and vary by ISP and geographic region. You can see where you’re being sent here on the YouTube redirector page. You’ll also note that when refreshing the page, you get different names. I get atl14s09, 17, and 23 pretty much all the time, which means my videos will be served up by r1.atl14s09.c.youtube.com through at least r10.atl14s09.c.youtube.com, from (I assume) an Atlanta Comcast/Google peering point.

I’m assuming the r followed by a number at the beginning of the URL is a round-robin number, as each number you increment there ends up incrementing the IP address.

Each ISP is going to have its own set of peering points, so unless you can locate a universal list of all peering points by all ISPs, you’re probably going to need to find your own points. It should be noted that if you’re connected on a home network, just go to the computer – it’s much easier than attempting to find out what you’re connecting to on a phone, although the redirector page will still tell you where you’re going no matter what.

A word of warning here: using TOR, Chrome Beta with bandwidth compression, or any other proxy will skew your results on the redirector page, as you’re connecting via a proxy or different IP.

Modifying the hosts file

Using a root browser, vi, notepad, etc., you’re going to modify the contents of the hosts file. So you understand what you’re doing, you’re creating a local DNS override, and if you mess up what you’re doing, pretty much nothing bad should happen.

Quick note: Adblock, Adaway, or anything else that modifies the hosts file is going to wipe out changes you make. So disable them while you’re running this experiment.

The format of the hosts file is: IP address you want to go to, a space or tab, the name. There are some more things you can do with it, but let’s say you wanted to block twitter.com. You would have a line that says 127.0.0.1 twitter.com, and any attempts to go to it would fail.

What to put in

Before you begin, verify that you can actually save the hosts file you’re working on. If you haven’t opened it with the correct permissions, you will end up getting done and not being able to save your work. Antivirus applications also have a tendency to freak out when the hosts file is modified, as that’s how a lot of browser hijacking is done.

So, you’ve refreshed the redirector page a few times, and probably have a sense of what keeps popping up regularly. For me it was the same three host names. You’ll need a list of actual YouTube servers, as opposed to caching servers.

According to this excellent post that will work as an out of the box solution for people in seeing LAX, DFW, SJC or IAD as their cached content servers, the IP addresses of the actual YouTube servers that are being cached range from 173.194.41.128 to 173.194.41.142. Using that information, we can build our host entries like so:

173.194.41.128 r1.atl14s09.c.youtube.com
173.194.41.129 r2.atl14s09.c.youtube.com
173.194.41.130 r3.atl14s09.c.youtube.com

173.194.41.138 r10.atl14s09.c.youtube.com

173.194.41.139 r1.atl14s17.c.youtube.com
173.194.41.140 r2.atl14s17.c.youtube.com
173.194.41.142 r3.atl14s17.c.youtube.com
173.194.41.128 r4.atl14s17.c.youtube.com

etc.

For me, disabling the first ten round-robin cache servers of the three that I get put on regularly takes 30 lines of entries. Each entry increments the IP address on the left until we reach the highest number we know YouTube has.

What this is doing is just throwing any cached server requests to the real servers, which are more than capable of serving the videos themselves.

Tip: Every hosts file is basically the same. If you do this on a computer, you can copy it to Android, Mac, or a smart toaster later.

Testing

On Windows the changes to the hosts file are immediate. I do not know whether they are across the board in Android, Chromium, etc., as some apps say you may need to reboot after changing the file.

Now go to YouTube and see if it works. If it’s still horrible, you may need to run netstat -p tcp on Windows or Android to see what you’re actually connecting to while you’re experiencing a YouTube buffering video.

Other ways

There are proxy services that will bypass your ISPs caches, and you can also modify your router to block, drop, redirect, or reject connections to the cached servers, which is useful for other devices like an XBox (which I can’t stream anything on).

Wrap-up

The fix is somewhat long and complicated for non-IT folk, and doesn’t have a cookie-cutter solution yet. If you make a hosts entry that works, please post it with the ISP name, and the region you’re located in (i.e. Comcast, Nashville TN). Maybe we could put something together that works for everyone and someone could make an app that applies it automatically.