CDN video: HLS performance measurement

HTTP Live Streaming

The recent Devoncroft’s 2013 Big Broadcast Survey (BBS) shows multi-platform content delivery is the most important trend considered by the broadcast industry. It is not a surprising news when we review how much time we spend viewing videos on non-TV devices.

Video streaming on iOS devices is supported by HLS (HTTP Live Streaming). HLS democratises video streaming: everyone can now easily stream adaptive bit-rate videos to iOS devices by using a low cost HTTP servers.

In comparison, it is much more challenging to stream videos to other mobile platforms. Here is a good reference article from LongTail: The Pain of Live Streaming on Android

We can easily stream videos to iOS devices. How do we measure the video performance? How do we validate HLS videos delivered from CDN are much better than those from customer origin? Which adaptive bit-rate video stream in my demo page – HLS Performance Measurement – provides better experience?

HLS Performance Measurement by PocketProbe

Rather than subjective measurement by our eyes (目測法), PocketProbe Free from Bridge Technologies provides an objective measurement of HLS video experience.


I did some PocketProbe tests around 12:30pm 20130626 HKT. Left is the test result of stream from customer origin (CO) and right is from CDN respectively.

PocketProbe test results

1.CDN stream was delivered more frequently at higher speed than the CO stream.
2.CDN stream was retrieved much faster than CO stream.
3.CDN stream was delivered constantly at the highest bit-rate while CO stream switched to lower bit-rates during the delivery.

We can SEE the CDN stream picture resolution and quality are better too.

iOS a great platform for video streaming. PocketProbe is a cool app for video streaming performance measurement.

Easy. Low cost.

CDN Load Balancing by Cedexis Openmix

Why multi-CDN?

Different CDNs have different network footprints. A CDN performs differently at different times due to different traffic patterns and loadings. One CDN may perform better than others for a specific geo region at a specific point of time.

Say your website is using two CDNs: provider A and E. Sometime A performs better than E. Sometime E performs better than A. Your website performs the best if your website is always using the best of them at any given time. Doing so you need two things
-a measurement system monitoring performances of both A and E
-an intelligent DNS which can change of content URL to CDN URL of either A or E based on their performances

EdgeCast and another CDN

In this demo, I use Cedexis Openmix to load balance two CDN services for four big images on this original webpage. About 60% of the page size is from these four images.

I use EdgeCast and another CDN to show the Cedexix Openmix CDN load balancing feature. There are some CDNs provide free services – Incapsula, CloudFlare, CDN.NET – at the end I create a 14-day free account at CDN77 because they are easy to use, do not require me to migrate my domain to their name servers and do not ask my credit card information.

I set up both EdgeCast and CDN77 to cache the contents from, the hostname of the four images on the original webpage, and the CDN URL hostnames are and respectively. I add two CNAMEs in my domain – and – to hide the CDN URLs.

To use a single CDN is simple: to change the image origin URLs to either one of the aforementioned CNAME URLs.
Using EdgeCast:
Using CDN77:

Cedexis Openmix

In order to dynamically load balance among these two CDNs by Cedexis Openmix, there are a few steps:
1.under Platforms: to add both EdgeCast and CDN77 to Platforms. Cedexis Openmix will load balance based on the performance of the Platforms entries.


2.under Openmix: to add a new DNS Application. I select Optimal Round Trip Time in the Application Type. I put in Fallback CNAME. Time to Live (of the Cedexis CNAME assigned later) is 20 seconds. In destination platform, I add both platforms defined in step (1): EdgeCast with CNAME and CDN77 with CNAME of Cedexis then assigns a Cedexis CNAME for this DNS application:



3.under Openmix: once I publish this DNS application, every 20 seconds Cedexis will change the CNAME of to either or if at that moment the best performing CDN is EdgeCast or CDN77 respectively.

4.EdgeCast and CDN77: to add as a CDN CNAME. Doing so when a HTTP request with Host header is received, EdgeCast will point it to the CDN URL CDN77 will point to the CDN URL

5.the final step: in the original webpage, to change to The result webpage is

To summarise: Cedexis update its DNS record so that is (CNAME to if EdgeCast performs better, OR (CNAME to if CDN77 performs better

The actual result? Between 12:00pm and 1:00pm 2013-06-17 Hong Kong Time, there were 847 DNS queries of, Cedexis Openmix resolved 62% queries to EdgeCast.



I used Catchpoint to do synthetic HTTP GET test to measure webpage response time from Asia, North America and Europe. Here are the test results between 7pm 2013-06-17 and 7pm 2013-06-18 Hong Kong Time:
Using EdgeCast only: 2466ms, 24768 tests
Using CDN77 only: 2528ms, 24768 tests
Load balance between EdgeCast and CDN77: 2493ms, 24768 tests. Cedexis Openmix routed 68% of traffic to EdgeCast.


With similar idea, I can use Openmix to load balance CDN and the original server. I need to add a custom Platform in Cedexis to measure the origin server. Here is a example of a 50-50 Round Robin load balancing between EdgeCast and the original server, the one listed on my demo site.