Make your embed Youtube video works in China

Objective:
To make your website videos great again.

A Forbes article says video is of the seven important areas which can improvement a website user experience and engagement.

Many companies agree and add Youtube videos to their websites. Why Youtube? No cost, easy to use (to embed videos in webpage) and easy to consume (to watch and to share, from enduser perspective). Besides some other downsides, one major challenge of using YouTube video on your corporate website is that YouTube is blocked in some countries (China, for example). So you neither can tell your story using Youtube nor you provide a good UX to all users.

Broken Youtube video

It is not unusual to see broken content… it is not great:(

Solution:
Even Google cannot make Youtube works in China. I cannot too. What I am trying to do is to add user location-awareness to the website. To make a website to show different content to users in different locations.

I Cloud can do all things. I am going to use different cloud services to demonstrate the solution.
The key element is a DNS service with Geolocation Routing capability, such as Verizon Route and AWS Route 53. There is one hostname an and URL only. No need to remember xxx.com, xxx.com.cn, xxx.com.jp, etc.
cdn.53.cheungwaikin.com/video.html

 

Geolocation Routing lets you choose the resources that serve your traffic based on the geographic location of your users, meaning the location from which DNS queries originate.

 

Demo setup:

  1. a webpage (a) with Youtube video, hosted on a AWS S3 bucket
  2. a similar webpage (b) using a mp4 video, hosted in another AWS S3 bucket
  3. webpage (a) is accelerated by KeyCDN for Europe, and AWS CloudFront elsewhere
  4. webpage (b) is accelerated by ChinaChina for China and Japan
  5. AWS Route 53 is configured to resolve the hostname cdn.53.cheungwaikin.com to
    • ChinaCache CDN URL for users located in China or Japan
    • KeyCDN CDN URL for user located in Europe
    • CloudFront CDN URL for user elsewhere
AWS Route 53 Geolocation Routing setup

AWS Route 53 Geolocation Routing setup

Verification:
We will use VPN service such as VyprVPN or Hola!VPN to connect to different locations and then browse cdn.53.cheungwaikin.com/video.html and see what content we get. Below video showing that I received the webpage (a) (with embed Youtube video), then I got the webpage (b) when I was using Hola!VPN to connect to Japan.

Cool?

Wish you a joyful weekend:)


How does a CDN work?

Last update: 20170628

Last Friday I was asked “how does a CDN work?” The first time I was asked the same question was year 1999. I acquired a lot of CDN knowledge and experience since I join the industry in 2012. CDN is not yet an everyday business in Asia nowadays. I want to help. An one page brief explanation. Hope you like it.

What is a CDN?

A Content Delivery Network (CDN) is a collection of web servers distributed among multiple locations to deliver content to users more efficiently.[1]

What is the job to be done by CDN?

To send web content to enduser faster. How? By moving data close to the endusers to reduce round trips and network.

Ilya Grigorik, web performance engineer at Google, says [2]:

  • Latency, not bandwidth, is the performance bottleneck for most websites.
  • Four major latency types: propagation delay, transmission delay, process delay, queuing delay
  • To improve performance of our applications… we need to reduce round trips, move the data closer to the client.


Centralized Server vs CDN [3]

How does a CDN work?

In essence, CDN puts your content in many places at once, providing superior coverage to your users. For example, when someone in London accesses your US-hosted website, it is done through a local UK CDN server. This is much quicker than having the visitor’s requests, and your responses, travel the full width of the Atlantic and back.[4]

How??

By connecting users who request your website content to CDN, instead of connecting directly to your web server.

Using traditional centralized server model, a website, say www .abc.com, will usually have a “A Record” in the DNS configuration (of abc.com) to inform people to connect to the web server with the specific IP address, say 1.1.1.1. We call this Customer Origin. When a user John in Beijing browses www .abc.com the following (behind the scene) steps will happen:

  1. John’s computer will ask the its local DNS resolver to figure out the IP address of www .abc.com
  2. the resolver will then ask the root DNS server what is the authoritative DNS server of abc.com
  3. The resolver will then ask the authoritative DNS sever of abc.com what is the IP address of www .abc.com and will get the answer of 1.1.1.1.
  4. John’s computer will then connect to 1.1.1.1 to get the web content of www .abc.com

With CDN, www .abc.com will have a “CNAME Record” in the DNS configuration to inform people to connect to the server with the specific hostname, say abc.customer.cdn.com provided by the CDN provider. When John browses the website www .abc.com, we will have the same first two steps above-mentioned, but subsequent steps will become:

  1. The resolver will then ask the authoritative DNS server of abc.com what is the IP address of www .abc.com and will get the answer of abc.customer.cdn.com
  2. The resolver will then ask the authoritative DNS server of cdn.com what is the IP address of abc.customer.cdn.com. The authorization DNS server of cdn.com will, based on the IP address of John’s local DNS resolver and the CDN provider’s routing algorithm, provide the answer, say 2.2.2.2, which is the IP address of the CDN cache server located in/near Beijing and is the best machine to deliver the content of www .abc.com to John
  3. John’s computer will then connect to 2.2.2.2 to retrieve the web content of www .abc.com

Say there is another user Mary in Singapore going to browse www .abc.com, we will have the similar four steps, but the CDN provider will use, say 3.3.3.3, which is the IP address of the CDN cache server located in/near Singapore which can serve Mary most efficiently.

Do I need to upload my website content to CDN?

Not necessary. In most cases, CDN will on-demand fetch content from the Customer Origin.

When the CDN cache serve receives the HTTP GET request of www .abc.com from an enduser, the cache server will check if the requested content of www .abc.com is already stored in its storage:

  • If yes, then the cache server will send the content stored on it to the enduser. We call this “Cache Hit”
  • If no, then the cache server will fetch the content from the Customer Origin, then store it and send it to the enduser. We call the situation of missed content “Cache Miss”, and the process of fetching content from Customer Origin “Cache Fill”. CDN will Cache Fill when it needs to update the stored content.

The higher the Cache Hit rate, the faster the content delivery. High Cache Hit rate better utilizes CDN and reduces Customer Origin loading.

How long will the CDN keep my web content?

CDN provider will usually honor the Cache-Control header setting in Customer Origin. During the Cache Fill process, the CDN cache server will cache both the web content and the HTTP header information, including the Cache-Control header attributes. CDN provider will keep the content according to the Cache-Control setting or based on the customer CDN configuration.

What else?

CDN nowadays offers more features and services. Below are the key benefits of using CDN:

  • Faster delivery of both cacheable and non-cacheable content
  • Higher availability by leveraging CDN’s global footprint and scale.
  • More secure by shielding the Customer Origin from public access.
  • Reduce the expense on Customer Origin resources (computer and connectivity)

That’s all.

Reference:

  1. High Performance Web Sites, Steve Souders
  2. High Performance Browser Networking, Ilya Grigorik
  3. CDN, Wikipedia
  4. Essential Guide to CDN, Incapsula
  5. Cache-Control header, Mozilla.org

Notes

I prepare this blogpost on the new iPad Pro 10.5” with Smart Keyboard. It is GREAT. Looking forward to iOS 11!

img_1194

[Update: 20170628] Apple iPad Pro 10.5” with iOS 11 beta

IMG_3538


How to use HolaCDN to load share from multiCDN

In my previous post I showed how to use HolaCDN to improve the performance of a mp4 video which is already accelerated using CDN service.

This time I would like to show how to use HolaCDN MultiCDN feature to improve the performance of a multi-birate HLS video. HolaCDN MultiCDN will automatically measure performance of multiples CDNs (defined by user) and select the optimal CDN to use.

In this demo, HolaCDN will select among the customer origin, a 3rd party CDN and HolaCDN.

In the screen capture below, Chrome Developer Tools shows the HolaCDN JS client loads video chunks from serveral hosts: s3video.cheungwaikin.com (customer origin), dc7tqspqe5l18.cloudfront.net (a 3rd party CDN) and 76.73.18.98 (a HolaCDN server).

chrome-network

The following picture shows the result generated by the command hola_cdn.get_stats() in the Chrome Develop Tools Console: 20% of traffic is from the customer origin, 18% from the 3rd party CDN, and the remaining is from HolaCDN.

chrome-console.png

How to set up HolaCDN MultCDN feature in this demo?
Step 1: create a new Zone named “hlsmulti” by defining the Page URL as http://aws.cheungwaikin.com/video-hls-flowplayer-cloudfront-hola-origin.html.

Step 2: in the MultiCDN selection algorithm, I input the $0.5 as the cost of HolaCDN and $0.05 as the cost of “My existing CDN”. Doing so to instruct HolaCDN to use My existing CDN unless HolaCDN improve performance (better video start/seek time, less buffering, higher video quality/bitrate).

Step 3: for demo purpose, HolaCDN will take actions – collects QoE statistics and deliver video content – if the user is using Chrome browser. For every 100 pageviews from Chrome users, all will colllect QoE statistics, and HolaCDN will try to deliver video content in 90 pageviews.

Step 4: define “My existing CDN”. In this demo, the are two “CDNs” – dc7tqspqe5l18.cloudfront.net and s3video.cheungwaikin.com.

Step 5: I use Static MultiCDN switching mode. If HolaCDN finds dc7tqspqe5l18.cloudfront.net and s3video.cheungwaikin.com perform better than HolaCDN, then 50% possibility that video will be retrieved from dc7tqspqe5l18.cloudfront.net and 50% possibility that video will be loaded from s3video.cheungwaikin.com.

holacdn-conf.png

That’s all.

In other to do a control experiment, I use Firefox to browse the demo page. Based on my setup, HolaCDN will do nothing and the browser will get all HLS TS chunks from the host defined in the html source code. Here is the result from Firefox + Firebug:

firefox-net.png

Want to try it yourself? Go to apply a HolaCDN free trial account now!


How to use Hola!CDN for MP4 progressive download

Hola is a Spanish word meaning Hello.

Hola! is a startup headquartered in Netanya, Israel. Hola!’s goal is to make the web faster, more open and cheaper to operate.

Hola!CDN is the first CDN purpose built for video. It is developed and operated by Hola!. Hola!CDN makes Internet video delivery Faster, Cheaper and More Reliable.

It is my pleasure to share with readers that I am now working at Hola!CDN!

In my last blogpost I talked my first Israel visit. The main objective of the trip was to attend Hola!CDN training.

The unique Hola!CDN architecture makes it possible to do Five things that other CDNs cannot do:

  1. Significantly speeds up your video delivery (start time, buffering, seek time, bit rate, failure rate)
  2. Provides accurate statistics on your viewers’ quality of experience
  3. Enables mid stream switching of multiple CDNs (configure Hola!CDN’s dashboard to work with several CDNs, taking each video chunk from the fastest and cheapest)
  4. Reduces video delivery cost substantially, with same low cost from first byte, to anywhere in the world
  5. Keeps your existing CDN – Hola will only participate where its faster and cheaper

The new architecture is comprised of a client side JavaScript code that manages the video delivery from the viewers’ browser. The integration of the Hola!CDN client side module with the Hola!CDN server module creates a fast, seamless user experience, and a low cost of delivery.

Detail Hola!CDN implementation guide can found at GitHub:  https://github.com/hola/cdn/blob/master/install.md

In this blogpost I would like to SHOW how to implement Hola!CDN in a webpage with MP4 progressive download.

Step 1: apply a Hola!CDN account account. We offer 14-day free trial account at holacdn.com

Step 2: configure Hola!CDN service. The minimum step is to add your Video Source(s). Video Source is the hostname of the CDN or customer origin that your video is cached/stored.

Hola!CDN customer portal

Hola!CDN customer portal

Step 3:  insert the codes in the header section to instruct the browser to downland the Hola!CDN JavaScript client with your specific configuration, and insert another codes at the end of the body section to instruct the browser to download the Hola!CDN JavaScript client asynchronously

HTML codes to implement Hola!CDN

HTML codes to implement Hola!CDN

Three steps. That’s all. No DNS change is required!

 

Below is the screen shot of Chrome DevTools of the original page:

Chrom DevTools of original page

Chrome DevTools of original page

This is the screen shot of Chrome DevTools of the page implemented Hola!CDN:

Chrome DevTools of the page implemented Hola!CDN

Chrome DevTools of the page implemented Hola!CDN

There are 6 HTTP requests in the original page and 65 in the page implemented Hola!CDN. Why?

Let’s talk more next time.


Hi Year of Monkey! Hi MSE!

Gong Xi Fa Cai! Wish everyone has a happy and healthy Year of Monkey.

It is the fifth day of the new Chinese New Year. It is time to write something new.

I am studying Media Source Extensions (MSE) recently. In a nutshell, MSE enables browsers to play streaming media without plugins.

What is MSE?
-W3C: MSE is a W3C specification that allows JavaScript to send byte streams to media codecs within web browsers that support HTML5 video. The first public draft was released in Jan 2013. Among other possible uses, this allows the implementation of client-side prefetching and buffering code for streaming media entirely in JavaScript.

-Microsoft: build custom adaptive streaming solutions using MSE. Such an adaptive streaming solution would look for the optimal bit rate based on network bandwidth and CPU utilisation to provide the highest quality streaming experience. For example, if the network bandwidth drops, you can use MSE to programmatically switch the video stream to a lower bit rate. This reduces the quality of the image, but allows the video to keep playing without interruptions.

-Netflix: This makes it possible for Netflix to download audio and video content from our content delivery networks and feed it into the video tag for playback. Since we can control how to download the audio/video content in our JavaScript code, we can choose the best HTTP server to use for content delivery based on real-time information, and we can implement critical behaviour like failing over to alternate servers in the event of an interruption in content delivery. In addition, this allows us to implement our industry-leading adaptive streaming algorithms (real-time selection of audio/video bitrates based on available bandwidth and other factors) in our JavaScript code. Perhaps best of all, we can iterate on and improve our content delivery and adaptive streaming algorithms in JavaScript as our business needs change and as we continue to experiment.

More and more video providers are moving away from browser plug-ins to MSE. Youtube and BBC are two examples.

Not all browsers are created equal. At the moment Chrome v34 (both desktop Android 4.4+ versions) and IE 11 on Windows 8 are the best browsers supporting MSE.

Safari on iOS does not support MSE.

Maybe I need an Android tablet.

Ref:
HTML5 Video at Netflix | Netflix

The Current State of Media Source Extensions | Wowza

The State of HTML5 Video | JW Player

How to Build a Streaming HTML5 Video Player | streamingmedia.com

PS: My CDN demo site demo.cheungwaikin.com will be under re-construction and not available between 26-Feb and 11-Mar.


Browser Compatibility. Flash Video with HTML5 Fallback.

HLS is not supported by Firefox on OSX

HLS is not supported by Firefox on OSX

H.263, H.264, H.265, VP8, VP9, .flv, .mpv, .ogg, .webm, .m3u8, …

I know what you’re thinking:

“What the hack are all these acronyms?!”

It is good to learn some acronyms, here is a great Video File Format Overview.

 

My demo site demo.cheungwaikin.com was launched about two years ago in Nov 2012. The objective was to show how to use EdgeCast services to increase web performance. The demo site is for myself and my customers. It is my pleasure that there are people find it useful too. I used Mac and iPhone to develop the demos. I tested if the demos were compatible on OSX and iOS. That’s all. I didn’t spend much effort in cross-platform compatibility, until recently. I received compatibility questions from visitors and they told me that my site did not render correctly on IE. I have a coming project on 13-Nov-2014 which requires cross-platform browser support. Now is the time to spend more effort on compatibility.

From Google Analytics I know that most of the demo site visitors are PC desktop users and using Chrome. From NetMarketShare, IE 11 is the number one in desktop browser popularity. Large portion of  mobile users are on iOS. In order to help most of my demo site visitors to see the site correctly, I think I have to support IE 11 and Chrome on Windows 7, Chrome/Safari/Firefox on OS X, as well Safari on iOS 8.

google analytics

Users demographics from the app Analytiks and Analytics To Go.

 

6 browsers. And a lot of work! They support different content formats. For example, regarding videos, Safari on OSX and iOS support mp4 and m3u8 while IE/Windows, Chrome/OSX and Firefox/OSX do not. In order to have my video demos work on these 6 browsers, they have to support Flash video for desktop users, .m3u8/.mp4/.webm for mobile users. Or in more professional wordings, I need a Flash Video with HTML5 Fallback strategy to cater my user demographics. Assuming desktop user is more probable, my site will try to send Flash video content via RTMP first, then send .m3u8 (for live streaming), .mp4 or .webm (for VOD) if Flash video is not support by user device.

I spend me some efforts to put up the two new demos with Flash Video with HTML5 Fallback strategy:
#28 CDN video: Flash Video with HTML5 Fallback for VOD
#29 CDN video: Flash Video with HTML5 Fallback for Live Video

It is cool that I check these two demos work on IE 11/Windows 7, Safari/Chrome/Firefox on OS X 10.9 and 10.10, as well as iOS 8! My demos are now much more user-friendly.

flash-not-support

 

For my mid-Nov project, software part is ready. I need to prepare the hardware portion. Let’s talk hardware next time!

App mentioned:
Analytiks
Analytics To Go

Other recommendations:
Security Now #477 podcast: Payment Tokenization – how Apple Pay works in details technically
Exponent #023 podcast: Apple Pay – how Apple Pay and payment industry works, commercially
For Mac Eyes Only podcast: Getting a Charge Outta Your Mac – how to maintain MacBook battery


Which CDNs they are using?

Which CDNs are your favourite websites using?

Based on HTTP Archive, on an average webpage we have today:
-1855KB of content size
-97 HTTP requests/objects
-17 domains

As webpage size and number of objects increase, we have longer waterfall diagrams. By studying the waterfall diagram, we can know what are the domains and where are there servers, and then we can figure out which CDNs the website is using (or no CDN).

CDN Planet – CDN Finder is a great tool which makes life easier. Just input the URL then it will check/guess which CDN a webpage is using.

Take http://www.gov.hk as an example, CDN Finder says: “We believe the site http://www.gov.hk is using EdgeCast as a CDN.”

Use this tool to check the Alexa top 25 websites in Hong Kong, Taiwan and US. Then we have the marketshare from six CDN providers of the Alexa top 25 websites in these three markets:

HK TW US
Akamai 50% 36% 36%
EdgeCast 6% 18% 18%
Limelight 6% 0% 5%
Level3 0% 0% 5%
Cloudfront 6% 0% 9%
Cloudflare 6% 18% 5%
others 25% 27% 23%

However, this table is just a rough estimation only.

 

Why?

Since more and more websites adopt multi-CDN strategy to improve performance, sometime we should review the results in details to figure out which CDNs are being used.

Take http://www.pixnet.net as an example. It is the most popular social networking site in Taiwan. Alexa ranking #4 in Taiwan and #11 in Hong Kong respectively. CDN Finder says: “We believe the site http://www.pixnet.net is using Akamai as a CDN.”

Let’s dig into the details and verify the statement.

The following table is generate based on the results from CDN Finder and Webpagetest:

Count Hostname CDN Bytes
36 mainpage.pixfs.net Akamai 101662
27 imageproxy.pimg.tw Akamai 586353
13 s4.pimg.tw EdgeCast 265490
11 s5.pimg.tw EdgeCast 513516
10 s8.pimg.tw EdgeCast 272489
10 s2.pimg.tw EdgeCast 426017
9 counter.pixplug.in 2340
9 s7.pimg.tw EdgeCast 93922
8 s9.pimg.tw EdgeCast 308673
7 s6.pimg.tw EdgeCast 175496
7 af.pixnet.net 105341
7 s1.pimg.tw EdgeCast 123025
6 stats.g.doubleclick.net Google 19384
6 s.pixfs.net Akamai 13205
4 s.pimg.tw EdgeCast 170437
4 libs.pixfs.net Akamai 18792
3 pic.pimg.tw EdgeCast 236873
2 api.pixnet.cc 1885
2 s3.pimg.tw EdgeCast 104232
2 visitor.pixplug.in 865
1 s.pixanalytics.com Akamai 785
1 ajax.googleapis.com Google 33907
1 d31qbv1cthcecs.cloudfront.net Amazon Cloudfront 1879
1 http://www.pixnet.com 134914
1 i18n.pixfs.net Akamai 3170
1 cdn.mxpnl.com Akamai 10039
1 api.mixpanel.com 411
1 ad.pixnet.net 4691
1 pixanalytics.com 851
1 d5nxst8fruw4z.cloudfront.net Amazon Cloudfront 528

The webpage size is 3644KB, with 193 HTTP objects from 29 domains. The table is summarised into this:

Count CDN Bytes Bytes % of Total
76 Akamai 734,006 20%
84 EdgeCast 2,690,170 72%
7 Google 53,291 1%
2 Cloudfront 2,407 0%
24 251,298 7%

The second table says: “We FIND the site http://www.pixnet.net is using both Akamai and EdgeCast as CDNs, with EdgeCast carries most of its traffic.

 

How to use tools is more important than what are the tools.