Why CDNs are trying to steer away from discussing video performance metrics.
If it can be measured, it can be optimized; that’s why practically all sites use Google Analytics to gain insights that help optimize the website’s performance, resulting in a more effective website to which customers keep coming back.
In contrast, try to ask video publishers about their video delivery performance -- they have all the analytics about their page loads, but are mostly in the dark when it comes to their users’ viewing metrics such as start times, buffering, and seek times -- although it’s perfectly clear that better video performance brings better engagement. How come?
Traditional CDNs Cannot Provide User Experience Metrics
Google analytics does not measure video performance; typically, it’s the CDN’s job to do that. But traditional CDNs, whose architecture contains only a server-side component, can only provide server-related data, namely: when the server has been hit with a request for a video and when the requested packages were sent to the client. Such CDNs are ignorant of any events taking place on the client side, such as the user pressing “play” or moving the video to a certain playback position (seeking). Even more important: they don’t have a clue about the user experience involved in these events; for example, the amount of buffering and the video start time.
Questionable relevancy:Akamai Luna Monitor Report displaying edge hits, namely: the percent of requests that were handled by CDN server and did not have to go all the way to the origin. These statistics provide no data on the real client-side user experience. [Taken from https://www.akamai.com/us/en/solutions/intelligent-platform/control-center/interactive-monitoring-and-reporting.jsp, and http://serverfault.com/questions/207077/akamai-vs-smaller-cdn-for-small-medium-size-ecommerce-traffic-caching-latency]
User Experience Revealed through Client-side Metrics
Client-side user experience can be collected only when using a 3rd generation CDN such as Google’s YouTube service, which uses client-side code. This allows YouTube to provide in-depth data about the user experience of each video uploaded to the site; the bad news is that this proprietary Google technology is only available for videos uploaded to YouTube website.
So if using YouTube’s CDN tools is not an option, how can a video publisher get user experience insights? By using a commercial 3rd generation video CDN.
One such CDN is HolaCDN that, just like YouTube, uses its combination of server-side and client-side components to both accelerate video delivery and collect video streaming events as experienced by the specific user.
Optimizable data: HolaCDN’s viewing experience analytics dashboard displays actual user experience, client-side metrics such as buffering time and seek time.
Traditional CDNs would Rather Focus on Irrelevant Figures
Traditional players in the CDN market prefer to steer the discussion away from user experience to irrelevant figures such as their number of servers, % served from the edge, and the amount of daily traffic delivered -- all of which are not indicators of the user experience, but of factors that may not necessarily affect it.
Imagine yourself sweating in a taxi sent by a big cab company in mid-August; how relevant are the figures proudly cited by the cab company, such as number of cars and years of experience? Wouldn’t you rather use Uber, which has no cabs of its own and is an emerging player, but focuses on user experience - wait time, cost, air condition and so on? On the same token, the traditional CDN’s number of servers can hardly comfort video publishers whose users witness long video start times or high amount of buffering; publishers need to have measurable, optimizable data about their users’ experience.
Getting to See Client-side Experience Metrics on your Video Page
So how can we steer the discussion back to user experience?
To get client-side viewing metrics, get an open source implementation at this github page and follow the instructions for inserting the code to your video page.
Once everything is ready, start measuring your users’ viewing experience. In all probability, you’d be surprised at the results and at the insights gained. Filtering the results by country, browser, OS or ISP will help you get an even clearer picture, and the resulting focused fine-tuning of metrics can help you significantly increase the number of engaged users.