Measuring Web Latency and Rendering Performance: Method, Tools & Longitudinal Dataset

This paper presents Webget, a measurement tool that measures web Quality of Service (QoS) metrics including the DNS lookup time, time to first byte (TTFB) and the download time. Webget also captures web complexity metrics such as the number and the size of objects that make up the website. We deploy the Webget test to measure the web performance of Google, YouTube, and Facebook from 182 SamKnows probes. Using a 3.5-year-long (Jan 2014 -Jul 2017) dataset, we show that the DNS lookup time of these popular Content Delivery Networks (CDNs) and the download time of Google have improved over time. We also show that the TTFB towards Facebook exhibits worse performance than the Google CDN. Moreover, we show that the number and the size of objects are not the only factors that affect the web download time. We observe that these webpages perform differently across regions and service providers. We also developed a web measurement system, WePR (Web Performance and Rendering) that measures the same web QoS and complexity metrics as Webget, but it also captures the web Quality of Experience (QoE) metrics such as rendering time. WePR has a distributed architecture where the component that measures the web QoS and complexity metrics is deployed on the SamKnows probe, while the rendering time is calculated on a central server. We measured the rendering performance of four websites. We show that in 80% of the cases, the rendering time of the websites is faster than the downloading time. The source code of the WePR system and the dataset is made publicly available.