The 1mm gap: Understanding the technical limitations in automated performance and environmental reporting tools

Written by Marketa Benisek - November 21, 2023

As the digital world continues to evolve, so do the tools and metrics we rely on to understand its impact. At Wholegrain Digital, we’re always aiming to push the boundaries of what’s possible when it comes to creating sustainable websites.

Occasionally, our team stumbles upon a website that seems perfect. One that offers a visual feast with high-res images, interactive elements, autoplaying videos, animations, designer fonts AND a low carbon score. Something doesn’t quite add up. We’re seeing close-to-perfect scores with imperfect methods that defy the rules of digital sustainability.

Is it too good to be true?

Exploring the cause behind these discrepancies not only helps us analyse sites better and improve our processes, but also aligns with our overarching goal to help people understand how we can collectively build a better web.

Let’s delve deeper.

How do site performance tools work?

Understanding the issue and why it’s important needs a little background about how site performance tools work. Typically, these tools simulate user actions by “scrolling” through the page (except they don’t really scroll, they zoom out to grab a screenshot of the page). This process normally captures a comprehensive snapshot of the page’s content. While the intention of the process is to act in the same way as a human user, the reality is a little different.


The simulated “scrolling” by a performance tool counts as one event. However, this doesn’t take into account all user interactions. In the case of the sites that look too good to be true, we found that some assets only load upon another user interaction—like moving the mouse by even 1mm—that often goes undetected by performance tools. We call this ‘the 1mm gap’.

This manoeuvre, whether intentional or a byproduct of design choices, impacts both the user experience and digital carbon footprint.

Man sitting in front of a laptop with his hand on a touchpad

Why does this matter?

The significance of the 1mm gap hit close to home when our team stumbled upon a website that seemingly defied the sustainable web design principles. It showcased a captivating visual experience with large images, interactive elements, animations and stylish fonts, while surprisingly scoring quite well on Website Carbon. This sparked our team’s curiosity and it set us on a journey down the digital rabbit hole.

At first glance, everything seemed great. The site was within the ideal page weight budget, a modest 0.5MB. But something didn’t quite add up. The second we moved the mouse to scroll through the page, all the “hidden” assets, especially videos, started to load, increasing the transfer size to a whopping 18MB. 

These practices also have implications for SEO, as they might marginally boost search rankings. But the problem is that, while these websites may outsmart the performance tools and therefore achieve better results than would be normally expected, they pose a problem to our mission—creating a sustainable web. These websites don’t actually reduce digital carbon emissions. On the contrary, once the user interacts with them (which is what happens as part of a normal user experience), they become incredibly inefficient and highly polluting.

As we collectively strive to make the internet more sustainable, the 1mm gap highlights the need to address nuanced challenges in the ongoing evolution of automated performance tools. Human input is key for understanding the full picture. Experienced web developers can complement automated performance tools, offering valuable insights into whether a site is genuinely sustainable and energy-efficient, or if it requires additional improvement.

Transparency for a greener web

Transparency is key when it comes to reporting on web performance and carbon emissions. Website owners should understand how specific design choices and technical implementations can affect their site’s performance and the planet, and encourage an informed approach to best practices within the digital space. 

Ultimately, any performance tool, including our Website Carbon Calculator, should be seen as a guide, not a definitive measure. They might not catch everything, including the 1mm gap we’ve uncovered. Our goal is not to point fingers, but to build a shared understanding within the web development community in order to build a more sustainable web together.

It’s not just about reporting numbers, it’s about how we build websites.

In today’s digital age, choosing sustainable practices is not just a good idea—it’s a must. We need to do this to prevent the worst impacts of climate change, especially because the digital sector plays a big part in it. Sustainable web design practices not only help the planet, they also create better experiences for users, making the digital world more responsible and ethical for all.

The 1mm gap is a technical limitation in an automated tool’s ability to see the full picture of the tested web page. In some cases, it might be exploiting these technical limitations to achieve better results on key performance and environmental metrics.

However, the bottom line is that rather than trying to manipulate the system, we should be working towards real change in our own practices and mentality across the industry. 

Positive outcomes and change is the result of positive intentions, and when it comes to reducing the impact the digital sector has on the planet, this is something we should all aspire to.

________________________

Big thanks to the team for their valuable contributions to this article, and especially to Josh Stopper, the former tech lead at Wholegrain Digital and founder of Beleaf, for inspiring this post and for his ongoing support in making the web more sustainable.