The paradox of compression and how it's brought us better pictures

Compression is bad, right? No, wrong. From enabling the internet to run on 70 year-old technology to bringing us better pictures, its use is essential. David Shapton explains.

Even as a naive ten-year-old, I remember my disappointment when I read about how telephones work. Sure, there were telecommunications satellites and other cool stuff, but that was niche and expensive back then. Regular telephones seemed like a lash-up that I could have put together in my bedroom with a soldering iron and a box of bits. They consisted of a carbon microphone (literally some compressed carbon powder and a metal diaphragm) for the microphone and a similarly simple arrangement for the earpiece. Apply a voltage, and you have a functioning telephone. All very clever compared to not having a telephone at all, but distinctly underwhelming in the context of other technology even at the time.

None of this mattered because it worked, and it worked because it was simple. It kick-started the global telephone system, which still relies on copper cables - some seventy years old or more. Without that primitive copper network, we wouldn't have the internet in our homes today.

Dial a Disk

Given the persistence of this fossilised infrastructure, it's surprising that we can now watch 4K videos and listen to CD-quality audio over the same cables. If you're old enough, you might remember the UK's "Dial a Disk" service, where you could call "160" and hear a record which "will be selected each week from current popularity ratings and a different record will be played each day during the cheap rate period" (text from the UK Post Office's own internal documents).

Dial a Disk was, frankly, terrible. And inexplicably popular. The sound was awful. It makes me wonder whether this is what it would be like today if digital networks had never been invented and we all had to rely on fax machines for visual entertainment.

Most houses still don't have direct fibre connections, and many still rely on the original copper cables. How is this even possible? There are a lot of reasons. First, the entire telephone system is no longer based around a switched analogue voltage: that went away decades ago. But the connection to the home is usually still copper. So how is it possible to get all that bandwidth down such an ancient artefact?

The key of compression

Ultimately the answer is compression. But compressed data is still data. Broadband is an eye-wateringly clever way to get far more data down a single copper cable than you'd ever think possible. It involves hundreds of carrier waves and techniques like phase modulation to squeeze the maximum number of bits into the available "space". And it works - mostly. It's highly dependent on the distances involved. I once lived in a long and thin village where houses at one end could get tens of megabits per second, and I struggled to get two. I now live right next to a roadside junction box and get 350 megabits per second.

Compression typically works by throwing away information you don't need or won't notice if it's discarded. So it's lossy. The original is better. Some types of compression are lossless, but they're typically massively less data-saving than lossy techniques.

But here's the point: it's a lot better than having no compression at all, where, without compression, you'd get no audio or video. It is, literally, better than nothing. But it doesn't end there. Because modern compression is so good, it can be audibly or visually indistinguishable from the original.

In the relentless search for added value - or, to put it in non-marketing-speak - to make more money, some online music services offer "lossless" audio. That's ok. I would pay the extra if I had a $50k music system. But I don't, and nor do most people. So for me, the standard, compressed offering is good enough.

Enabling 4K

But delivery to consumers is only part of the story. The notion that compression can give you better quality even - or especially - extends to professional digital cinema. How was the first 4K cinema camera even possible? Though compression. Here's my interview with Red's colour and codec specialist Graeme Nattress which covers a lot of this ground.

Back then, compression made what was impossible possible. It gave us better pictures than before. Now, with even higher resolutions, compression brings us pictures that we could only have dreamed about a few years ago.

Remember that while compression tends to use similar broad principles, not only do techniques differ in kind and in detail, but it is used for radically different purposes too. Compressing raw video from a digital cinema camera has a dramatically different set of priorities to compressing video for streaming, satellite or digital terrestrial delivery. With cinema cameras, the emphasis is always quality: if your compression doesn't deliver a good enough picture, buy more storage and choose a lower compression ratio. But for delivery to the living room, quality has to fight with cost and bandwidth. Even if you can afford high data rates for the best possible picture, you might have to compete with hundreds of other channels going down the same pipe.

None of this changes the basic equation, which is that compression is an enabling technology. It has brought us better pictures, and better audio, than would have been possible without it.