RED v Nikon, and a brief history of lawsuits

Here we go again. The subject of raw recording in-camera is in the news again due to RED taking action against Nikon. What is it about, and how did we get here?

The subject of patents over the recording of compressed raw formats in-camera has been a thorny issue for some years now, and it's now abundantly clear that the conflict hasn't been settled yet.

Some years ago there was a start-up camera company, let's call it RED, which developed a visually lossless raw compression system, the patent details of which can be read here. In short, the summary of the patent is worded as follows.

"Embodiments provide a video camera that can be configured to highly compress video data in a visually lossless manner. The camera can be configured to transform blue and red image data in a manner that enhances the compressibility of the data. The data can then be compressed and stored in this form. This allows a user to reconstruct the red and blue data to obtain the original raw data for a modified version of the original raw data that is visually lossless when demosacied. Additionally, the data can be processed in a manner in which the green image elements are demosaiced first and then the red and blue elements are reconstructed based on values of the demosaiced green image elements."

The patent refers to compressing the raw sensor data from a camera in a recording device (not just the camera itself). RED was one of the first companies to allow the recording of compressed raw video in-camera, making post production with what was traditionally a quite awkward format, much more usable. Remember, this was back in 2007 when DV, HDV, HDCAM, and Digibeta still very much ruled the roost and we were still arguing over whether HD would become standard. Most camera manufacturers at the time had no interest in producing 'affordable' large sensor 'cinema cameras' like we see across the board now. Apart from Silicon Imaging, with its 2/3" based camera and CineForm RAW codec, famously used to shoot parts of Slumdog Millionaire, RED was on its own.

REDCODE, as the company's codec is known, was developed by Graeme Nattress, and uses wavelet compression to achieve its high performance with minimal effect on the image. Wavelet compression wasn't new, even back then. But RED's patent isn't really about compressing the video signal in the traditional sense, but in the precise way it captures and compresses the raw RGB sensor data before allowing reconstruction of the signal and de-bayering (demosaicing) once it's inside the NLE. In other words REDCODE doesn't do any de-bayering inside the camera. RED says that, depending on the compression ratio chosen, REDCODE can either be 'visually lossless' or 'mathematically lossless'.

This is in contrast to a codec such as H.265, which will require a full demosaic process to happen in-camera and then inter-frame compression applied, thus eliminating all traces of the original sensor data.

It doesn't take much to understand that any alternative raw recording system that wishes to compress things in a visually lossless way will also have to find a way to store the RGB data from the sensor in an efficient manner. It's doing this that is the contentious part of the patent dispute.

Previous litigation

RED has taken action against other companies in the past for infringing on the patent. In 2013, RED took action against Sony, but was countersued for other technology patent infringements that Sony claimed it owned. In the end the two companies had the case struck off, and an agreement was reached. If you've ever wondered why you need a bulky additional recorder attached to the back of Sony camcorders in order to record raw video, it's a direct result of this dispute. In 2019, Apple tried to have RED's patent disregarded because it said that the methods used were obvious and not unique, and that RED's description of demosaicing was far too vague to form the basis of a patent. However, ironically, Apple lost due to the judge stating that its case was "unclear and incomplete." So much for that, then.

RED has in the past also filed against Nokia and Kinefinity. However, RED dismissed the filing against Kinefinity without prejudice in late August 2021, and cameras like the MAVO are now 'rawless', maxing out instead with ProRes4444/XQ.

Whilst this might have appeared to have been the end of things, with companies such as Apple and Atomos paying royalties to feature ProRes RAW in their products, it seems that the saga is not over yet.

Nikon's Z9 mirrorless camera, as anyone who hasn't been living under a rock knows, can now record both ProRes RAW and the company's own N-RAW formats. Nikon's raw codec is based upon TicoRaw, which, when you read the description of how it works, would appear to be a red rag to a bull. The question on pretty much everyone's lips is, what is Nikon's strategy here?

Kinefiity's MAVO edge 8K camera.

Red rag to a bull

Nikon will be very well aware of the Sony and Apple cases. It might be that it considers that none of the previous cases have had cut and dried judgements made, with most of them resulting in voluntary case dismissal on the part of RED or both parties, and Apple's loss being a result of not building a convincing enough challenge.

That's not to say Apple could ever have satisfied the judge of its case against RED's patents, but the fact that there has not yet been any slam-dunk court decision on the sole strength of the patents means that Nikon could well fancy its chances.

RED's case against Nikon is strongly worded. It states that Nikon's use of in-camera raw is, "likely to cause irreparable harm to RED, which cannot be adequately compensated by money damages, lost sales and profits, reduced business, and injury to its general reputation and industry standing."

Whilst it will be extremely interesting to see where this goes, I think that it is likely to end up in a settlement, much like previous cases. However, if Nikon decides that it is going to go all out to challenge the validity of the patents this is going to be very interesting to watch indeed.

It will also be interesting to see what happens with Tico Raw in general. If RED wins this battle, Tico Raw will face a huge uphill battle to gain acceptance. Without the option of implementing the codec inside cameras, there's much more limited scope for it.