We need to pay more attention to latency

Some nice new firmware further reduces latency in Accsoon’s SeeMo cellphone and tablet monitoring device - but the rest of the industry needs to look at lag, too.

Earlier this month, Accsoon released a firmware update for the SeeMo cellphone monitoring device which is intended to improve its latency. We looked at this nifty gadget a while ago, and this is a very welcome development in a world where manufacturers sometimes don’t pay the sort of attention to delay and latency that they really should. 

Laggy pictures are very much a malady of the modern world, if only because the pioneers of electronic moving pictures didn’t have the technology to introduce delays even if they’d wanted to. In the earliest television cameras, the time between a photon hitting the front of the lens and another photon coming out of the monitor could be very short indeed. Okay, so Philo Farnsworth’s first cameras could only detect photons that hit the pickup tube at the exact moment the beam scanned across that part of the tube. That wasn’t particularly great, because the resulting camera was staggeringly insensitive. Still, even once we’d moved on to tubes with enough storage to accumulate light over the period of a frame, early cameras were very fast.

The time taken for an electron to move down a vacuum tube is very short. So is the time taken for a transistor to process a signal – often a few nanoseconds, where a nanosecond is equal to the time an electrical signal would take to travel about six inches of copper wire. Okay, analogue TVs often used ultrasonic delay lines, but that’s one video line’s worth. That’s 64 microseconds (64,000 nanoseconds) in PAL, and even then its purpose was not to delay the video overall, but to aid in colour decoding. For comparison, one PAL frame’s worth of delay would be twenty thousand microseconds – and yet it’s disturbingly common for modern equipment, both displays and cameras, to involve delays of several frames.

Buffers and codecs

Often, those delays are built into the processing logic of cameras and displays, which seem to wait for an entire frame to arrive before beginning to do mathematics on it. That’s not ideal, and nor is it strictly, technically inevitable, but working in whole frames presumably makes firmware engineering quite a lot easier. It is at least possible to address that with better code. Blackmagic, to its huge credit, did some very useful work on the Ursa Mini line some time ago which massively improved its viewfinder latency, much as Accsoon has apparently done with SeeMo.

The other source of delay is in video compression, where codecs like H.264 and its derivatives must look at more than one frame of video in order to analyse the changes between them. That imposes certain minimum bounds on the amount of delay involved, which is why certain types of video transmitter, for instance, find it hard to reduce delay below a few frames.

The SeeMo device works in a somewhat similar way, sending compressed video to an Android or iOS device over USB. People have mused about using phones and tablets as monitors ever since someone first noticed that modern cellphones often have incredibly nice, daylight-readable displays. Those TFT-LCD panels developed using the vast development budgets of the cellphone industry, and they’re much, much nicer than anything the film industry could come up with on its own. The only way to get video into one, however, is via either wireless or USB connections, which will necessarily involve some compression.

As fast as tethered

Video released by the company (below) suggests that SeeMo is now at least as fast as what looks like a Blackmagic Video Assist, which is used as the HDMI-attached comparator. It’s worth being clear about some of the crucial fundamentals here: things like frame rate matter, because if any compression codec is working on (say) groups of pictures n frames long, then n frames is less real wall-clock time at 60 frames per second than it is at 25. Sure enough, latency performance of both displays seems better at 60fps than it does at 25, by a bit more than twice – who’d have thought! – but what’s good is that the SeeMo seems to keep pace with the hardwired display very nicely.

One crucial thing we just don’t know (though we have asked) is what sort of group-of-pictures length the SeeMo is using. The only purpose of compressing the video is to get a picture across a fairly high bandwidth USB connection. As such it would make a lot of sense to compromise compression efficiency (not necessarily image quality, but quality-to-bitrate ratio, since we have a whole USB connection to ourselves) to reduce latency. That might allow the design to use a very limited group of pictures, or even an I-frame mode. It would still be possible to leverage the device’s H.264 decoding silicon. Possibly all that is exactly what’s being done here. We are necessarily speculating.

In the end, though, it’s tricky to do anything else, because we don’t know fast the Blackmagic display is. Your correspondent owns one, but given cameras are often laggy themselves, it’s hard to isolate the delay to a particular piece of equipment without advanced SDI test gear. In the company video, the Accsoon certainly keep up. Faster is better, so it’s good to see it, but it is hard to be specific.

People used to the absolutely cast-in-stone immediacy of an optical viewfinder on film, or even classic analogue cameras, might draw something else from all this. Regardless the performance of any particular piece of equipment in the signal chain, the absolute glass to glass delay, the time between a photon hitting the lens and one coming out of the monitor, is at least two tenths of a second at 25fps. That’s more than four frames. Let’s be absolutely crystal clear that this is not something Accsoon can do anything about, in general – this is a combination of monitoring and camera delays. Even the viewfinder on the Sony A74 is a tenth, or two frames, behind reality.

So, the conclusion from this is a heartfelt plea to the industry in general that if everyone paid as much attention to latency as Accsoon seems to have done, the world would be a better place.