Shooting background plates for virtual production

How VFX Supervisor Sean Varney uses the URSA Mini Pro 12K to create content large enough to overfill LED walls (and why that's a good idea).

As you might well have guessed this is another manufacturer-supplied case study, but it’s also a pretty good one. That’s due in part a) to its subject matter and the fact that in this current stage of its evolution virtual production is an entertaining sandpit for filmmakers to experiment and learn the best techniques to create content using LED Walls and Volumes.

And b) it’s because of who it is who’s talking. VFX Supervisor Sean Varney’s credits include Doctor Who (2023), His Dark Materials (2018), Children of Men (2006), AVP: Alien vs. Predator (2004) and Sherlock Holmes (2009). That’s a decent list. And while he can’t namecheck the show he’s been doing the work with the URSA Mini Pro 12K on just yet, the bets are good that it will be of that ilk.

Sean Varney: "The scope of the 12K image has additional benefits on set."

So, the goal is to shoot large format plates with high dynamic range, then work to find the move and 4K plate within them. And together with Blackmagic and lens manufacturer Zeiss, Varney tested the URSA Mini Pro 12K for this purpose on a 3K x 12K Wall ahead of using it in anger for the virtual production of the new series.

“Zeiss is developing a range of very clear Primes specifically for shooting super clean and sharp plates for LED Walls and Volumes,” he says. “Alongside Blackmagic, Zeiss understands the need to reproduce reality on the wall for back projection-like scenarios. Therefore, plate capture has to be as sharp and as big as possible and devoid of in-camera aberration in order to give the DoP the clearest canvas on which to apply the look they want from their principal camera package. The last thing you want is to introduce aberrations into the process from the plate.”

More lighting required

The problem is that while LED Walls emit considerable light, they cannot be relied on solely to light the whole scene. Illumination from the Wall must be used in-conjunction with a practically lit foreground space, but this can also be a partial set extended by the Volume. 

“At the same time as shooting background plates you can use the URSA 12K to shoot HDR images for lighting in BRAW,” says Varney. “You can capture references so your DP can see where key lights and environment lights are in order best match lighting with objects on stage sets in the foreground.

“In order for the whole of the expanded 12K image to be used as interactive lighting reference, I’d advise pairing the URSA 12K with a wide [15mm, 21mm] lens when shooting plates,” he continues. “You will likely only want to use the 4K frame in the centre of the image to be the actual back projection that is lit to the actors but the 4K on each side of your frustum can be wrapped around the rest of the Wall to give you an interactive lighting set up. 

“Because you have 14 stops of original range from the Blackmagic RAW we can bake it out as a play and then put it through the media centre to match/adapt it to suit the lighting variables that the DOP wants.”

At the VP stage, operators at the unnamed media centre used DaVinci Resolve to adapt lighting and colour of the background footage before putting it on the Wall. Blackmagic’s colour pipe and editing environment is integrally linked to Blackmagic RAW and was developed alongside the URSA 12K, which Varney says means it’s a much easier process to get video out to the wall; you just stay within the BMD environment.

The URSA Mini Pro 12K: big pictures allow you to easily extract a 4K frame

Additional benefits

“The scope of the 12K image has additional benefits on set,” he says. “If the director asks if can you shift the plate to the left then you’ve got loads of room to do so. It’s like shooting an extreme version of open gate. You can move it left, right, up or down until you get that horizon line right. When I shot plates on location I would shoot them at three different heights which gave me a variant horizon so when I get to the stage we’re in a good place.

“The magic extra that URSA 12K gives a VFX team when capturing plates is the use of extreme high resolution for image manipulation in editorial,” he adds. “The same plates can be used to generate punch-in shots, pans across the frame, adding shake and using tracking markers well outside of guide frames. If you need to composite in multiple elements within a frame you can use the locked-off 12K plate and extract another shot. That is really useful.”

Of course working with 12K images brings its own issues. But Varney and his team record to very fast SSD cards and with optimised playback for Apple M1 and M2 workstations he says it is “very easy to process 12K. It’s painless to transfer Blackmagic RAW 12K into an ACES colour space along with RAW footage from an ARRI or a RED, so you can mix and match.”

One other thing worth considering as well. “What the URSA 12K also gives you is the opportunity to use a smaller and more affordable Volume because the plates are of such high quality they can be used to extend your shooting environment,” concludes Varney. “It is a hugely useful technique because you are able to capture a single plate and use the entirety of the Volume.”