Sony’s clever new AI brain makes live production easier than ever

Sony's new AI aims to make realtime pro AV video production easy for people with limited resources and training.

In schools, large businesses and government offices, video has become an essential communications tool. Indeed, the professional AV industry is several times larger than that of broadcast with non-media and entertainment uses of video tech outpacing that of film and TV. The difference, of course, is that video production is what film and TV producers do but the skills are not core in other markets where the responsibility for installing, managing and maintaining kit falls to under-resourced AV and IT departments.

NewTek, Blackmagic Design and others have come up with generally inexpensive solutions for this that make a virtue of being relatively easy to use with little training while delivering professional quality video.

Sony has gone a step further, introducing AI into the mix with a new solution targeted expressly at the budget end of corporate and educational video production.

The product, which debuts at Europe’s leading proAV exhibition ISE next week, is robotically titled REA-C1000 and is built to allows users to create video in real time, without the need for specialised training, additional staff or equipment.

The base model uses motion and face detection and colour and shape to analyse the input it receives from cameras then automatically extracts the object in focus to combine it with other images in real time. Effectively this makes the REA-C1000 the brain connected to any camera and AV setup.

What is particularly interesting about the product, which is being sold later this spring, are its range of software add-ons which have been designed with the Pro AV market in mind.

So, for example, there is an option to extract handwritten objects from black and white boards using a claimed ‘unique’ moving/unmoving object detection algorithm. This, says Sony, allows audiences to clearly see written content and diagrams, while simultaneously being able to see the gestures and expressions of the presenter – useful for lecture theatres or remote learning where students aren't physically in the same room as their teacher.

There’s a chroma keyless function that can extract the presenter and overlay them onto any background without the need for a dedicated green screen. The unit can be set to smoothly track a presenter so that they constantly stay in frame and in focus throughout a presentation.

Another optional function, Focus Area Cropping, allows an image from one camera to be simultaneously shown as two different output images. Users are able to get an overview image in 4K and a cropped image in HD. The cropped area is selectable to be static or dynamic to follow the speaker’s movement.

The REA-C1000 can also detect and react to the gestures of audience members, showing both the whole image and zoomed image in real time. When an audience member stands up, the ‘brain’ can trigger the camera to electronically auto-zoom in on that person -perhaps to measure students’ reactions or cover audience Q&A sessions.

The unit will connect directly to Sony’s PTZ cameras and also its XDCAM camcorders and via an HDMI connection it can be integrated with AV setups involving multiple displays, large screen projections with monitors, or even live-streaming projects.

Presumably the recorded output will still need to be edited and packaged but it could make content like corporate videos, webinars and e-learning materials more appealing and even hassle-free.