Today at TwitchCon, NVIDIA announced some interesting new tech that can be quite beneficial to the livestreaming community. The company has been leveraging the RT and Tensor cores in its current generation of Turing-based GeForce RTX graphics cards to enable features like real-time ray tracing, but that same hardware will be used to power the company’s new RTX Broadcast Engine.
The RTX Broadcast Engine will leverage the onboard Tensor cores to enable a wide variety of effects in real-time including style filters, augmented reality, and greenscreens. In the case of RTX Greenscreen, NVIDIA is using artificial intelligence to scrub the background in the livestream, leaving just your face and body to be superimposed over game content or any other background that you choose. The big benefit to using a GeForce RTX to enable the virtual greenscreen feature is that no additional hardware is required to setup and maintain, which helps lower the cost of entry for streamers looking to spice up their channel.
Of course, the greenscreen effect looks nearly perfect in NVIDIA's demo above, but we'll have to wait to see how everything pans out in the real world once software is updated to take advantage of the SDK.
RTX AR uses face tracking technology on footage from your web camera to transfer your facial movements (lips, eyes, etc.) onto a 3D model. Think Apple Memojis and you have a general idea of what to expect. Finally, RTX Style can transform the style of your feed based on a style that you choose, although out of all three technologies, this one seems the least "useful" in our eyes. It looks a little gimmicky, but it may prove popular with a segment of the livestreaming audience.
The audience for livestreaming is huge, with NVIDIA saying that 750 million people globally are taking to their streaming platform of choice to watch the world's best gamers duke it out against the competition. The RTX Broadcast Engine definitely has potential, but app developers will have to add support in their software. In the case of RTX Greenscreen, support is already planned for the Twitch-centric OBS. Both StreamLabs and XSplit have also thrown in their support for the RTX Broadcast Engine and the accompanying SDKs.