Gaussian Splatting: How Proof Visualized 'Jurassic World Rebirth' (Part 2 in a Series)
While exploring the potential of Gaussian splats (explained in Part 1 of this series) visualization company Proof had an Aha moment.
“As a visualization company, one of our biggest challenges is location work, because we’re usually on during preproduction,” Proof founder and president Ron Frankel relates. “Locations aren’t necessarily locked and they’re not going to send us out and do a full on LIDAR scan, and frankly, we don’t want all that data anyway, because it’s too big and heavy for us to use [at that stage]. So we end up kind of kludging together aerial photography, maybe some Google Maps, Google Earth, just to try to get some rough approximation of what this location is going to look like.”
He notes that Gaussian splats appear fuzzy on these simulations, but for previs, “it doesn’t have to look final. It just has to look like the location, and they gave us a way of creating a more realistic looking, scaled environments. …. and we got it cheaper, better and faster.”
Proof got involved with Gareth Edwards-directed Jurassic World Rebirth for Universal during prepro, working with overall VFX supervisor David Vickery of lead VFX studio ILM. “We had the script, so we knew a lot of the film was going to be set on water [and] we needed to hit the ground running, because we had a really short preproduction and production period and postproduction period on the film,” says Proof partner and creative director Pawl Fulker.
Location scouting began on several beaches in Southeast Asia and Proof created simulations of these environments, involving Gaussian splats, which appear as fuzzy areas in the below example. “We would never be able to get this level of accuracy if we were just kind of going off of Google Maps and Google Earth,” Fulker says. “But this is a photoreal representation of the beach with all of its rocks and all of its bushes and trees.”
Frankel elaborates on the above example, explaining, “the boat is a 3D asset, but because the Gaussian splat has three dimensions, we’re able to embed it in the environment.”
Previs work continued at London’s Elstree Studios, where the movie’s VFX-savvy director was able to operate a virtual camera in these environments to frame the shots (see image below). “We’re running off of Unreal [Engine] using a [HTC] Vive setup. All it needs is four small Vive cameras and our V-cam, so we can set it up literally anywhere,” Fulker reports.
(Images courtesy of Proof)
He goes on to describe the freedom this gave Edwards, saying “he can record camera moves [which is some cases] became the previs. He was also doing, like, snapshots … so he was almost storyboarding it at the same time. He’s having shot ideas on the fly.”
Fulker adds that this supported the collaboration between various departments. “It’s unambiguous. Everyone can see what is there, because it’s effectively real.”
What’s Next
Since the application on Jurassic World Rebirth, Proof has employed these Gaussian splat-supported visualization techniques for additional productions, including Paul Greengrass’ The Lost Bus for Apple Original Films.
For wildfire drama The Lost Bus, Fulker explains, they used Gaussian splatting with drone footage. “Paul was very particular about the path of the fire and it all being completely graphically accurate. So we got some drone footage [to make a splat] and planned out some shots like that.”
Frankel relates that Proof is now looking into the expanded the use of Gaussian splatting in virtualization, revealing “one of the things that we’re actually exploring is making Gaussian splats out of, essentially, AI generated video.”
Currently, the Proof system is used “if you can go film a location, but if you can’t film the location — if it’s a fantasy world — then this isn’t necessarily a technology that has any application,” he explains. “We’re looking at ways in which we can take concept art or concept videos and then turn those into Gaussian splats so those can become three dimensional worlds that we can use for our visualization work — which then opens up a whole new opportunity, because we don’t necessarily have to fully model those worlds.”
Check back tomorrow for Part 3 of this series of Gaussian splatting. Below, Proof shares its suggested best practices for quality splats:
Dos for Quality Splats:
Maintain steady camera movement with consistent overlap between frames
Capture in good nature lighting condition with minutes.
Have something in the scene pop a known size—a meter rule is ideal.
If using a 360 camera have one export forward facing and tilted down 35 degrees and one looking backwards tilted up 20 degrees or so.
Don’ts for Quality Splats:
Avoid rapid camera movements, excessive motion blur and lens flares.
Avoid having moving objects, people or animals in shot whenever possible.
Naturally moving objects like rivers and waterfalls will appear as a blur.