Interview with Cart and Horse About Virtual Production and Unreal

An incredible studio named Cart and Horse recently came out with a project highlighting something I didn’t know much about: virtual production. If you don’t know much about it either, this article will give you a ton of insight and behind the scenes info on virtual production, why it’s so useful and interesting, and how the technology is quickly evolving.

Virtual production with apps like Unreal is a cutting edge tool with incredible possibilities. Let’s get introduced to Cart and Horse and learn more about it. But first, check out their video so you can see what we will be discussing!

Give me a quick overview of Cart and Horse. Who are you and what do you do?

Cart and Horse is composed of Dan Maurer (myself) and Kris Bargen. We started the small studio to focus on 3D animation and Virtual Production.

We formed the studio together a couple of years ago based on our common love for 3D motion design. I came from a background in the digital product/interactive world using game engines (Unity/Unreal) to do work alongside 3D animation and some lite “run-and-gun” video production.

Kris has an in-depth background in video production as well as animation and content creation for marketing. 3D motion design and animation are where we met in the middle and put our brains together.

Cart and Horse Virtual Production Unreal Engine Interview

What is virtual production and why are you so excited about it?

Virtual production is a pretty broad term for using CG in filmmaking. The area we have been focusing on is how virtual production is used in live-action filmmaking with giant LED walls as the background.

It is another version of VFX, however, instead of the shots being filmed on a greenscreen, and all of the CG work being done in post, the shots are created and planned in preproduction and then filmed live on a large LED Volume. The motto that has been coming out of this is “fix it in pre.”

Why is virtual production so useful?

Virtual production is by no means a silver bullet for filmmaking, however, it does have many advantages that make it very useful.

One of the biggest advantages is the ability to plan shots and work on the CG aspect in preproduction. To expand on that, we (the virtual art department) work with the directors, the art department and the director of photography (DP) throughout the planning process.

This allows them to see and react to the shots in real time, which reduces most of the guesswork and helps problem-solve before anything even gets to the set. This process and workflow can be pretty intense, but it is super helpful.

Another reason that this method has been so useful is that it is shot on a giant LED wall. You get the benefits of shooting in a studio (light, temp, environment control) and at the same time the flexibility to “shoot anywhere”.

There are some limitations to shooting on a screen, like moire and being confined to the size of the wall, which can create some limits as to how the shots need to be composed. On the flip side, the amount of locations and environments that can be shot in is limitless.

In addition to having endless environments, you also get to use the lighting from the environment in your shot. Adding animation and using the movement in the background goes a long way, but not having to composite environment lighting or reflections that you don’t get from a greenscreen significantly helps add depth and realism to the shot.

Tell us about this specific project. Why did you create it? What was your goal for the project?

There were several goals that we aimed to achieve by doing this project. Since this technology is still new for this industry, there is an education gap that still needs to be bridged.

The first and biggest goal was to help showcase and explain how this technology can be useful outside of a large film production or locked-down commercial shoot.

This project allowed us to tell the story of virtual production, while allowing artist to flex their creative muscles and push their work into Unreal Engine. In the end, we created 8 different environments and tied them together throughout the story which were all shot in one studio over 2 days.

Tell us about the general process of the project and the workflow you used.

Collaboration is key. We worked with entire team in a collaborative workspace. It was typical video production for planning and storyboarding and then we assigned environments to artists and started building with a few rounds of reviews.

Then we compiled everything together and made composition, camera and lighting tweaks with director and DP. Then it was shipped off to studio (Golden Hour) where we then worked to resolve technical issues on the wall and prep for filming (Dan).

Physical props were made at the same time (Kris) and then tested in camera to confirm before the day of shooting. From there, shots were captured and footage was graded live on set to be sent to editing.

What software was involved in this project, and for what parts?

Unreal Engine is the big one. Cinema 4D was used to create assets. We also used external assets to move quickly like those from The Pixel Lab. Any other tools (Photoshop, Ae) were used to create assets and get them in the engine.

What was the most challenging part of the project?

Wearing so many hats (producing, art department/prop making, talent) and designing the environments/shots to work with the director, the director of photography, and the tech limitations of the wall with Unreal. The collaborative part of this process was a new exercise as well.

We had defined roles, but little hierarchy of authority, so everyone could voice ideas and problem-solve. This presented new challenges of decision-making, responsibility, and communication that are normally straightforward.

But it was overall a fantastic experience and proved that being more collaborative is generally a huge addition to a project.

We also ran into unexpected issues on the tech side of things as well. Working with Ndisplay, running nodes, and everything about getting all the tech to talk together is a new variable.

Not everything that we put together and planned out in Unreal translates to the tech on the wall. We had to continue to optimize and troubleshoot things both while testing and even on the fly while filming.

Everything is so new and updates are constantly making things better and broken at the same time. Overall it’s a constant balance of what we set out to do, how to optimize it, and then deal with unexpected things as they come up.

What was the most rewarding part of the project?

Seeing all of the hard work, dedication, skills, and effort put in from the entire team coming together in camera on the set.

It is really cool seeing a final graded image on the monitors, as the camera is moving and filming. Arguably the best part are the relationships with people. We had so much fun.

This project gave us a reason to keep in touch over the past few months, meet new amazing people, and stay connected on social media going forward. We really hope this is the beginning of many more projects together.

Tell us a bit about how you use the material pack from The Pixel Lab in this project.

This pack launched as we were getting late in the game on this project. The timing was perfect. We compiled all of the scenes into one project for polish and final direction.

The mutating metals and mutating woods allowed us to drag, drop and iterate fast in the final hours of polishing. The VDB clouds have also been helpful for adding in clouds or quick volumetric lighting effects to help fill in shots.

Since Unreal is real-time, we were able to iterate with the material instances on the fly to match colors, change patterns, and see options live on set.

What are you most excited for in the future of 3D animation/design?

It has been great to use a variety of skills from different industries over the years and have them be useful for a variety of outputs beyond rendered animations. It has brought on a lot of new challenges, but that has also opened up new opportunities for work and fun creative challenges.

There are so many big things happening so quickly it is honestly hard to keep up but we really have seen how it is building on itself.

We are super curious about the future and how long until another major change. Different tools, techniques and processes continue to overlap and integrate with each other. We know that the learning we are gaining by continuing to push ourselves will be fruitful.

Is AI going to take our jobs?

No, not “our jobs.” But like all technology it will change the way things are done. And we will need to adapt with change. Just like GPU rendering opened up many new possibilities at smaller scales for 3D work, AI will do the same for individuals in different areas.

It will make wearing multiple hats much less painful, which will help open up more opportunities.