Star Wars Changed Visual Effects — AI Is Doing It Again | Rob Bredow | TED

The opening shot of "Star Wars: Episode IV", the original one, this is an epic shot. It broke all new grounds and really redefined that era of filmmaking. Plus, it's just like, perfectly designed.

Now behind the scenes, you may or may not know, this was the camera rig. This is the Dykstraflex. It's a computerized, motion-controlled camera rig.

It was innovative. It could be moved from stage to stage, it enabled these new, dynamic camera moves on models and miniatures, for the very first time. Again, redefining that era of filmmaking.

Today, we’re entering a new era of technology. But instead of being introduced to it with a spectacular opening shot of a movie, we're reading about it in the press. And as someone who's been a visual effects artist for over 35 years, some of these headlines are pretty disappointing.

They seem to say "AI is coming, and it's coming for our jobs. " And the tools, they're getting pretty good, too. These photographs from a TED conference, it never happened.

Of course, these are simple text prompts, image generation systems. Look at some of the details it's getting right. Look at the shadows under her chin, It even knows that there’s a red bounce card below, to get that lighting just right.

Her hand notably, exactly five fingers. That used to be hard for AI. If you look at her hand, it's subtly motion blurred, like she's making an important point.

It's getting a lot of details right. But I work on movies, I should be safe for a little while, right? It's going to take them -- Well, you know where this is going.

Google, Meta, OpenAI, all showing sophisticated video generation systems. These appear to be doing some of the same things we are doing right now, frame by frame, with multiple artists often working weeks at a time to make a single 3-to-5-second shot in the movies you love. It's going to be pretty intimidating.

And we're not the only ones who are a little concerned about the future of AI. OK, we've been here before, and I believe this is an interesting, you know, collision, right, of tech and art. And I believe what George Lucas created in Industrial Light and Magic, his visual effects company, ILM, he founded this 50 years ago to solve the visual storytelling challenges in his films.

And what he did there is he put artists side by side with engineers. They were working together to be able to innovate and create the future. This is artist-driven innovation.

I'm passionate about this. So what we're going to do today is we're going to look at 50 years of history at ILM. We're going to pull some of those cinematic game changers that have happened over this period of time.

And we're going to try to draw some universal themes out of that. And in the end, I'm going to premiere for you, for the very first time, a short created by an artist using the latest generative AI tools, so we can see the trajectory we are on and the trajectory we want to be on next. OK, let's dive in here.

Let's turn the clock back to the Jurassic era. This isn't really the Jurassic era, there's real scientists in the room. This is the "Jurassic Park" era, my Jurassic era, in 1993.

On this film, they were going to use as many full-scale dinos on set as possible. Just amazing, amazing movie. I mean, this film really ended up redefining the way visual effects were done in movies.

But it wasn't planned that way from the beginning. They were going to do these huge, full-scale dinosaurs, which they did, and they were amazing. But for the shots where they couldn't have a full-scale dinosaur run like a T-Rex, they were going to do stop-motion animation like you see here.

And all that changed one day when Kathy Kennedy, the show's producer, was walking the halls of ILM and noticed this test looping on a monitor. The animators had actually positioned this monitor, hoping she would see it. This is early days of CG, computer graphics, and this looks promising enough to commission a test with skin and lighting.

So they did that test just real quick. And Dennis Muren, the film's visual effects supervisor and famous stop-motion animator, Phil Tippett, they took this down to Spielberg and to Kathy Kennedy, and they looped it a few times in their screening room. And as the lights came up, they knew, cinema history had changed.

It wasn't going to be the same. Spielberg caught Tippett’s eye, and he could see his expression on his face. He said, "How are you feeling, Phil?

" And Phil said, "I feel like I'm going extinct. " (Laughter) And Spielberg's an amazing filmmaker. He goes, “That’s a good line, I’m going to put that in the movie.

” He gave that line to Dr Grant, as he’s walking up the rotunda. But I think artists right now, many of us, are feeling that same threat of extinction. But there's good news here.

This is not what happened on "Jurassic Park. " Innovation thrives when the old and new technologies are blended together. "Jurassic" broke all new ground in computer graphics.

All new techniques for the first time had been used at this scale. But many of the techniques were exactly the same as before. The animators taped themselves on video for reference.

They drew these amazing storyboards of these shots that are now indelibly imprinted in all of our minds. Engineers at ILM even went so far as to make this. This is the dinosaur input device.

It's a stop-motion armature. As you can see, each of the joints has a little encoder on it so it can send the data back to the computer. So the stop-motion animator can do frame-by-frame animation like they normally would.

The computer would record it, that animation could be sent across the Bay, from Phil Tippett's shop to the artists at ILM, who could put the skin on and light it and render it, even touch up the animation digitally to create the final shots in the film. Literally a combination of the oldest animation technique we know, stop-motion, and the brand new computer graphics techniques back in 1993. This has been true recently with AI technology as well.

This is from "Indiana Jones and the Dial of Destiny. " Here you can see Harrison Ford performing on set in that iconic role he created, but he's looking a little younger for the opening of this film. Our artist hand-tuned generative AI models on past performances of Indiana Jones, of course, with Harrison Ford's permission.

In addition to that, our artists also created a full CG, computer graphics version of his performance. This is CG skin, CG eyes, all the details that go into that. So why did we do it two different ways?

Well, it turns out the AI system is really, really good at getting that likeness, pulling all the details out of Harrison's performance that he could do unencumbered on set. Get that into a likeness and a correct age that we were looking for for this character. It wasn't as good at the details on the controls.

That's when our CG model came to the rescue, and our artists created a blend of the two to create the opening 20 minutes of this movie. And I'd say some of the best work of this kind ever done. And it's not bad if Harrison Ford says, "It's pretty good, it feels like me.

" So that's blending the old and new, leaning on the experienced artists to drive even the brand new techniques. And that takes us to how technology can actually work hand in hand with creative. I believe that we're designed to be creative beings.

It's one of the most important things about us. That's one of the reasons we appreciate and we just love it when we see technology and creativity working together. We see this on the motion control on the original "Star Wars" or on "Jurassic Park" with the CG dinosaurs for the first time.

I think we just love it when we see creativity in action like this. Tech and creative working together. If we fast forward to 2020, we can see the latest real-time virtual production techniques.

This was another creative innovation driven by a filmmaker. In this case, it's Jon Favreau, and he had a vision for a giant Disney+ “Star Wars” series. First time we ever were going to do a series of "Star Wars" on TV.

This is for "The Mandalorian," and it was a big show, but we didn't have a normal film schedule. We had to shoot it all very, very fast. So rather than flying the cast around the world to try to find these exotic locations, we brought locations to them on a single soundstage in Manhattan Beach, where we had configured this LED wall to act as a virtual production background so we could immerse our actors, our entire crew, all of the departments, in these virtual worlds.

And even some locations you just saw that weren't from our world, they're from outside of our world. Interestingly, Kathy Kennedy, the same producer who produced "Jurassic Park," she now runs Lucasfilm, she was one of the people who got behind this idea early, before any of us knew that it was going to work. The technology leading up to this had been created hand in hand with filmmakers over the years.

So this is back on "Rogue One. " The supervisor on that show, John Knoll, this is his iPhone footage. You can see the first time we were using LED lights on set to light the dynamic lighting in a cockpit.

A couple years later, I was supervising "Solo: A Star Wars Story," and we thought, could we take what they had done before and could we make the content high-enough quality so you could point the camera directly at it? And we started making shots like this for the very first time. This is hyperspace in the cockpit of the Millennium Falcon, and you can see my friend, the DP Bradford Young there, and he invented this shot.

We had shot hyperspace tons of times before, but in the moment, he saw the light coming in and he realized this was a moment of hope in the film. So as he pans over to Han Solo, you can see the reflection of hyperspace in Han's eyes. Just a beautiful collaboration between tech and art that elevates the storytelling and gives our artists new tools.

That's what we love. So that catches us up to "The Mandalorian," where, as always, it starts with words on a page, the script. And then our artists break it down, and I’ll break it down for you, how they add all the different elements.

So what you see there is a CG ship, added in post-production, but the rest of the background, that was live on the day on that LED wall for all the departments to work together to create life in that scene and block these shots out. I don't know if you see those walkers in the background, you'll never guess. They were stop-motion animated over at Tippett’s shop.

Yes, the same Phil Tippett, far from extinct, all these years. So it's a combination of the oldest technique in the book, with the latest real-time computer graphics to bring an illusion together on the screen in front of the whole cast and crew. So you're not supposed to show your magic tricks, right?

But I've shown you all the tricks, so now perhaps you can figure out where the set stops and the LED wall begins in this frame. I just held it there. Just maybe draw an imaginary line in your mind.

Here, I'll show you the answer. Hopefully, we fooled you. Our artists work really hard to make it a seamless blend between the two.

And now we can pop these LED stages up anywhere in the world when it's the right creative fit for a filmmaker. OK, so that's blending the old and new, how tech and creativity working hand in hand create things we just love. And last, maybe most importantly, our theme about artist-driven innovation.

So what happens when you put the latest AI tools in the hands of talented artists, both to see how good these tools are these days, and what does it do to our artists' imagination? Where does it take their imagination? So I'm going to premiere for you a brand new short film.

This is just a test, this is not a final product. This was created by one artist. His name is Landis Fields.

He created this in two weeks to explore what it would feel like if you sent a probe droid out to a brand new "Star Wars" planet. What would it see, and more importantly, would this kind of show, if we made this -- and I'm not announcing a new show, this is just a test -- if we made this, what would it feel like? So I'll let you watch a couple minutes.

(Music) (Music ends) ["Star Wars" Field Guide] (Applause) It's pretty fun to see artist expression leveraging the latest new tools. Now this, as I mentioned, it’s just an early work in progress. Those are not final "Star Wars" character designs.

That's not a finished product. For a show, for a movie, we need a team, for so many reasons. We at Lucasfilm and ILM, we don't think we're just around the corner from one or two people making a film in a dark room by themselves.

We benefit so much from the diverse skill set of the artists and the technicians that work on our films. They bring so much creativity to our projects. That's what we want to capitalize on.

What you see here, you can kind of think of it as a mood board, but a moving one. And how much more powerful is it to have a tool like that, to get an early impression of a show as you're deciding what to do next? It is natural for us to think about jobs as these tools continue to evolve and get more powerful.

If you go back to the days of models and miniatures and motion control, there was only a couple dozen people in the whole world who knew how to build and operate these complicated machines. Today, there's literally hundreds of thousands of artists around the world who know the latest CG techniques, the latest computer graphics, visual effects techniques. There's a huge opportunity for creative, talented people with these skills.

And as our tools continue to evolve, we take our responsibility seriously to grow and develop the talent that's working on our projects. This is how we've been doing it for the last 50 years. In fact, some of the artists who worked on the original "Star Wars" series are still working with us at ILM today.

So that's artist-driven innovation. Like you've seen on our productions over the decades, like you just saw with Landis Field's most recent test, that Field Guide. Now the tech companies on their own, they don’t have the whole picture, right?

They're looking at a lot of different opportunities. We're thinking about it from a filmmaking perspective. And storytellers, we need better artist-focused tools.

Text prompts alone, they're not great ways to make a movie. And it gets us excited to think about that future where we are going to be able to give artists these kinds of tools. And as these tools do get so powerful, we do need to use them thoughtfully with the full permission of the talent, like you saw in the example of Indiana Jones -- (Applause) Where Harrison Ford gave his permission for his likeness to be used, and we had full access and the rights to the training data.

So that's what we're focused on at Lucasfilm and ILM. The next generation of creative tools, leaning on all these amazing innovations. And then we can just imagine that next game changer, that next Star Destroyer moment, that's going to light up screens around the world.

It's going to be even more imaginative. It can be even more creative than that original that inspired us all in 1977. Thank you very much.