Tuesday, December 13, 2016

Article: Meet John Knoll, the Creative Genius Who Brought Rogue One to Life

Meet John Knoll, the Creative Genius Who Brought Rogue One to Life

In one corner of John Knoll’s office at Lucasfilm stand three racks of imposing black computer servers. The sleek 6-foot-tall towers, complete with mechanical switches and fans, flash blue LEDs. Each bears the insignia of the Galactic Empire from Star Wars and a name—Death Star 748, Death Star 749. Imperial computers, these are.

As impressive and menacing as the machines appear, they aren’t real. They’re just faceplates wired with Arduino controllers to make the lights blink and flutter like actual computers. They are, in other words, visual effects—and a look into the mind of Knoll, the 54-year-old chief creative officer of Industrial Light & Magic, Lucasfilm’s famed VFX arm.

A museum’s worth of movie props and models decorate Lucasfilm’s labyrinthine halls—the flotsam of Star Wars, Star Trek, E.T. … a half century of iconic cinema. But Knoll’s servers (or, rather, faceplates) aren’t from a movie. They’re what made the movies. They come from the machines that spent roughly 13,000 hours rendering digital effects for the three Star Wars prequels, on which Knoll was a lead effects supervisor. The march of Moore’s law turned the server farm that created those movies into scrap. Or, for Knoll, a project.

“It took a few weeks,” Knoll says, shrugging. “I play around with a lot of different things.”

Sure, Knoll has the demeanor, appearance, and excitable gee-whizziness of a STEM-obsessed, garage-full-of-half-built-projects dad. But unlike most of those dads, when Knoll takes on a hobby, he gets so good at it that he sometimes changes entire industries.

Like the time Knoll got interested in using computers to edit pictures. He and his brother ended up creating Photoshop—maybe you’ve heard of it? Or when he started tinkering with commercial computer software and incidentally reinvented how moviemakers use it to generate images for film. Or when he got so good at making Star Wars movies that he—well, that brings us to his most recent hobby.

In a darkened conference room outfitted with a big flatscreen monitor and speakers, a team of effects specialists is sitting at the center table. One starts typing into a keyboard, pulling up movie scenes. In the first, a tight grouping of TIE fighters chases an X-wing as it skims the surface of a Star Destroyer—familiar players in the Star Wars spaceship armada. The shot is part of an epic battle between the Empire and the Rebel Alliance that’s a key sequence in Rogue One, the newest entry in the Star Wars universe. It’s a prequel again. Knoll is running VFX again. And more.

The X-wing pulls up as it approaches the battle cruiser’s conning tower. “So are we going to have him blow up that dome on his way over?” asks Vick Schutz, the CG supervisor, pointing to one of the Epcot-looking polyhedrons that for reasons no one in the room is quite sure of sit atop all Star Destroyers.

Rogue One, the first of the non-“saga” Star Wars films (which is to say, it doesn’t focus on Luke Skywalker’s bloodline), comes out on December 16. Knoll is not just supervising the 1,600-plus effects shots. He is also responsible for the movie’s existence. In his free time, Knoll came up with the plot.

“People are usually good from a technical side or a creative side, but not both,” says Gareth Edwards, the movie’s director. “John Knoll is definitely a filmmaker.” Edwards should know; before he became a director, he spent years doing visual effects. “When it comes to Star Wars, some people get excited about meeting Harrison Ford,” Edwards says. “For me, it was John Knoll.”

When he was a kid, Knoll built models—World War II fighters, spaceships, and vehicles of his own invention. And like most kids building models in the 1970s, he bit hard on the effects in the original Star Wars. Suddenly, model builders (model builders!) were moviemaking heroes, and Knoll obsessed over articles on their work in American Cinematographer and Cinefantastique. But then he did something most kids wouldn’t do.

In 1978, Knoll’s dad, Glenn Knoll, a nuclear engineer at the University of Michigan, was scheduled to speak at a conference in Anaheim, California, and he took John and his other two sons along. Thrilled to be near Hollywood, John checked the hotel-room phone book to see if it had a listing for Industrial Light & Magic. There it was. A minute later John was actually talking to Grant McCune, the head of ILM’s model shop. In as professional a voice as he could muster, John explained that he was a model maker and talked his way into a tour. McCune realized he had been talking to a 15-year-old only when John’s dad dropped him off at ILM in Van Nuys the next morning.

Knoll ended up hanging out for the entire day, watching the team construct models and choreograph the camerawork for the original Battlestar Galactica TV show. It was the first time he’d ever seen people—average people, people like him—designing special effects for a living.

Knoll went back to LA for film school at USC, the alma mater of George Lucas and Robert Zemeckis, and while still a student started showing his portfolio around Hollywood. One of his first gigs was for Greg Jein, who built the miniatures for Close Encounters of the Third Kind and Star Trek: The Motion Picture. “I was impressed by the fact that he could cut and draw a straight line,” Jein says, “not something all model makers can do with complete accuracy.” Freelancing for Jein, Knoll built things like alien-­spaceship landing gear for the TV miniseries V.

By the time he graduated in 1984, Knoll was finding steady work. Which was its own kind of problem. “As soon as you take your hobby and make it into your profession, it sort of kills it as a hobby,” he says.

But his Star Wars obsession came through for him again. Knoll remembered reading about the computer-controlled camera that visual effects pioneer John Dykstra invented for the first movie in ILM’s early days. The “Dykstraflex” shot precise, repeatable sequences that helped make miniatures look real and allowed effects makers to composite multiple shots—critical for creating convincing space battles.

Knoll decided to build his own.

He took a three-year-old Apple II computer and connected it to a controller for a milling machine that could drive four stepper motors. Then he attached an adjustable camera stand. It was cheap and it worked: Knoll used the camera to create a two-minute cartoon that became his senior thesis.

It also landed him his dream job: A year later ILM—the company Knoll had cold-called as a kid—hired him to work in its camera department, moving him to the company’s new offices in the San Francisco Bay Area. “My very first day, I was working on the original Dykstraflex camera,” Knoll says. “Even being next to it was a thrill.”

Once again Knoll’s hobby had become his profession. But that meant he needed something else to do with his free time. So he started coding.

Knoll calls the photo “Jennifer in Paradise.” It shows his then-girlfriend (now wife) on a beach in Bora Bora, facing away from the camera, her hair draped over one shoulder. She looks off at an island in the distance. The picture is a historic artifact in the annals of photography and software: It’s the first color image ever Photoshopped.

In 1986, Knoll, still just a camera operator, asked for a tour of ILM’s fledgling computer graphics group and the machine it had built: the Pixar. Inspired by the device’s ability to digitally manipulate large images, Knoll went home and started trying to write his own graphics programs on his Mac. “I wrote a little ray tracer and some processing code,” he says. You know, for fun.

By coincidence, Knoll’s brother Tom was working on the same thing. For his PhD in computer vision at the University of Michigan, Tom was writing software that could do things like adjust the brightness of a digital image or detect the edge of an object in a photo. After a visit, John took a copy of Tom’s software home and started playing with it. In a few weeks, John was pretty sure they were onto something.

“We should sell this,” he eventually said.
“Are you crazy?” Tom responded. “Do you have any idea how much work writing a commercial application is?” Tom started writing one anyway.

Still young and unmarried, Knoll was on the night crew at ILM, working from 7 pm to 5 am. That meant he had his days free to shop their hobby project to software companies. While visiting Apple, Knoll asked to borrow a flatbed scanner, which he used to upload a photograph of Jennifer.

In front of techies up and down Silicon Valley, Knoll cloned an onscreen Jennifer, then cloned her again. He created a new island in the background, shrouded in mist. It was astonishing. Finally Adobe, maker of the application Illustrator and the printer language PostScript, signed up to license and distribute the software. After trying and failing for months to come up with a good name, they defaulted to simply calling it Photoshop.

Around this time, partially because of his work on the program, Knoll got his first chance to work in computer graphics for a movie—James Cameron’s The Abyss.

The biggest CG project ILM had done up to that point was a knight made of stained glass in Young Sherlock Holmes. It was roughly six shots. “This was like 16 shots,” says famed visual effects supervisor Dennis Muren. “Jim didn’t know if it was going to work.” Knoll had a reputation for problem-solving; Muren asked him to look into The Abyss.

The CG scenes centered on what the team called the pseudopod: a liquescent, tentacle-like creature that could morph its surface into faces. “We were in new territory,” Cameron says. “But John was unflappable.”

Knoll figured he’d better cover all the angles—literally. He brought a still camera to the set and captured the scene from every possible light and camera location. “For reflections, I needed to photograph the whole environment all the way around,” Knoll says. Almost by accident, he had invented a fundamental approach to compositing. “Now it’s become standard practice to have still cameras photograph the environments of every setup we do.” Knoll pasted the photos together using Photoshop.

“We managed to get it done on time and on budget,” Muren says. “And it looked really good.”

Meanwhile, Photoshop took off. By version 3, Adobe decided it needed to own the software. “So they made us a really good offer and just bought us,” Knoll says. Tom went to work for Adobe, which now has more than 10 million Photoshop users. John stayed at ILM.

“In high school and college, I’d set a bunch of goals for myself,” Knoll says. “I wanted to be the lead effects supervisor on one of these really big, innovative visual effects productions, something on the scale of a Star Wars movie. And I wanted to work on a project that wins the Academy Award for best visual effects.”

That is what moviemakers call foreshadowing.

In 1995, George Lucas announced that he wanted to clean up and rerelease the Star Wars trilogy. Originally he just wanted to create new prints—the negatives were in terrible shape. But while cleaning and digitizing them … why not polish up some of the effects?

The project, which would become known as the Special Editions, happened to line up with Knoll’s latest hobby: over-the-counter graphics software. Knoll knew something the rest of ILM didn’t. He could replicate—even improve—many of the effects, cheaply and easily, on his Mac.

Knoll’s engineer mindset and DIY attitude can seem more attuned to Star Trek than the more fantastical mysticism of Star Wars. (Ask him about this and all he’ll say is, “Well, I came from a family of scientists and engineers …”) And as it happened, while working on the Star Trek: The Next Generation TV show, Knoll was the one who had figured out how to take the starship Enterprise to warp speed. He used a classic technique called slit-scan, filming models of the Enterprise with a moving camera and long exposure through a partially blocked lens. The result: The Enterprise appears to stretch and then snap! Boldly going.

In 1993, Paramount Pictures started making the first TNG movie, Generations. ILM did the effects; Knoll was the supervisor. He asked the computer graphics department what it would cost to create a better warp effect. “The numbers that I got back were so high I didn’t think I could even turn them in to Paramount,” Knoll says. “So I decided, all right, I’m going to do this myself.”

He went home and sat down in front of his Mac. Five weeks later, the shot was done. Snap!

That’s why, two years later, when ILM started working on the Star Wars Special Editions, Knoll asked Tom Kennedy, a VFX producer on the project, if he could try the same thing—rebuilding the space-battle scenes with off-the-shelf software. Kennedy was skeptical. He set up a bake-off, Knoll versus the entire ILM computer graphics department: They would work on similar shots of X-wings and compare the results.

Four days later, Knoll was done.

“This was in addition to John’s day job,” says Kennedy, who left ILM in 1999. “I had this mental image of him at home with his children hanging off him competing with an entire CG team.”

After a month, the computer graphics department still hadn’t finished. Kennedy pulled the plug. “John took on as many of those battle shots as George wanted him to do.”

Star Wars fans still fret about the Special Editions. The space battles are better, but changes like the shock wave rings in the explosions of Alderaan and the Death Star? Pass. (Oh—spoiler alert.)

Lucas thought the revisions looked great, though. So when he came up with his next big movie, he called Muren—and Knoll. In 1996 they drove to Lucas’ Marin County retreat, Skywalker Ranch, to see 3,600 story­board panels laying out Star Wars: Episode I—The Phantom Menace. “Every couple of boards there was something we’d never done before,” Knoll says. There were shots with hundreds of CG objects onscreen at one time (the most ILM had attempted was 16), shots that would require the graphics team to simulate soft, believable cloth, shots that would require them to build rigid robots that would blow apart convincingly.

It wasn’t just that Knoll’s team would have to invent new software—it was the sheer volume of effects they would have to create. Most blockbusters at the time had something on the order of 360 effects shots. Phantom Menace needed more than 2,000. And Lucas wanted to make three prequels. “George’s attitude was just: You’ll figure it out,” Knoll says.

He was now officially a visual effects supervisor on something as big and ambitious as the original Star Wars. “It was five times bigger than the biggest show I’d ever been involved in,” Knoll says. “What I kept telling myself was ‘This has gotta be how the guys on the original Star Wars films felt.’”

While many people in the company were panicking over the scope of the project, Knoll calmly laid out a plan that involved ramping up full-scale effects production for 18 months instead of the typical two. “They would listen to that speech and then go, ‘OK, sounds reasonable.’ And they would leave, and I’d go, ‘Whew, I hope that works!’”

That can-do attitude was typical. “I have never heard John say ‘No, that can’t be done,’” says producer Jon Landau, who worked with Knoll on Avatar. “John takes on a challenge and finds a way to solve that challenge.”

And solve it, he did. Inside ILM the prequels were recognized as a heroic feat—something that fundamentally changed what the company felt it was capable of. The project also managed to get Knoll close to his other goal. Phantom Menace and Attack of the Clones were both nominated for VFX Academy Awards. (Knoll finally won, two years later, with Pirates of the Caribbean: Dead Man’s Chest.)

In 2012, Disney bought Lucasfilm, ILM, and Skywalker Sound from Lucas for $4 billion. The company then announced that Lucasfilm would begin a slate of new Star Wars movies.

Knoll had, over the years, daydreamed his own fan fiction. And early word on the plots of the new movies didn’t impress him. “The first couple were kind of backstory,” Knoll says. “How Han and Chewie got to be the characters that we know in Episode IV. And I thought, ‘Yeah, but what I really want to see is more of an action-adventure story—something that has some of the Star Wars themes, maybe it touches on things that we know, but it’s all-new characters.’”

Like what? Knoll kept coming back to this:

It is a period of civil war.
Rebel spaceships, striking
from a hidden base, have won
their first victory against
the evil Galactic Empire.

During the battle, Rebel
spies managed to steal secret
plans to the Empire’s
ultimate weapon, the DEATH
STAR, an armored space
station with enough power to
destroy an entire planet.

You recognize it—it’s the Opening Crawl, the setup kicking off Star Wars (OK, OK, A New Hope). Knoll wondered, who were these spies? We don’t hear anything more about them. Maybe they were a SEAL Team 6 in the Star Wars universe on a Mission: Impossible–style caper. That’s a movie Knoll would like to see.

He couldn’t let it go. For weeks he button­holed friends in ILM’s spaceship-and-monster-filled hallways with the words “Picture this …” In the company’s cafeteria at lunch, he’d refine the story, live. People loved it. “It would get a little more elaborate in each telling,” Knoll says. “We have this annual trivia night, where we raise money for charity. And I sat down at one of these tables with a bunch of friends of mine, and we had about a half hour before the thing was going to begin, and somebody said, ‘Hey, tell me your Star Wars idea.’ So I did like a half-hour version of it.”

The response? Pitch this to Kathy.

Kathleen Kennedy had become president of Lucasfilm when Disney took over. She’d known Knoll for decades. Like everyone, Kennedy had first heard about Knoll as the coinventor of Photoshop. But then, on a company retreat, she was on a Trivial Pursuit team with him. “There’s something about John’s ability to retain detailed and complex information that is really on a genius scale,” she says.

Still, when Knoll came to pitch a movie, Kennedy felt trepidation. “Is everyone at the company going to come at me with Star Wars movies?” she thought.

But she listened. And she liked what she heard. “You hear a lot of pitches, and they are pretty convoluted,” Kennedy says. “When you hear something that at its core is a simple but big idea—that is really rare.” Green light.

Back in the dark conference room at ILM, Knoll’s effects gang still has to decide whether to blow up those troublesome polyhedrons. Rogue One will be in theaters in a little over nine weeks. All Knoll’s team wants is for the boss to look at a shot and say the magic word: final. Then they can move on.

“So the reference from Jedi is, it’s just a giant fireball,” says Hal Hickel, the animation supervisor. “There’s nothing even in it.”

“Like it’s made of gasoline?” Knoll asks.

“Maybe it feeds into the generator that runs the shield,” someone else jokes.

Scripts often don’t answer issues like this, offering little more than an elaborate version of “spaceships fight.”

“Storywise, the battle wants to be broken up into seven or eight beats,” Knoll says. One of the beats is that the Rebels have to annihilate two Star Destroyers. It’s up to Knoll and ILM to figure out how.

So … explode the polyhedron? “Fine with me,” Knoll says. Half the room groans. The shot had been close to complete. Not anymore.

Knoll’s job also requires painful, almost microscopic scrutiny. At one point he reviews a Star Destroyer torn in half in battle—the reflections, the textures, the realism of the bent metal. The model maker is working from the book Incredible Cross-­Sections of Star Wars: The Ultimate Guide to Star Wars Vehicles and Spacecraft to make sure that what an audience sees inside the ship matches what’s known about Star Destroyers. No one wants to be the subject of a subreddit dedicated to power converters and the jerks who put them in the wrong place.

And it’s not just space battles. In one scene Jyn Erso, the Rebel hero played by Felicity Jones, has a conversation with Rebel captain Cassian Andor, played by Diego Luna. They’re on an alien planet, and Knoll spots a problem. “I’m detecting a little too much magenta on his face.” You might not have seen it, but it could have made the world seem less visceral, less real.

They watch every shot over and over. Knoll sends most of them back for revisions—some minor, some substantial. Everyone is slumping in their chairs. They just want a final.

Next up: a sequence of two Star Destroyers about to collide, part of the beat that ILM had to solve. The shot is impressive—the immensity of the cruisers overwhelming, the cinematography stunning. Knoll smiles.

“Final,” he says. The room cheers.

Then a scene of a droid walking down a hall.

“Final.” Another cheer. They’re getting there.

Rogue One is almost done. And John Knoll needs a new hobby.

-- https://www.wired.com/2016/11/john-knoll-rogue-one-star-wars/