Inside ‘The Irishman’: How VFX Supervisor Pablo Helman Brought Young Robert De Niro to Life

The first time Pablo Helman met Martin Scorsese, it was like a scene from one of the director’s movies. “It was everything I thought it would be,” Helman told Decider in a phone interview. “This black car comes in, and then here comes Scorsese dressed in this really long black coat and a hat.”

It was May 2014, and the visual effects supervisor and director were in Taiwan, scouting for Scorsese’s 2016 film, Silence. “We hit it off really great,” Helman recalled. “He’s a great filmmaker but also he’s a great person. Very funny and very talkative.”

By this point, Helman had been working at Lucasfilm’s visual effects company, Industrial Light & Magic (ILM), for nearly two decades. When he mentioned to Scorsese that ILM was working on the new frontier of “digital humans,” the director was intrigued. “He says, ‘You know, I’ve been trying to do this script for 10 years.’ He didn’t tell me the name of it.”

Five years later, Helman and ILM debuted a brand-new technology to the world via The Irishman, which is now streaming on Netflix. Robert De Niro, who was 74 at the time of the shoot, stars as Frank Sheeran, a real-life mafia member whose memoir I Heard You Paint Houses, is the basis of the film. We first meet Sheeran as an old man in a nursing home, but—as Sheeran reflects on his friendship with mobster Russell Bufalino (Joe Pesci) and his role in the disappearance of union leader Jimmy Hoffa (Al Pacino)—we meet him in his 20s, 30s, 40s, 50s, and 60s, too. That means most of De Niro’s screentime in The Irishman is computer-generated. But, thanks to Helman and ILM, it’s still 100 percent De Niro’s performance.

Here’s how the de-aging technology in The Irishman works: At all times, there are a minimum of three cameras on the actor’s face. A highly sensitive software uses those three points of view to detect the difference between light and shadow on the actor’s skin. Frame by frame, the software uses each pixel as a “marker” to render a completely computer-generated version of the actor’s face.

It’s groundbreaking technology, mostly because it doesn’t involve putting any marker or technology on the actors’ faces, as motion-capture technology—such as Will Smith’s de-aging in Gemini Man—often does. (There were, however, markers on The Irishman actors’ shoulders, chest, and clothes.)

“Marty told me we could not use markers on the actor’s face—no helmets or little cameras or anything that would get in the middle of the performance,” Helman said. Scorsese also banned keyframe animation enhancements—aka any CG-enhancements that didn’t come from De Niro’s original performance on set. “Marty said, ‘Do not touch the performance.'”

Instead, Helman spent two years “basically deconstructing De Niro’s and Pesci’s and Al Pacino’s face.” For each De Niro scene that required de-aging, he would create at least two computer images—one for De Niro at 74, and one for the age Frank Sheeran was supposed to be in the film.

Helman had countless meetings, both in person and over FaceTime, with Scorsese. “I’d show him finished shots of the younger versions of the actors, side by side with whatever we had on set. The conversations—and this was really great—were basically about performances. How does Marty feel about the performances I was showing him? Does he feel the same as he felt when he selected the performances either on set or the editing room? How can we bring this sense of this concern he saw in De Niro at 74 now [that] he’s 46?”

One of the more difficult scenes to de-age, Helman recalled, was one in which De Niro meets with Whispers DiTullio (Paul Herman) in a diner, and barely says a word. “The less dialogue there is, the more difficult the work that we’re doing,” Helman said. “Last month, I told Bob, ‘We spent a lot of time looking at your face. Do you realize you can move your eyebrows to the left and to the right independently from each other? Did you practice that in the mirror?’ He said, ‘No, I never realized that.'”

To keep track of the different versions of De Niro, Pesci, and Pacino’s characters, Helman and Scorsese created a chart. “The chart started Robert De Niro 1944. and it ended in 1975. He starts out more or less late 20s, let’s say, then he goes to 36, then 41, then 42, 43, 45, and 55. And between 55 and 60, that’s when we start switching over to makeup [rather than CG]. For Joe Pesci, the ages go from 53 or 54 all the way to 83 when he dies. Then Hoffa starts at 44 and ends at 62 when he gets killed.”

Figuring out the right look for each age for a three-and-a-half-hour movie told in a non-linear fashion was slow, meticulous work. “It’s kinda like a puzzle,” Helman said. “We scheduled the work so that we had shots from the beginning, middle, and end so that Marty would have a pretty good idea of where we’re going.”

What about the actors’ bodies? Were those changed digitally to reflect the age of the characters? “We did change the bodies a bit,” Helman said. “At the beginning, when they’re younger, they’re thinner, and towards the middle of the script, it changes to whatever bodies they have.”

The anticipation around The Irishman‘s de-aging has been building for years, and while the film’s overwhelmingly positive reviews have it on a path to Best Picture, not everyone’s a fan of the way the technology turned out. Online reactions were particularly brutal after the first photos of de-aged De Niro dropped, sparking memes. At the world premiere of The Irishman at the New York Film Festival in September, several audience members laughed when de-aged De Niro first showed up on the screen as a truck driver.

“I think the problem with trailers and [first-look] photos, especially in a movie about performances, is that you don’t understand the context. Once you see it in context, it changes the way you perceive it,” Helman said in response to the negative reactions. “The first time that we see Frank Sheeran as a young person in the truck—we worked on that transition for months. We gave Marty all kinds of different options to transition into it, including maybe not having those shots, just going directly into the gas station. That would have been kind of a smoother transition, per se. But Marty was really interested in getting the audience to get into the world of the younger Frank Sheeran in an immersive, fast and quick way. If it was jarring, the more to it. That was Marty’s intention.”

Helman also noted that the decision to change De Niro’s eyes from brown to Sheeran’s blue—something several critics have said was unwelcome—stemmed from Scorsese’s desire to “always search for the truth.” Helman compared it to De Niro’s transformation into the crime boss Al Capone in 1987’s The Untouchables. “He gained a lot of weight, lost some hair, and he shows up and you say, ‘Wow, is that De Niro?’ We all understood that we’re dealing with an iconic actor. Yes, everybody knows that Bob has brown eyes, but it was really important to the design of the character.”

Despite the critics, Helman couldn’t be prouder of the technology he helped to develop, which he believes will only improve in time and pave the way for actor-friendly CG technology for decades to come. He’s already using the same technology on a few new projects at ILM, though he declined to mention specific titles, joking, “Hopefully it’ll be completely invisible, and you won’t know!”

As for The Irishman, he wants it to be a beacon of hope for actors discouraged by the increased use of CGI in Hollywood. “I can’t wait for every actor to look at this movie and say, ‘Does that mean I don’t have to wear 138 markers on my face anymore? I don’t have to spend two hours in makeup? I don’t have to calibrate the cameras? I can just relax and be an actor?’ That’s basically why we did all this. At the end of the day, it’s for the actors, and for the performances.”

Watch The Irishman on Netflix