We have seen actors dubbing the voices of playback singers and it is a very common thing to us. But have you hear of dubbing expressions? Let me give you a more vivid expression. Two people are sitting side by side and videos of each individual are being captured. One is giving certain facial expressions and the other one is sitting quietly. But the video which are capturing the two persons will imprint the expression of the former onto the face of the other with a help of a special technology.
A team of devoted researchers from the U.S. and Germany are working on discovering this new technology where expressions of one person in a video can be grafted on the face of the other in a video. The technology is called real time facial expression reenactment The technique realistically grafts expressions made by one person in a source video onto the face of another person in a target video in real time.
Scientists from the globally acclaimed Max Planck Institute for Informatics, University of Erlangen-Nuremberg and Stanford University are working in collaboration with NVIDIA. The image above can give you the idea of facial expression reenactment in a nutshell. The facial expressions and lip movements are taken from the source video and grafted onto the target. The resultant composite video wil put the expression of the person in the source video to the persion in the target video.
The gestures will grafted keeping in study the skin color, shape, head pose and other factors. Each and every detail will be captured and will be encoded in terms of color and depth data. A presentation will be given at a computer graphics conference in Kobe, Japan, in early November. The study has not yet been able to graft the voice. “It’s harder to fake someone’s voice, it’s not quite there yet,” team member Matthias Niessner said in an interview.
There is a dark shadow behind this technology as people may misuse it to produce treacherous products and fake videos. “It’s important to make people aware of it and give people the technology so that they can detect manipulations,” said Niessner, when asked about it. “It would be way worse if somebody else, like some secret service, did it and didn’t tell the public about it,” Niessner said.