As the US elections are approaching, threats of interference in the elections are also mounting. This is why a team of researchers have found a way of identifying deepfake videos by using heartbeat detection.
All different platforms are making sure that they don’t play any direct or indirect role in tampering of the elections this time.
Perhaps this is why both Twitter and Facebook decided to take concrete actions. They both took down the accounts related to the Internet Research Agency (IRA).
IRA is the Russian agency that interfered in the US elections last time and was responsible for misleading 126 million Facebook users through misinformation.
Deepfakes are the latest addition to the threats that can prevent fair elections. Let’s see how this latest way of identifying deepfake videos work.
Identifying deepfake videos using heartbeat detection
It is definitely astonishing how heartbeat detection can help in identifying deepfake videos but it is possible.
So far, the existing deepfake models have used traditional media forensics methods that track distortions on the edge of the face or unnatural eyelid movements.
But now there is a new thing out in the market – photoplethysmography (PPG). The way blood flows can cause subtle skin colour changes. PPG uses these subtle visual changes and translates them into a human heartbeat.
This can be useful in identifying deepfake videos because current generative models can not copy human blood movements.
PPG cells for detecting “deepfake heartbeats”
The Binghamton University and Intel researchers have found out that most of these doctored videos leave behind “deepfake heartbeats”.
Deepfake heartbeats are the unique generative and biological noise signals. PPG detect these signals from 32 different spots on the face which researchers call “PPG cells”.
The experiment paper says, “We propose a deepfake source-detector that predicts the source generative model for any given video.”
It further states, “Our key finding emerges from the fact that we can interpret these biological signals as fake heartbeats that contain a signature transformation of the residuals per model.”
Along with determining if the video’s authenticity, it also helps in detecting the source model of that video in question.
DeepRhythm model
The AI researchers from the Alibaba Groups, Nanyang Technological University, Tianjin University and Kyushu University came up with DeepRhythm.
DeepRythm is another deepfake detection model which uses visual PPG to identify deepfake videos.
The authors of this research said that the heart rhythm patterns of the deepfake videos are different from that of the real ones because they are diminished by deepfake methods.
The team said, “Experimental results on FaceForensics++ and Deepfake Detection Challenge-preview data set demonstrate that our method not only outperforms state-of-the-art methods but is robust to various degradations.”
