How long before we see fan edits use "deepfake" voices and similar such tech?

FAST6191

Techromancer
OP
Editorial Team
Joined
Nov 21, 2005
Messages
36,798
Trophies
3
XP
28,321
Country
United Kingdom
For as long as there have been people funding art the artists they fund have chafed under their direction, and thus artist remixes of "their" work are nothing new. Fan edits are not a new concept either (how many places will have a local take on a song or poem possibly hundreds of years old?) but the DVD generation and rise of home PC video editing changed things somewhat, even more so DVDs with deleted scenes. Not sure what to point at for the current pinnacle of this sort of thing but a place to start would likely be Star Wars the Phantom Edit, a fan edit designed to correct the problems with Episode , nice intro/refresher1https://starwarsfans.fandom.com/wiki/The_Phantom_Edit . Right now most uses are spoofs, silliness and porn but it is likely only a matter of time before it is a routinely used tool in the arsenal.

We are probably a ways off regenerating/creating whole scenes but voice generation sample requirements have got rather small (not to mention many actors will have hours and hours of sample data to feed it), the same things that provide face swapping also provide for lip syncing. In the commercial side of things we have been seeing product placement edited in for some years now, though much of that is more traditional some poor bastard has to go frame by frame tweaking a mask. Obviously we have also had singers... brought back from the dead for the better part of 20 years now but that was also more traditional stitching in many cases. I am however also interested in "fake" songs, though I imagine it will be your favourite artist does karaoke or a better version of a mashup/"duet".

I also wonder if it will be fun trying to find "official" cuts of films on the future equivalent of torrent sites.

Personally I would like to see what critics might do here -- as it stands we can already see people edit photos and obviously text as well. "Everybody is a critic" does then get "money where your mouth is" if it results in but a few hours of work to demonstrate a failure. I also wonder if such things will be able to be distributed as user generated code -- if the voice samples can be generated from the film itself or some easy to obtain library then it likely ends up functionally no different than cut this frame to this frame, move this frame to this frame, mute this/overlay this sound, put this picture over this part of the frame (possibly with a little tweak), all of which I can already (and have long been able to) distribute as a script for a video editor.
I am also curious what might go for more adult themed edits (such things may include some more sex but language is where I would be looking first). For instance how many fansubs in anime add in the swears where the original might be lacking or milder?
 

Veho

The man who cried "Ni".
Former Staff
Joined
Apr 4, 2006
Messages
11,383
Trophies
3
Age
42
Location
Zagreb
XP
41,193
Country
Croatia
I'm waiting for someone to take the Scott Pilgrim movie and replace Michael Cera with pretty much anyone else, it would improve it significantly.


As for the voice deep fakes, it reminded me of this:




That being said the tech is just not there yet, it's far from plug and play, you can't just plug any video into another video and get seamless results, it takes days and days of collecting the right frames (or editing them to make them right) to feed the algorithm to get good looking results - in a single scene. Move to the next scene and you need to edit for lighting and color correction and everything else all over again. We're not even talking different faces/actors, just editing a face from a different scene to fit into the desired one.
So you can get a passable approximation of a voice, but the mouth would inevitably be of a different color and slightly off-center, the whole thing looking like Homer Simpson, or those stop motion puppets with the interchangeable mouths, but each one is a slightly different shade:

CUucIHA.jpg
 

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
    Flame @ Flame: Never fight uphill, me boys.