LTX-2.3_-_V2V_Just_Talk_add_lipsynced-voice_to_any_video - With Custom Audio File

#75
by PeZiK - opened

Hi RuneXX,

I am trying to create videos of talking animals - cats, dogs, turtles, goldfish, etc. I think your workflow is great for this, but it seems to be missing an option to insert a custom audio file that matches the video length. I’ve tried to figure it out but haven't had any luck. Can you help? Also, one question - will the output video be identical to the original, just with the added lip-sync motion if there was a custom audio option? Thank you :)

PeZiK changed discussion title from LTX-2.3_-_V2V_Just_Talk_add_lipsynced-voice_to_any_video - With Cutom Audio File to LTX-2.3_-_V2V_Just_Talk_add_lipsynced-voice_to_any_video - With Custom Audio File

This particular workflow you just prompt the dialog/voices.

It could be added a custom audio input though, i'll add an optional one.
But there are already custom audio workflows you could use. Or even the Qwen / Fish Audio ones that is sort of a combo of both. You upload an audio clip to clone, and then prompt what to say based on the sound of that input voice.

Thank you RuneXX.

A custom audio input file to control the lip-sync animation will definitely be a game-changer for me. Do you have a donation link?

Thats all the donation i need ;-) hearing people having fun with it ..
And happy to hear you are

I might add, I am interested in something like this too. I have a nice way to generate voices, would love to see if I could get it to animate to that voice. V2V_Dub_It_Lip-synced_dubbing_multilanguage has done a lot for me, just would be nice if the audio I have is too slow for what LTX generates then it doesn't match anymore.

I might add, I am interested in something like this too. I have a nice way to generate voices

You have your own custom audio you mean?
You could try the I2V custom audio workflows then. Or do you mean with a V2V workflow?

I might add, I am interested in something like this too. I have a nice way to generate voices

You have your own custom audio you mean?
You could try the I2V custom audio workflows then. Or do you mean with a V2V workflow?

Yes I have my own custom audio... V2V workflow would work best, but I should look at the I2V custom audio workflows too, I want to see if I can "lip-sync" an already existing video with audio I have... but I don't know if that is possible with LTX yet. I also just noticed some other new workflows you made that I want to try and play around with.

, I want to see if I can "lip-sync" an already existing video with audio I have... but I don't know if that is possible with LTX yet

Should be possible, but a little limited since the way would be to mask the mouth area (or face for more motion).
Will add a custom audio to the lip sync workflow.

Alternatively take the first frame of your video and let LTX be more creative, and re-do a video based on that frame, your audio and prompt

Sign up or log in to comment