Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

Program Announcements / Lip Sync Lab

Author
Message
Steverino
21
Years of Service
User Offline
Joined: 27th Aug 2002
Location: United States
Posted: 23rd Mar 2003 23:09
This is a DB Pro re-coding of the tool I used to do the lip sync for my random talking head Uncle Weevy (currently moonlighting as my avatar, still available for download at http://www.stevetiffany.com/uw.html). The old Lip Sync Lab only ran on the Amiga, but now, thanks to all the code I lifted from Easily Confused and IanM, it's up and running on the PC.

The package also includes DBP source for the infamous Peter Piper demo, which a few forum old-timers may remember, to demonstrate how to use the data that Lip Sync Lab generates.

It's at http://www.stevetiffany.com/zips/LipSyncLab.zip (1.59 MB)

____________________________________________
Surrealist writing toy -- http://www.iconpoet.com
Shadow Robert
21
Years of Service
User Offline
Joined: 22nd Sep 2002
Location: Hertfordshire, England
Posted: 24th Mar 2003 06:18
lmao... oki this is definatly one freaky program (^_^)
but hmmm me thinks you need to put these talents to some good use ... how about you make me a lipsync library for Max (you know you want to )

Tsu'va Oni Ni Jyuuko Fiori Sei Tau!
One block follows the suit ... the whole suit of blocks is the path ... what have you found?
Steverino
21
Years of Service
User Offline
Joined: 27th Aug 2002
Location: United States
Posted: 24th Mar 2003 07:16
I hear Max already has plug-ins where you feed in a WAV and the text, and it spits out perfect lip sync automatically!

My approach may be labor-intensive, but the price is right.

____________________________________________
Surrealist writing toy -- http://www.iconpoet.com
Danmatsuma
21
Years of Service
User Offline
Joined: 2nd Mar 2003
Location: Australia
Posted: 24th Mar 2003 14:16 Edited at: 24th Mar 2003 14:46
Heehee!! looks like fun Steverino, downloading....

10 minutes later...

It is, it IS fun!!

Wombat wombat wombat wombat.....
I just know It's gonna give me nightmares

Easily the simplest way to learn the basics of lip sync, have to mod it a bit for an aussie accent though

ZX Spectrum 48k Issue 3, Radio shack Tape drive, Rank arena 12" T.V. set.
Shadow Robert
21
Years of Service
User Offline
Joined: 22nd Sep 2002
Location: Hertfordshire, England
Posted: 24th Mar 2003 17:35
i know there is... but its pretty crap - and actually the animations are pretty pathetic, they look like they're plastic doll stuff
really what i'd want to do is have a grid for dragging and dropping the animations that i create myself for each of the sounds ... that way i can include the tounge in there too ... which is stupid how so many animators miss out the tounge, as its a very important part of talking and if you look at a mouth that it just opening and closing - it might do it right, but without the tounge for like N, T, TH etc... it just looks weird ya know?
like poor American Cross platforming

Tsu'va Oni Ni Jyuuko Fiori Sei Tau!
One block follows the suit ... the whole suit of blocks is the path ... what have you found?
Steverino
21
Years of Service
User Offline
Joined: 27th Aug 2002
Location: United States
Posted: 24th Mar 2003 19:29
There are crude Saturday-morning cartoons that get by with only 4 mouth shapes, where something like The Nightmare Before Christmas I think used a dozen, and Shrek was totally fluid. I decided on 7 as a compromise. The animation looks reasonably good, but it's easy to use and you don't have to build a model with a dozen different mouth-limbs. It wouldn't be too hard to give Lip Sync Lab a mouth shape 8, though, if you wanted something with a little tongue.

Danmatsuma -- Royt, mite!

____________________________________________
Surrealist writing toy -- http://www.iconpoet.com
Mirthin
21
Years of Service
User Offline
Joined: 13th Oct 2002
Location: Land of the Rising Haggis
Posted: 24th Mar 2003 23:09
In The Nightmare Before Christmas Jack had over 400 different heads altogether.

Who wants cake? I've got a little slice of hell for everyone.
Shadow Robert
21
Years of Service
User Offline
Joined: 22nd Sep 2002
Location: Hertfordshire, England
Posted: 25th Mar 2003 00:41
i think Shrek was done in Maya... like Final Fantasy was - they did flow, pitty i've never had the chance to use thier Lip Sync stuff actually

i mean from what i've seen the animations are done and set to sounds, then you drag'n'drop to the animation frames for that part of the skeleton by setting the timeframe... it'll then merge the frames, kinda like IK will add the physics from the properties you gave it.
So you get a very flowing and natural-equé motion

i'm always interested in the many different techniques employed for this sort of animaiton though because it is interesting to say the least. My only major quibble with animating within Max is the total lack of a decent scene animation manager - so everything you do is done by keyframes on the main timeline. I mean there is a Manager for it, but it is so blasted technical and everything you end up spending 20-30mins working on howto just setup the frames you've already animated, and without a library to pick from you have to then copy&paste from the current timeline which you can't do per limb or bone but per object.

(which is probably why most animators actually export Max scenes to Animation|Master)

Tsu'va Oni Ni Jyuuko Fiori Sei Tau!
One block follows the suit ... the whole suit of blocks is the path ... what have you found?
Steverino
21
Years of Service
User Offline
Joined: 27th Aug 2002
Location: United States
Posted: 25th Mar 2003 01:56
Okay, now I understand how your drag'n'drop would be integrated. That would be good. I've never actually worked with Max or Maya, but I've dabbled with Hash A:M enough to be impressed by their approach. That was A:M '99, so it's probably even better now.

I used the Nightmare Before Christmas example because doing lip sync by using Hide Limb on the 6 unwanted mouths and Show Limb on the 7th is a lot like what you're doing in stop-motion animation. Now that DB Pro can do mesh deformation, though, I bet it'd be possible to use Lip Sync Lab data for "targets" that your character's face would smoothly interpolate between. That could look really decent.

I can't imagine making 400 different heads!

____________________________________________
Surrealist writing toy -- http://www.iconpoet.com
Danmatsuma
21
Years of Service
User Offline
Joined: 2nd Mar 2003
Location: Australia
Posted: 25th Mar 2003 06:41 Edited at: 25th Mar 2003 06:43
Seems to me it'd be easy enough to do something with 3d mesh's in dbpro based on just the data output from steverino's 'lil proggie, just use that simple 2d interface, and instead of applying the data to 2d images, use it to govern mesh animations in a 3d object... Then you can write a 'lil utility to send your files back out to max or whatever.
Anyhow I might try something like that in classic when they send me the enhancements using memblocks..

The tongue stuff you could include in your custom mesh animations, and once you had it all back in max you could make it nice and smooth with morphing

ZX Spectrum 48k Issue 3, Radio shack Tape drive, Rank arena 12" T.V. set.
Shadow Robert
21
Years of Service
User Offline
Joined: 22nd Sep 2002
Location: Hertfordshire, England
Posted: 25th Mar 2003 21:31
yeah i've only used the 2000 edition myself and wasn't for that long though ... was certainly interesting, but i have this thing where if i start in a program i wanna do as much as i can in it. Would be annoying to like take what i've done export to A:M then once i've finished animating export back to max to reneder

you know i think perhaps if i can ever get gameSpace to a level where i want to add some serious animating perhaps you could help me work on the Animation Manager ... right now all i have for it is bone animation with Interpol (similar to Milkshapes lol) - and is in a platform you already know quite well.

i'll see about sending the UIS your way in the next 2 weeks see what you can do to make a basic animator maybe.
its a bit like a 'Visual' DBpro to use like VC++ could be helpful to see how someone fairs with it other than myself hehee

well talk to ya laters about it

Tsu'va Oni Ni Jyuuko Fiori Sei Tau!
One block follows the suit ... the whole suit of blocks is the path ... what have you found?
Steverino
21
Years of Service
User Offline
Joined: 27th Aug 2002
Location: United States
Posted: 26th Mar 2003 00:58
That sounds like it's quite a bit beyond my abilities, but I could maybe take a look.

____________________________________________
Surrealist writing toy -- http://www.iconpoet.com
Easily Confused
21
Years of Service
User Offline
Joined: 22nd Oct 2002
Location: U.K. Earth. (turn right at Venus)
Posted: 29th Mar 2003 00:15
You may be interested to know I've update the DrawMonoSnd() function for you.

http://www.darkbasicpro.com/apollo/view.php?t=7137&b=6

Programming anything is an art, and you can't rush art.
Unless your name is Bob Ross, then you can do it in thirty minutes.
Steverino
21
Years of Service
User Offline
Joined: 27th Aug 2002
Location: United States
Posted: 29th Mar 2003 05:48
Thanks! I have incorporated this update into Lip Sync Lab.

____________________________________________
Surrealist writing toy -- http://www.iconpoet.com
k0shi
21
Years of Service
User Offline
Joined: 30th Dec 2002
Location: Cyberspace
Posted: 29th Mar 2003 11:42 Edited at: 29th Mar 2003 11:44
im downloading this now ...
maybe u could make a animation program with 3d charcters and 3d lips in sync with the voices.that would cool, and a great tool for 3d rpgs.

If I could drop dead right now, I'd be the happiest man alive.
....:::Koshi:::....
LREM 8888
21
Years of Service
User Offline
Joined: 15th Mar 2003
Location: United Kingdom
Posted: 29th Mar 2003 12:34
I haven't downloaded Lip Sync Lab yet, but it sound decent. I just want to know can't you just make it so you can just import your own mouths or something? I know it works by showing and hiding limbs, but why can't you do that with objects?

Oh, and it would be helpful if you could make maybe a MaxSCRIPT for 3DS Max to do it. The Lip Sync in Max is boring. You have to work it by deforming meshes into other ones, and the timing's up to you.

Steverino
21
Years of Service
User Offline
Joined: 27th Aug 2002
Location: United States
Posted: 30th Mar 2003 02:31
It generates a file full of Data statements in the following format, which you would paste into your DB program:

_wavName:
Data NumberOfMouths,1stMouthShape,1stMouthWait,2ndMouthShape,2ndMouthWait, etc....

In a 2D program you might handle the data like this:



In a 3D program, it might look something like this:



But those are only suggestions. You can do whatever you want with the data. Probably using it as morph targets would look the best, once we figure out how to do that, but you can use it to hide limbs, hide objects, whatever you want.

____________________________________________
Surrealist writing toy -- http://www.iconpoet.com

Login to post a reply

Server time is: 2024-03-28 09:47:53
Your offset time is: 2024-03-28 09:47:53