1、a new interface paradigm for motion capture based animation abstract. this paper proposes a new user interface paradigm for motion capture based animation systems, providing intuitive and efficient ways to visualize the main motion capture concepts and operations. a prototype system was built, imple

2、menting the prop anewinterfaceparadigmformotioncapturebasedanimationsystems fernandowagnerserpavieiradasilva1,2 luizvelho1 pauloromacavalcanti2 jonasdemirandagomes1 nando,lvelho,roma,jonasvisgraf.impa.br 1impainstitutodematematicapuraeaplicadaestradadonacastorina,110,22460riodejaneiro,rj,brazil 2lcg

3、-laboratoriodecomputac aogra ca,coppe-sistemas/ufrjcaixapostal68511,21945-970,riodejaneiro,rj,brazil abstract.thispaperproposesanewuserinterfaceparadigmformotioncapturebasedanimationsystems,providingintuitiveande cientwaystovisualizethemainmotioncaptureconceptsandoperations.aprototypesystemwasbuilt,

4、implementingtheproposedinterfacemodelandsupportedbya exiblearchitecturethatissuitabletoworkwiththemotioncapturemethodology. keywords:motioncapture,animationsystems,computeranimation,graphicinterfaces,guiparadigm,motioncontrol. 1introduction themotioncapturetechniquehasalreadysetitsplaceinthefutureof

5、computeranimation.thistechniqueprovidestoolsforhigh-qualityanimation,evenwhenreal-timeisrequired. initially,themaindrawbackofmotioncapturetechniqueswasthelackofef- cientwaystomodifythecapturedmotion,byadjustingorimprovingspeci cpartsthatneededtobechanged,withouthavingtorepeattheentireacquisitionproc

6、essagain. abstract. this paper proposes a new user interface paradigm for motion capture based animation systems, providing intuitive and efficient ways to visualize the main motion capture concepts and operations. a prototype system was built, implementing the prop lately,however,severaltechniquesw

7、ereproposedtoprocesscaptureddata91112,providingtoolsformotionanalysis,modi cationandreuse.thismakesmotionlibrariesmorevaluableforawideclassofanimators. mostofcurrentanimationsystemso erthepossibilityofusingmotioncaptureddatatogenerateanimations,butseveralsystemstreatthistechniqueasan“extratool”,orev

8、enasasimpleplug-in.therefore,ane ectivedescriptionofmotioncapturebasicconceptsisnotprovided. aninterestingandalternativeapproachwouldbetheconstructionofanani-mationsystemthatusesmotioncaptureasthekerneloftheentireanimationprocess.thisleadstoanunlimitedrangeofpossibilitiestomanyanimators.asanexample,

9、thismotioncapturebasedsystemcouldbeintegratedwithhigh-endmotioncapturehardware,thuscreatingapowerfulenvironmentofmotionacquisitionandprocessing. inthiswork,weproposeanewuserinterfaceparadigmformotioncapturebasedanimationsystems,supportedbyanextensiblearchitecturethatincorporatesthe“stateofart”inmoti

10、oncaptureprocessingtechniques,andallowstheuseofstandardanimationmethods,suchaskeyframingorinversekinematics,aspowerfultoolstoimprovethesystems exibility. themaincontributionofthispaperistheintroductionofanelegantwaytodescribe,attheuserinterfacelevel,thebasicmotioncaptureabstractions.wetreatcapturedm

11、otionasapotentiallyreadyanimation,whichcanbemodi- edbyasetoftoolsembeddedinthearchitecture.inthatway,aninterfacedescriptionofmotionoperationsandassociatedobjectsarede ned. aprototypesystemwasbuilt,implementingtheconceptsdescribedinthiswork.thissystemwasusedtodemonstratethepotentialoftheproposedinter

12、faceparadigm. section2ofthispaperintroducesthebasicinternalstructuresoftheprototypesystem,togetherwithabriefdescriptionofitsarchitecture.section3presentstheproposedinterfaceparadigmformotioncapturebasedsystems.finally,conclusionsaregiveninsection4. 2systemsarchitectureandinternalstructures inthissec

13、tion,wepresentaframeworkfortheanimationsystem.also,abriefdescriptionofthearchitectureusedintheimplementedsystemisprovided.thiswillgiveabetterunderstandingoftheapproachusedtobuildtheproposedinterfaceparadigm. abstract. this paper proposes a new user interface paradigm for motion capture based animati

14、on systems, providing intuitive and efficient ways to visualize the main motion capture concepts and operations. a prototype system was built, implementing the prop 2.1basicinternalstructures thefundamentalstructureusedinthesystemiscomposedbytwoentities:anactorandmotions. theactoristreatedasaskeleto

15、n.itstopologyisrepresentedbyagraphformedbyjointsandlinks.itsgeometryisrepresentedbyasetofconnectedlimbs(figure1).thisdescriptionisadequatetobeusedinamotioncaptureani-mationsystem,sinceitre ectsthestructureofanarticulated gure.fordataacquisition,markersareattachedatthejointsofaliveperformer(thereal a

16、ctor).figure1:topologicalandgeometricaldescriptionofanactorusedinthesystem.attheprogramminglevel,theactorisrepresentedusingamodi edversionofzeltzersapj(axispositionjoint)structure3,adaptedtoworkwithmotioncaptureddata. motionsarebestrepresentedascurvesintime(figure 2). figure2:examplesofjointmotioncu

17、rves. abstract. this paper proposes a new user interface paradigm for motion capture based animation systems, providing intuitive and efficient ways to visualize the main motion capture concepts and operations. a prototype system was built, implementing the prop normally,thecaptureddataconsistsofmar

18、kerspositionaland/orangularvariation,sampledbythecapturehardwareduringthenumberofframesre-quiredtocompletetheactorsperformance.thisdescriptionisusedineachdegreeoffreedom(dof)oftheactor. internally,theinteractionwiththeuseriscontrolledbyadynamicdatastructurethatrepresentsthecurrent“state”ofallwindows

19、andmaindatastructuresexistinginthesystem. 2.2anarchitecturefocusedonmotioncapture wedevelopedaconceptualarchitecture,designedtoworkwiththemotioncap-tureparadigm.itfocusesonsometechnologicalaspectsandembodiesseveraltechniquestodealwithcaptureddata,thusallowingthecreationofreusablemotionlibrariesbyusi

20、ngabuildingblockparadigm. theframeworkofthearchitecture(figure3)isformedbythreebasicmodules(input,processingandoutput),eachoneresponsibleforaspeci csetoftasks.thedatastructuresusedinthesystemweredescribedin2.1. figure3:systemsarchitecture. allmodulesaresupportedbytheconceptualinterfacethatwillbedesc

21、ribedinsection3.foramoredetaileddescriptionofsystemsarchitecture,pleasereferto4. inputmodule theinputmodulefocusesonproblemsconcerningtheinterpretationandpre-processingofmotiondata.skeletonde nition leswerecreatedtoestablishrelationshipsbetweentheskeletonexpectedbytheincomingdata1andarchi-tecturesin

22、ternaldefaultskeletonde nition.also,geometricalalgorithmsfor1dependingonthemotiondataformat. abstract. this paper proposes a new user interface paradigm for motion capture based animation systems, providing intuitive and efficient ways to visualize the main motion capture concepts and operations. a

23、prototype system was built, implementing the prop 3deulerangleextraction2,pre- lteringandhierarchicalanglegenerationareprovidedinthismodule. processingmodule theprocessingmodulecomprisesthesetoftoolsformotionanalysis,manip-ulationandreuse.threebasicmotionoperationtypesarede ned: ltering;concatenatio

24、n;andblending.theirobjectiveistoprovidee cientwaystomodifytheoriginalcaptureddata.withthesetools,theuserisabletogeneratenewclassesofmotions,inheritingthealivenessandcomplexitytypicalofnaturalmotion. filteringoperationscanbeappliedtothejointcurvesofamotion,toreducenoiseorevenmodifyspeci ccomponentsof

25、themovement.in9,williamsusedamultiresolution lteringapproachtodecomposethemotioninfrequencybands,thusallowingmodi cationsinahigherlevelofabstraction. withconcatenationoperations,longeranimationscanbeproducedbycombiningseveralmotionsinsequence.smoothtransitionsbetweenthecombinedmotionscanbeproducedus

26、ingalgorithmsbasedonblendingofmotionparameters4.spacetimeconstraints12canalsobeusedtogenerateseamlessanddynamicallyplausibletransitions,withexcellentresults. blendingoperationsarenormallyusedtocombinespecialcharacteristicsofdif-ferentmotions.inthiscase,theexistenceoftoolsformotionsynchronizationandr

27、eparametrizationisveryimportanttohelpintheblendingprocess,ensuringacoherentresult. outputmodule themainobjectiveofthismoduleistoprovidewaystostorethecompositioncreatedbytheuser,thusmaintainingandexpandingtheexistingmotionlibrary.auniversaldataformatwasde ned,embodyingthemaincharacteristicsofmostmoti

28、ondataformatsavailablenowadays.consequently,thesystemcanbeusedasarobustconversortomotioncapturedataformats. userinterface theuserinterfaceusedinthesystemisbasedonavisualrepresentationofmotioncapturebasicconcepts,suchasmotionsandoperations. webelievethatitisalsointerestingtosupplyanon-graphicaluserco

29、mmunica-tion,usingaexpressionlanguagethatrepresentsalltheactionsthatwouldbedoneusingthegraphicaldescriptionprovidedintheuserinterface. 2inthecaseofdatawithpositionalinformationonly. abstract. this paper proposes a new user interface paradigm for motion capture based animation systems, providing intu

30、itive and efficient ways to visualize the main motion capture concepts and operations. a prototype system was built, implementing the prop wearecurrentlyworkingonaexpressionlanguagesimilartothatdescribedin12.usingthislanguage,theuserwillbeabletogeneratecomplexanimationswithmotionoperations,usingcomm

31、andsthatwillactuallyexecutethecallbackfunctionssupportingtheuserinterface.thesecommandsmaybestoredinatext leandcanbereusedoredited. control acontinuousloop(figure4)veri esthestatusofallinterfaceobjectsandwin-dows,reportinganychangestoaspecialfunctionthatmanagesthosechangesthatactuallymustbedonedueto

32、userinteraction. figure4:userinterfacecontrolwithastatecheckingloop. 3agraphicinterfaceformotioncapture asdiscussedbefore,oursystemisbasedonanarchitecturethattreatstheproblemofdealingwithcapturedmotions.however,thefunctionalityofitsarchitecturewouldbeshadowedbytheconventionaluserinterfaceparadigmuse

33、donmostanimationsystemscurrentlyavailable. ourgoalistodescribethebasicstructuresofthesystemsarchitectureinaconciseway,providingpowerfulinterfacetoolsthatwillmakeiteasiertoexecutemotioneditingoperations.moreover,thisinterfacemustbeextensible,allowingtheincorporationofnewoperationsandtechniques. wenot

34、edthateachcapturedmotionispotentiallyareadyanimation,andthereforemustbetreatedaccordingly.thispromptedustoadoptaninterfaceparadigmusedinsomepost-productionvideoworkstations10asastartingpointtoourwork. abstract. this paper proposes a new user interface paradigm for motion capture based animation syst

35、ems, providing intuitive and efficient ways to visualize the main motion capture concepts and operations. a prototype system was built, implementing the prop inthefollowingsub-sections,wewilldescribethemaininterfaceobjectsandconceptsdevelopedundertheproposedmethodology.allimagespresentedinthissectio

36、nareactualscreenshotsoftheprototypesystem,whichimplementstheconceptsdescribedinthispaper. 3.1actor inourinterfaceparadigm,theactorisvisualizedbymeansofitscomponents:theskeletontopologyandgeometry.theskeletonstructureisshowninawin-dowasagraphcomposedofjointsandlinks(figure5).thiswindowdisplaysinforma

37、tionabouteachjointandthelinksbetweenthem.theinterfaceallowsjointselectionand/or grouping. figure5:skeletongraphwindow. therepresentationoftheactorcanbevisualizedwithdi erentstylesintherenderingwindow,asshowninfigure6.theselectedjointinthegraphwindowappearsindi erentcolorandsizeintherenderingwindow.t

38、hisselectionalsoa ectsotherinterfaceobjects,aswillbediscussedlater. abstract. this paper proposes a new user interface paradigm for motion capture based animation systems, providing intuitive and efficient ways to visualize the main motion capture concepts and operations. a prototype system was buil

39、t, implementing the prop figure6:jointselection(detailfromskeletongraphwindow)andvisualfeedbackintherenderingwindow. 3.2motions inthepost-productionvideointerfacemodel,videoandsoundsequencesarevisualizedashorizontalbars,whichcanbegrouped,positionedandcombinedinatimelinecanvas,inordertoproducethe nal

40、composition. inourparadigm,wetreatamotionasahorizontalbar(figure7),whosewidthisdeterminedbythenumberofframesofthecapturedmotion.thisbaralsocontainsinformationaboutthemotionname.notethearrowmarkerattherightendofthemotionbar,whichindicatesthatmotionresizing(reparametrization)is allowed. figure7:motion

41、representationasaguiobject. wewillusuallyvisualizethemotionbarusingaframerulerassociatedwithit.thisgivesamoreaccuratetemporalperceptionofthemotion. themotionscratchpad abstract. this paper proposes a new user interface paradigm for motion capture based animation systems, providing intuitive and effi

42、cient ways to visualize the main motion capture concepts and operations. a prototype system was built, implementing the prop inoursystem,wehavecreatedaninterfaceobjectwhosepurposeistoactasamotionorganizer,providingaglobalperceptionofallmotionsplacedonit.wecalledthisobjectthemotionscratchpad(figure 8

43、). figure8:motionscratchpad-themotionorganizer. usingthescratchpad,theusercanchooseseveralmotionsandorganizethemintheobjectcanvas.allmotionscanbeindividuallyplayedorresized,andtheusercandropthemtothemotionoperations.thescratchpadisactuallythegatewaybetweentheinputandtheprocessingmodules. thescratchp

44、adisthecontainerwheremotionfragmentsarestored,waitingtobeused. jointcurves jointcurvesarethebasiccomponentsofmotion.theyarevisuallyrepresentedbyaninterfaceobjectthatdisplayscurveshape.italsoprovidesnumericin-formationaboutthedi erentdatachannelsateachframe(i.e.,threepositionandorientationvalues,fore

45、achjointcurve).thevisualrepresentationofajointcurveisshowninfigure9. withthisrepresentation,itisstraightforwardtoimplementseveralcurveeditingtechniques911,allowingapreciseandinteractivecontrolofthecurveshape.foreachjointoftheactor,theremaybeseveraljointcurves(oneforeachdof)attachedtoit.thesecurvesar

46、egroupedinainterfaceobject,thejointcurveswindow,thato ersaglobalviewofthecurves,andhasadirectconnectionwiththeskeletongraphwindow.whenajointisselectedinthegraphwindow,itscurvesaredisplayedandusefulinformationisprovided,asshowninfigure10. abstract. this paper proposes a new user interface paradigm fo

47、r motion capture based animation systems, providing intuitive and efficient ways to visualize the main motion capture concepts and operations. a prototype system was built, implementing the prop figure9:visualrepresentationofajoint curve. figure10:jointcurveswindow. themotionwindow finally,wedecided

48、tocreateaninterfaceobjectthatactsasaconnectionbe-tweenmotionandactordescriptions.themotionwindow(figure11)isac-cessedviaadouble-clickinamotionbar,andallowstheselection(orgrouping)ofspeci cjointsoftheactor. themotionwindowiscomposedofseveralbars,eachonerepresentinganactorjoint(detailinfigure11).visua

49、lly,itlookslikeazoominthemotionbar.thisrepresentationisintentional,sincethemotionisformedbythecurvesthatareattachedtoactorsjoints. themotionwindowwillproofitsutilitywhenusedinconjunctionwithmotionoperations,allowingtheapplicationofanoperationtoaspeci csetofjointsoftheactor. abstract. this paper prop

50、oses a new user interface paradigm for motion capture based animation systems, providing intuitive and efficient ways to visualize the main motion capture concepts and operations. a prototype system was built, implementing the prop figure11:motionwindow-connectingactorandmotiondescriptions. 3.3motio

51、noperations motionoperationsarethemostimportantobjectsinthearchitecture.therefore,theymustbevisualizedinawaythatmakestheprocessintuitivetotheuser.inourinterfaceparadigm,eachmotionoperationhasitsownwindow.whenrequested,additionalobjectsareusedtohelpintheprocess,providingafullcontroloftheoperationpara

52、meters. allmotionoperationwindowshaveabasicsetofauxiliaryobjects:aninteractiveplayersliderandaframeruler.theseobjectsalsofollowtheinterfaceconceptsdescribedearlier. filtering the lteringoperation(figure12)isrepresentedinawindowwithtoolswhichallowtheselectionofaspeci cregionofthemotiontobe ltered(and

53、withalistoftheexisting lters,whichcanbeaccessedbypressing concatenation abstract. this paper proposes a new user interface paradigm for motion capture based animation systems, providing intuitive and efficient ways to visualize the main motion capture concepts and operations. a prototype system was

54、built, implementing the prop figure12:filteringoperationwindow. theinterfaceobjectthatrepresentstheconcatenationoperationwasdesignedtoprovideagoodvisualperceptionofthecompositionasawhole. themotionsselectedbytheuserinthescratchpadaredroppedintothecon-catenationwindow.initially,theyarepositionedinsuc

55、hawaytoperformadirectconcatenation(i.e.,withoutblendinginterval),asshowninfigure 13. figure13:concatenationoperationwindow(initialmotionarrangement). usingtheexistingtoolsformotionpositioningandreparametrization,theusercande neblendingintervalsthatwillpermitasmoothtransitionbetweenthecombinedmotions

56、(infigure14,representedbythedarkerregionsbetweenmotions). however,someinterfaceconstraintswerecreatedtoavoidundesirableresults.theseconstraintsensurevisualcoherence,andguidetheuserintheoperationprocess. abstract. this paper proposes a new user interface paradigm for motion capture based animation sy

57、stems, providing intuitive and efficient ways to visualize the main motion capture concepts and operations. a prototype system was built, implementing the prop thedefaulteasy-in/easy-outblendingparameterscanbemodi edbydouble-clickinginthedesiredblendingintervaloftheconcatenation window. figure14:ble

58、ndingintervals(detailfromconcatenationwindow). blending blendingisthelasttypeofmotionoperationde nedintheprimarysetoftheprototypesystem. asintheconcatenationwindow,allmotionsselectedbytheuserareplacedinacanvas,providingaglobalviewoftheoperation.forblending,however,newobjectswereintroducedtoassistinthespeci cation. thesenewobjectsarethet