InteractivelyModelingwithPhotogrammetryWedescribeaninteractivesystemtoreconstruct3Dgeometryandextracttexturesfromasetofphotographstakenwitharbitrarycameraparameters.Thebasicideaistolettheuserdraw2Dgeometryontheimagesandsetconstraintsusingthesedrawings.Becausetheinputcomesdirectlyfromtheuser,hecanmoreeasilyresolvemostoftheambiguitiesanddifficultiestraditionalcomputervisionalgorithmsmustdealwith.Asetofgeometricallinearconstraintsformulatedasaweightedleast-squaresproblemisefficientlysolvedforthecameraparameters,andthenforthe3Dgeometry.Iterationsbetweenthesetwostepsleadtoimprovementsonbothresults.Onceasatisfying3Dmodelisreconstructed,itscolortexturesareextractedbysamplingtheprojectedtexelsinthecorrespondingimages.Allthetexturesassociatedwithapolygonarethenfittedtooneanother,andthecorrespondingcolorsarecombinedaccordingtoasetofcriteriainordertoformauniquetexture.Thesystemproduces3Dmodelsandenvironmentsmoresuitableforrealisticimagesynthesisandcomputeraugmentedreality.Realismincomputergraphicshasgreatlyevolvedoverthepastdecade.Howeververyfewsyntheticimagessimulatingrealenvironmentscanfoolanobserver.Amajordifficultylieswiththe3Dmodels;creatingrealisticmodelsisanexpensiveandtediousprocess.Unfortunatelythegrowingneedforthislevelofaccuracyisessentialforrealisticimagesynthesis,moviespecialeffects,andcomputeraugmentedreality.Oneattractivedirectionistoextractthesemodelsfromrealphotographs.Althoughtwodecadesofcomputervisionresearchhasledtoimportantfundamentalresults,afullyautomatedandreliablereconstructionalgorithmingeneralsituationshasnotyetbeenpresented,atleastfor3Dmodelssatisfyingcomputergraphicsgeneralrequirements.Misinformationincomputervisionalgorithmsresultingfromfalsecorrespondences,missededgedetections,noise,etc.cancreateseveredifficultiesintheextracted3Dmodels.Webaseourpremiseonthefactthattheuserknowswhathewantstomodel,andwithinwhichaccuracy.Hecandecidewhatmustbemodeledbygeometry,andwhatcouldbesimulatedbyasimplergeometrywithatextureappliedonit.Toprovidethisfunctionality,wedevelopedafullyinteractivereconstructionsystem.Gettinganaccurate3Dmodelrequiresthesolutionofseveralproblems,whichareallinterrelated.Wemustfirstcomputecorrectcameraparameters,andthenusethecamerasandconstraintstoreconstructthe3Dgeometry.Afterdiscussingsomerelatedwork,weoutlineourgeometryreconstructionsystem.Asetofcorrespondencesandincidencesresultinsimpleandefficientlinearconstraints.Althoughtheseconstraintsarenotnew,theimprovementsobtainedinaccuracyandspeeddemonstratetheimportanceofconsideringallofthemtogether.Userinterventionateverystepofthisprocess,resultsinmoresatisfyinggeneralreconstructed3Dmodels.Simple3Dgeometrywillbeeffectiveonlywithgoodqualitytextures.WefocusinSection3onamorecomplete,view-independent,treatmentoftextures.Texturesareextractedforeach3Dgeometryfromallimagesitprojectsto.Thebesttexels(2Dtextureelements)arethencombinedintoasingletextureaccordingtovariouscriteriaincludingvisibility,projectedareas,colordifferences,andimagequality.Bysolvingaccuratelyeachproblem,wewillbetterunderstandtherobustness,stability,andprecisionofourtechniques.Itshouldbecomeeasierlaterontoextendourinteractionswithmoreautomaticcomputervisionandimageprocessingtechniquesinordertoalleviatesomeofthemorecumbersomeandtedioustasks,whilekeepinguserinterventionwhererequired.Theresultsofoursystemshouldhelpuscreatemoreprecisetexturedsyntheticmodelsfromreal3Dobjectsinlesstimethancurrent3Dmodelers,andmorerobustlythanfullyautomatedgeometryextractionalgorithms.Twentyyearsofactiveresearchon3Dreconstructionfrom2Dimagesincomputervisionandroboticshaveleftaconsiderablelegacyofimportantresults.Thefirstproblemtoaddressconcernscameracalibration,putingcameraparameters.Thisisadifficultandunstableprocessoftenimprovedbytheuseofspecifictargets.Byputtingincorrespondencepointsorlinesbetweenimages,itbecomespossibletocalibratecameras.Similarlywithknowncameraparameters,onecanreconstructa3Dsceneuptoascalefactor.Intheseclassicalapproaches,segmentationsandcorrespondencesareautomaticallydetermined.OnetypicalexampleoftheresultsobtainedbytheseapproacheswasrecentlypresentedbySatoetal..AfewrecentprojectssuchasREALISE,Fac?ade,PhotoModelerandAIDAproposetointegratemoreuserinterventionintothereconstructionprocess.Theyarederivedfromprojectivegeometry,andareappliedtothereconstructionofman-madescenesfromasetofphotographsandcorrespondences.REALISEintegratesuserinterventionearlyinthecorrespondenceprocess,lettingtheuserspecifythefirstcorrespondences,andthenreturningtoamoreclassicalapproachtoidentifyautomaticallymostoftheothercorrespondences.Themorestableinitialsolutiongreatlyhelpstoreducetheerrorsofsubsequentiterations.Neverthelessthesameerrorsoffullyautomaticsystemscanstilloccur,andtheusermustthendetectandcorrecttheoriginoftheerrors,whichisnotasimpletaskasthenumberofautomaticcorrespondencesincreases.Fac?adedevelopsaseriesofparameterizedblockprimitives.Eachblockencodesefficientlyandhierarchicallyseveralconstraintsfrequentlypresentinarchitecturaldesign.Theusermustfirstplacetheblockswitha3Dmodeler,andthensetcorrespondencesbetweentheimagesandtheseblocks.Non-linearoptimizationofanobjectivefunctionisthenusedtosolveforalltheseconstraints.Thesystemhasproventobequiteefficientandprovidesprecise3Dmodelswithlittleeffort.Howeveritrequirestheusertobuildwiththeblocksthemodelhewantstoreconstruct.Webelievethismightbemoredifficultwhengeneral3Dmodelscannotbeaseasilycreatedwiththeseblocks.PhotoModelerisacommercialsoftwareforperformingphotogrammetricmeasurementsonmodelsbuiltfromphotographs.Oncethecameraiscalibrated,theuserhastoindicatefeaturesandcorrespondencesontheimages,andthesystemcomputesthe3Dscene.Themodelsobtainedappearquitegood,althoughitseemstobealengthyprocess(theyreportedaweekforamodelof2003Dpoints)whichusesimagesofveryhighresolution(around15MBeach).Wealsonoticedmanylongthintrianglesandgapsinsomeoftheirmodels.Thesystemcanapplytexturescomingfromthephotographsbutdoesnotseemtoperformanyparticulartreatmentsincetheshadows,highlights,etc.arestillpresent.Nodetailsareprovidedonthealgorithmsused.TheAIDAsystemisafullyautomaticreconstructionsystemthatcombinessurfacereconstructiontechniqueswithobjectrecognitionforthegenerationof3Dmodelsforcomputergraphicsapplications.Thesystempossessesaknowledgedatabaseofconstraints,andselectstheconstraintstoapplytothesurfaceunderreconstructionafterperformingasceneinterpretationphase.Webelieveitmightbesafer,lesscumbersome,andmoregeneraltolettheuserchoosewhichconstraintshewantstoapplytoits3Dprimitivesratherthanlettingthesystempicksomeconstraintsfromaknowledgedatabasecreatedspecificallyforthetypeofscenetoreconstruct.Forthesereasons,weintroduceasystemessentiallybasedonuserinteraction.Theuserisresponsiblefor(almost)everything,butalsohasthecontrolon(almost)everything.Thisshouldprovideacomprehensivetooltoimproveonthemodelingfromreal3Dobjectsandonthecomputergraphicsqualityofthese3Dmodels,whileofferingtheopportunitytofocusonthedetailsimportanttothedesigner.SystemOverview.Wehavedevelopedaninteractivereconstructionsystemfromimages.Theimagesdefinethecanvasonwhichallinteractionisbased.Theycancomefromanytypeofcameras(evenavirtualsyntheticcamera)withanysettingsandposition.Theuserdrawspoints,lines,andpolygonsontheimageswhichformourbasic2Dprimitives.Theuserinteractivelyspecifiescorrespondencesbetweenthe2Dprimitivesondifferentimages.Hecanalsoassignotherconstraintsbetweenreconstructed3Dprimitivessimplybyclickingononeoftheirrespective2Dprimitives.Theseadditionalconstraintsincludeparallelism,perpendicularity,planarity,andco-planarity.Atanytime,theusercanaskthesystemtoreconstructallcomputablecamerasand3Dprimitives.Thereconstructed3Dprimitivescanbereprojectedontheimagestoestimatethequalityofeachrecoveredcameraandthe3Dprimitives.Theuserthenhasthechoicetoiterateafewtimestoimproveonthemathematicalsolution,ortoaddnew2Dprimitives,correspondences,andconstraintstorefinethe3Dmodels.Thisprocess,illustratedinFig.1,demonstratestheflexibilityandpowerofourtechnique.The3Dmodelisreconstructedincrementally,refinedwhereandwhennecessary.Eacherrorfromtheusercanalsobeimmediatelydetectedusingreprojection.ContrarilytoDebevecetal.,theuserdoesnotcreateasyntheticmodelofthegeometryhewantstorecover,althoughthereconstructed3Dmodelcanaseasilybeusedtoestablishnewconstraintsbetween2Dandreconstructed3Dprimitives.Eachimagethuscontainsasetof2Dprimitivesdrawnonit,andacameracomputedwhenthesetofresolvedconstraintsissufficient.Tobootstrapthereconstructionprocess,theuserassignsasufficientnumberof3Dcoordinatesto3Dprimitivesviaoneoftheircorresponding2Dprimitives.Forinstance,six3Dpointsinoneimageallowthecomputationofthecorrespondingcamera.Oncetwocamerasarecomputed,all3Dgeometrythatcanbecomputedbyresolvingtheconstraintsisreconstructed.Withtheassignedandthenewlycomputed3Dvalues,theconstraintsareresolvedagaintoimprovethereconstructedcameras.Thisprocessiteratesuntilnomoreconstraintscanberesolved,andthe3Dgeometryandcamerasarecomputedtoasatisfactoryprecision.Typically,aconvergenceiterationsolvingtheequationsystemsforcomputingallthecamerasand3Dpositionstakesbetween0.05and2seconds,1dependingonthecomplexityofthescene(50to2003Dpoints)andtheconstraintsused.Allourconstraintsareexpressedaslinearequations,typicallyforminganoverdeterminedsetofequations.Aleast-squaressolutiontothissystemiscomputedbysingularvaluedecomposition.Weusethissolutionfortheunknowncameraparameters,andtheunknown3Dcoordinatesofpointsandlines.Asanexactcorrespondenceishardlyachievablebydrawing2Dpointsontheimageplane,wecomputethebestcameraparametersintheleast-squaressense.Hartleydemonstratesimpleconditionsunderwhichlinearsystemsofequationsusedtodeterminethecameraparametersareaspreciseastheirnon-linearcounterparts.Moreoverthistechniqueissimpletoimplement,efficient,general,alwaysprovidesasolution,andweobservedthatitismorerobustthanthenon-linearsystems.Additional3DConstraints.Mostman-madescenesexhibitsomeformofplanarity,parallelism,perpendicularity,symmetry,etc.Usingcorrespondencesonly,reconstructedgeometryoftendoesnotrespecttheseproperties,whichcanleadtoobjectionableartifactsinthereconstructed3Dmodels.Itisthereforeveryimportanttointegratethistypeofconstraintinareconstructionsystem.Theyareunfortunatelydifficulttodetectautomatically,asperspectiveprojectiondoesnotpreservethemintheimage.Theusercanhoweververyeasilyindicateeachsuchconstraintdirectlyontotheimages.Theyareintegratedintoourconvergenceprocessbyaddingequationstothesystemoflinearequationsusedtocompute3Dcoordinates.Althoughtheyarenotstrictlyenforcedbecausetheyaresimplypartofaleast-squaressolution,theyoftenresultinmoresatisfying3Dmodelsespeciallywithrespecttotheneedsofcomputergraphicsmodels.Coplanarity:Aplanarpolygonwithmorethanthreeverticesshouldhaveallitsverticesonthesameplane.Foreachpolygonwith3Dvertices,weaddaplanarityconstraintoftheformthatwillbeusedduringthecomputationofeach.Polygons,points,andlinescanalsobeconstrainedtolieonthesamesupportingplane.Tocomputethisplane,theusercanspecifyanormaldirection.Wecomputethebestvalueforbyusingtheknown3Dpoints.Ifthereisnoinformationabouttheplaneorientation,wecomputethebestplaneintheleast-squaressense,thatpassesthroughatleastthreeknown3Dpointsofthecoplanarprimitives,andapplythesameconstraint.Parallelism:Similarly,severalpolygonsandlinescanbeparalleltoeachother,providingadditionalconstraints.Foreachpolygon,wegetitsorientationfromitsplaneequation,ifavailable.Theseorientationsallowustocalculateanaverageorientationthatwillbeattributedtoalltheparallelpolygons,eventhoseforwhichnoorientationcouldbefirstcalculated.Aplanarityconstraintisaddedtothecomputationofthepolygons3Dpoints.Forparallellines,wecomputetheiraveragedirection.Perpendicularity:Ifthenormalstoperpendicularpolygonsareknown,wecanaddotherconstraintsforthecomputationof.Twoperpendicularpolygonshaveperpendicularnormals,thusforeverypolygonorthogonaltoasetofparallelpolygons,wehave.Manyotherconstraintsshouldbeexploited.Symmetrycouldconstraincharacteristicssuchaslengthsorangles.Similaritybetweenmodelscouldspecifytwoidenticalelementsatdifferentpositions.Incidenceofpointsandlinescanbeextendedtodifferentprimitives.Theseareonlyafewoftheconstraintsweobservein3Dscenes.Eachbasicconstraintdescribedabovecanbeusedasabuildingblockformoreelaborateprimitives.Acubeforinstancebecomesasetofplanarfaces,withperpendicularityandparallelismbetweenitsfacesandsegments,andconstrainedlengthbetweenits3Dvertices.Ratherthanlettingtheuserspecifyalltheseconstraints,anewprimitiveforwhichallofthesearealreadyhandledrepresentsamuchmoreefficienttoolfortheuser.Thesenewprimitivescanbedescribedinalibraryofprimitivesorganizedhierarchically.Debevecetal.showshowthisrepresentationcanalsoreducesignificantlythenumberofconstraintstoresolve.Wecanalsoweightthecontributionsoftheconstraintsdependingontheirimportanceinthecurrentreconstruction.Thedefaultweightsassignedtoeachtypeofconstraintcanbeeditedbytheuser.Theresolutionofourequationsystemsissimplyextendedtoaweightedleast-squares.ResultsofGeometryReconstructionToevaluatetheprecisionandtheconvergenceofouriterativeprocess,weconstructedasimplesyntheticscenemadeofsevenboxes.FiveimagesofresolutionwhererenderedfromcamerapositionsindicatedbythegreyconesinFig.3(left).2Dpolygonsweremanuallydrawnandputincorrespondenceswithin60minutesona195MHzR10000SGIImpact.The3Dcoordinatesofsixpointsofthecentralcubeonthefloorwereenteredtobootstrapthesystem.ThethreecurvesinFig.3(right)representthedistanceinworldcoordinatesbetweenthereal3Dpositionofthreepointsinthescene((-2,3,0),(0,2,-2),(1,2,-2))andtheirreconstructedcorrespondents.200iterationswithoutanyconstraintsotherthanthepointcorrespondencestookabout5minutes.Wethenappliedsuccessivelytheconstraintsofplanarity,coplanarity,andparallelismbetweenallthe3Dpolygons.Calculatingalltheseconstraintstypicallyaddsafewtenthsofasecondperiterationdependingonthecomplexityofthe3Dscene.Thethreecurvesreachaplateauafteracertainnumberofiterations.Thisdoesnotmeanthatthe3Dmodelisthenperfectlyreconstructed,butratherthatthesolutionisstableandshouldnotchangesignificantlywithmoreiterations.Whenweintroducetheplanarityandthenthecoplanarityconstraintsforindividualpolygons,thepointsmoveslightly.Inthisscenewhereparallelismispreponderant,theadditionofthislastconstraintimprovessignificantlythereconstructionforallthreepoints,whichisshownbythedropofallthreecurvesafteriteration300.Theintroductionofanewconstraintcansometimesperturbthewholesystem,affectingmoretheless-constrainedelementsasdemonstratedbythesuddenspikeincurve2.Inmostobservedcases,thesystemquicklyreturnstoanimprovedandmorestablestate.InFig.3(center),wereprojectinwireframemodethereconstructedmodelusingthecomputedcamerafromoneoftheoriginalimages.Distancesbetweenthe2Ddrawnpointsandthereprojectedreconstructedpointsallliewithinlessthanonepixelfromeachother.When3Dconstraintsareusedtoimprovethemodel,thisdistancecanreachuptotwopixels.The3Dscenethencorrespondsmoretorealitybutdoesnotfitexactlythedrawnprimitiveswhenreprojectedwiththecomputedprojectionmatrix.Theconstraintsthuscompensatefortheinaccuracyintroducedbytheuserinteractionorbytheprimitivesfarfromtheenteredcoordinates.Becausetheuserdraws2Dprimitivesattheresolutionoftheimage,gettingamaximumoftwopixelsisconsideredsatisfactory.Sub-pixelsaccuracyisobtainediftheseprimitivesaredrawnatsub-pixelprecision,butthislengthentheuserinteractiontime.ExtractingTextureTextureshavebeenintroducedincomputergraphicstoincreasetherealismofsyntheticsurfaces.Theyencodeviaasurfaceparameterizationthecolorforeachpointonthesurface.Whilethecontributionoftexturestorealismisobvious,itisnotalwayseasytoextractatexturefromrealimages.Onemustcorrectforperspectiveforeshortening,surfacecurvature,hiddenportionsofthetexture,reflections,shading,etc.Alltheselimitationshaverestrictedthetypeofextractedrealtextures.Howeverourreconstructedgeometryandcamerasprovideagreatcontextwithinwhichwecanextractthesetextures.Mostcurrentapproachesarebasedonview-dependenttextures.Havaldaretal.usetheprojectionofthe3Dprimitiveinalltheimagestodeterminethebestsourceimageforthetexture.Thenweapplytothetexturethe2Dtransformationfromtheprojectedpolygoninthisbestimage,totheprojectedpolygonintheimagefromanewviewpoint.Unfortunately,this2Ddeformationofthetextureisinvalidforaperspectiveprojection,andpronetovisibilityerrors.Debevecetal.reprojectseachextractedtextureforagivenprimitiveasaweightedfunctionbasedontheviewingangleofthenewcameraposition.Thetechniqueprovidesbetterresultswithview-dependentinformation.However,neglectingthedistancefactorintheweightscanintroduceimportanterrors,andaliasingcanappearfromtheuseofocclusionmaps.Moreoverallthetexturesmustbekeptinmemoryaspotentiallyallofthemmightbereprojectedforanynewviewpoint.NiemandBroszioidentifiesthebestimageforanentirepolygon(accordingtoangleanddistancecriteria),andsamplesthetexturefromthisimage.Becauseadjacentpolygonscanhavedifferentbestimages,theythenproceedtosmoothouttheadjacenttexels,possiblyalteringthetextures.Inoursystem,atextureisextracteduponuserrequestforagivenprimitiveprojectinginanumberofimages.Thetextureistheresultofrecombiningtheestimatedbestcolorsforeachpointofthesurfaceprojectedineachimage.Eventhoughextractingasingletextureispronetoerrorsaswillbediscussedlater,itismoresuitableforgeneralimagesynthesisapplicationssuchasapplyingittodifferentprimitives,filtering,anduseofgraphicshardware.Foragiven3Dpoint,itsprojectioninoneimagewillmostlikelybeavisiblepointbecausetheuserdrewitssupporting3Dprimitiveasacorresponding2Dprimitiveontheimage.Howeversomeportionofthe3Dprimitivemightbeblockedbyanother3Dprimitiveclosertotheimageplane,thusleadingtoanincoherentcolor.Asimpletestdeterminesthezonewithanocclusionriskbyintersectingthe2Dprimitivewithall2Dprimitivesonthisimage.Ifthereisintersection,wemustdeterminethe3Dintersectionbetweenthecorresponding2Dpointontheimage(twoplanes)andthepotentiallyoccluding3Dpolygon(athirdplane).Ifthereisintersectionandthedepthissmallerthantheoneofthe3Dpoint,wesimplymarkthisoccludedcolorsampleasinvalid.Weextractacolorforeachtexelineachimage.Thefinalcolorforthistexelmustbecomputedfromthesecolors.Thesizeinpixelsofthetexelprojectedintheimageisagoodindicationofthequalityofthecolorextractedforthistexel.Thelargertheprojectedareaofatexel,themoreprecisethetextureshouldbe.Therefore,forallthevalidcolorsofagiventexel,weweightitscolorcontributionasarelativefunctionoftheprojectedareasofthetexelinallselectedimages.Ofeketal.storeseachpixelineachimageintoamipmappyramidalstructureforthetexture.Colorinformationispropagatedupanddownthepyramid,withsomeindicationofcertaintyaccordingtocolorvariations.Thestructurefitseachtexeltotheimagepixelresolution,thusadaptingitsprocessingaccordingly.Howeverunlessveryhighresolutiontexturesarerequired,webelievetheextracostofpropagatingtheinformationinthepyramidandtheinevitablelossofinformationduetothefilteringbetweenlevelsmightnotbeworththesavings.Oursolutionissimple,butmightrequiresamplingmanytexelsprojectingwithinasmallfractionofapixelarea.Howeverallinformationiskeptattheuser-specifiedtexelresolution,andassuch,wegetmuchflexibilityinwaysofinterpretingandfilteringtheinformation.Itisalsofairlysimpletointegratevariousnewcriteriatoimproveontheprocessofcombiningtheextractedtexels.Unfortunatelywemustbeawarethatseveralsituationsmightinvalidateanysuchtextureextractionalgorithm.Anyview-dependentfeaturethatchangestheaspect(color)ofa3Dpointasthecameramovesmightbeasourceoferrors.Theseincludespecularreflections(highlights),mirrors,transparencies,refractions,ignoredsurfacedeformations(sharpgroovesandpeaks),participatingmedia,etc.Withoutuserintervention,acombinationoftheseartifactscanhardlybehandledautomatically.Whenonecoloratonetexelisverydifferentthantheothersfordifferentimages,wesimplyrejectitscontribution,assumingitwascausedbyview-dependentfeaturesornoiseintheimage.Whenallthecolorsareverydifferentfromeachother,wesimplymarkthistexelasinvalid.Wewilldiscussintheconclusionhowthesedifferencescouldbeusedtoextractsuchview-dependentinformation.Thecolorofapixelatsilhouettesandedgesofpolygonsincludesnotonlythetexelcolor,butalsobackgroundandothergeometrycolors.Cameraregistrationerrorsalsointroduceslightmisalignmentsbetweenthereconstructed3Dprimitiveandeach2Dprimitiveitshouldprojectto.Weagainsimplymarksuchatexelasinvalid.Wethereforeendoutwithanextractedtexturewithsomeundefinedtexelcolors.Fortunately,thesetypicallyrepresentasmallportionoftheentiretexture.Wecurrentlyfillinthesetexelsbyapplyingasimplefilter,althoughsomefillingalgorithmshaveproventobequiteefficient.Theyshouldbeevenmoreeffectiveforgapsasnarrowasthoseobservedsofarinourtests.交互式建模與攝影我們描述一個互動系統(tǒng)，以重建三維幾何，并從中提取了與任意攝像機(jī)拍攝的照片設(shè)置紋理參數(shù)。其基本思想是讓用戶在圖像上繪制二維幾何和設(shè)置限制使用這些圖紙。由于輸入直接來自用戶，他可以更容易地解決困難的含糊和最傳統(tǒng)的計算機(jī)視覺算法必須處理的問題。作為一個加權(quán)最小二乘問題制定一套幾何線性約束有效地解決了相機(jī)的參數(shù)，然后三維幾何圖形。這兩個步驟迭代導(dǎo)致兩種結(jié)果的改善上。一旦滿足三維模型重建，它的顏色紋理提取樣品在相應(yīng)的圖像投影紋理元素。所有與多邊形關(guān)聯(lián)的紋理，然后再裝一個，及相應(yīng)的顏色組合根據(jù)一套標(biāo)準(zhǔn)，以形成獨(dú)特的質(zhì)感。該系統(tǒng)產(chǎn)生的3D模型和環(huán)境，更逼真的圖像合成和增強(qiáng)現(xiàn)實(shí)合適的電腦。在計算機(jī)圖形學(xué)中的現(xiàn)實(shí)主義已經(jīng)發(fā)生巨大變化，在過去的10年。但很少能模擬真實(shí)環(huán)境愚弄觀察員合成圖像。一個主要的困難在于三維模型，創(chuàng)造現(xiàn)實(shí)的模式是一個昂貴和繁瑣的過程。不幸的是，為達(dá)到這一精度水平不斷增長的需求是現(xiàn)實(shí)的圖像合成的必需，電影特效，計算機(jī)增強(qiáng)現(xiàn)實(shí)。一個有吸引力的方向是從實(shí)際中提取的照片，這些模式。雖然兩名計算機(jī)視覺研究數(shù)十年，導(dǎo)致重要的基本成果，在一般情況下完全自動化和可靠的重建算法尚未提交，至少在計算機(jī)圖形三維模型滿足一般要求。在計算機(jī)視覺算法，從虛假的誤導(dǎo)造成的書信，錯過了邊緣檢測，噪聲等，可以建立三維模型中提取的嚴(yán)重困難。我們立足于一個事實(shí)，即用戶知道他要什么型號，我們在其中的準(zhǔn)確性的前提。他可以決定什么必須由幾何模型，以及可以通過一個與它應(yīng)用了一個簡單的幾何紋理模擬。為了實(shí)現(xiàn)這一功能，我們開發(fā)了一個完全互動的重建系統(tǒng)。得到一個準(zhǔn)確的三維模型需要幾個問題，這些問題都是相互關(guān)聯(lián)的解決方案。我們首先必須計算正確的攝像機(jī)參數(shù)，然后使用相機(jī)和約束的三維幾何重建。在討論了一些相關(guān)的工作，我們闡明我們的幾何重建系統(tǒng)。一種簡單的書信和發(fā)病率和有效率的線性限制結(jié)果集。雖然這些限制是不是新的，改進(jìn)的精度和速度獲得展示了它們放在一起考慮的重要性。用戶在這個過程的每一步，在一般的結(jié)果更令人滿意的干預(yù)重建的三維模型。簡單的三維幾何結(jié)構(gòu)將會只有具備良好素質(zhì)的紋理效果。我們集中更完整，觀點(diǎn)獨(dú)立，紋理處理。每個紋理提取所有圖像三維的幾何項(xiàng)目。最好的紋理元素（2D紋理元素），然后合并成一個單一的紋理根據(jù)不同的標(biāo)準(zhǔn)包括能見度，投影面積，顏色的差異，以及圖像質(zhì)量。通過準(zhǔn)確地解決每一個問題，我們將更好地了解魯棒性，穩(wěn)定性，和我們的技術(shù)精度。它應(yīng)該成為日后在社會上更容易更自動延長計算機(jī)視覺和圖像處理技術(shù)的相互作用，以減輕較繁瑣，繁瑣的一些任務(wù)，同時保持在需要用戶干預(yù)。我們系統(tǒng)的結(jié)果應(yīng)有助于我們建立在更短的時間比目前的立體模型更精確的實(shí)時紋理合成三維物體模型，以及更強(qiáng)勁的幾何比全自動提取算法。20年來積極研究從二維三維重建在計算機(jī)視覺和機(jī)器人形象留下了一個重要的結(jié)果。第一個問題，解決問題相機(jī)標(biāo)定，即計算攝像機(jī)參數(shù)。這是一個困難和不穩(wěn)定的過程通常由使用改進(jìn)的具體目標(biāo)。通過在圖像間對應(yīng)點(diǎn)或線推桿，就有可能以校準(zhǔn)相機(jī)。同樣已知攝像機(jī)參數(shù)，可以重建一個三維場景最多，其規(guī)模因素。在這些經(jīng)典的方法，分割及書信自動確定。通過這些方法之一所取得的成果典型的例子是最近提出的佐藤等人。他們是從射影幾何派生，并應(yīng)用于人造從照片和書信集場景重建。結(jié)合用戶干預(yù)早在對應(yīng)過程中，讓用戶指定的第一書信來往，然后回到一個更經(jīng)典的方式來確定自動成為其他通訊最。較穩(wěn)定的初步方案大大有助于減少以后的迭代中的錯誤。不過全自動系統(tǒng)仍然可以出現(xiàn)同樣的錯誤，并且用戶必須再檢測和糾正錯誤，這不是作為簡單的數(shù)量的增加自動對應(yīng)任務(wù)的起源。外交事務(wù)委員會?開發(fā)了一系列的參數(shù)塊圖元。每個塊編碼效率和層次結(jié)構(gòu)在建筑設(shè)計中經(jīng)常出席一些制約因素。用戶必須先在三維建模與塊之間，以及與這些圖像塊然后設(shè)置對應(yīng)。非線性優(yōu)化的目標(biāo)函數(shù)，然后用于解決所有這些限制。該系統(tǒng)已被證明是相當(dāng)有效的，并提供精確的三維模型與不費(fèi)吹灰之力。但它要求用戶與塊建設(shè)模式，他希望重建。我們認(rèn)為這可能是比較困難時，一般的三維模型可以很容易被這些塊創(chuàng)建。是從照片上執(zhí)行建立攝影測量模型的商業(yè)軟件。一旦相機(jī)校準(zhǔn)，用戶在圖像上顯示的功能及通信，計算和系統(tǒng)的三維場景。這些模型得出似乎相當(dāng)不錯，雖然這似乎是一個漫長的過程（他們報告了1200點(diǎn)的三維模型1周），它使用了非常高清晰度圖像（約15MB的每個）。我們也注意到在一些型號的許多細(xì)長的三角形和差距。該系統(tǒng)可以應(yīng)用紋理從照片來，但似乎沒有執(zhí)行，因?yàn)槿魏翁囟ǖ年幱疤幚?，重點(diǎn)等依然存在。沒有任何細(xì)節(jié)上的算法來提供。阿伊達(dá)系統(tǒng)，是一個完全自動重建系統(tǒng)，它結(jié)合了對象的三維計算機(jī)圖形應(yīng)用模型生成識別表面重建技術(shù)。該系統(tǒng)具有知識數(shù)據(jù)庫的限制，并選擇適用的限制下到地面后重建執(zhí)行現(xiàn)場解釋階段。我們認(rèn)為這可能是更安全，減少麻煩，并讓更多的普通用戶選擇的限制，他要申請，而不是讓系統(tǒng)挑選創(chuàng)造了具體的場景類型重建一個知識數(shù)據(jù)庫中的一些限制，它的三維圖元?；谶@些原因，我們基本上介紹用戶交互為基礎(chǔ)的系統(tǒng)。用戶是負(fù)責(zé)（幾乎）一切，但也有對（幾乎）所有的控制權(quán)。這應(yīng)提供全面的工具，從真正改善3D物體建模和計算機(jī)圖形處理這些三維模型的質(zhì)量，同時提供機(jī)會，把重點(diǎn)放在重要的細(xì)節(jié)設(shè)計。系統(tǒng)概述。我們已經(jīng)發(fā)展到一個互動的圖像重建系統(tǒng)。這些圖像定義上所有的互動為基礎(chǔ)的畫布。他們可以來自任何類型的相機(jī)（甚至是虛擬合成相機(jī)）與任何設(shè)置和位置。用戶繪制點(diǎn)，線，和我們的形象，構(gòu)成基本的2D圖元多邊形。用戶以交互方式指定不同的圖像之間的二維圖元對應(yīng)。他也可以通過簡單地分配各自的二維圖元一點(diǎn)擊重建三維圖元之間的其他限制。這些額外的限制，包括平行度，垂直度，平面和共面。在任何時候，用戶可以要求系統(tǒng)重建所有可計算的相機(jī)和三維圖元。重建后的三維圖元，可投射的圖像來估計每回收的三維圖元攝像頭和質(zhì)量。然后，用戶的選擇來迭代幾次改進(jìn)的數(shù)學(xué)解決方案，或增加新的二維圖元，書信和約束，以完善的三維模型。這個過程，如圖所示。這表明我們的靈活性和技術(shù)力量。重建的三維模型，逐步，精在有需要時。從用戶的每個錯誤也可以立即檢測到使用再投影。用戶不會創(chuàng)建一個幾何，他要收回綜合模型，雖然重建的三維模型可以很容易被用來作為之間建立二維和三維圖元重建新的限制。因此，每一個圖像上包含了繪制二維原語集，并計算相機(jī)時，解決制約集就足夠了。要引導(dǎo)重建過程中，用戶分配一個足夠數(shù)量的三維坐標(biāo)的三維圖元通過其相應(yīng)的二維元件之一。例如，允許在一個6點(diǎn)的三維圖像中相應(yīng)的相機(jī)計算。一旦兩個攝像頭計算，所有的3D幾何可以被解決的制約因素計算重構(gòu)。隨著新的分配和三維計算值，約束決心再次提高重建相機(jī)。這個過程循環(huán)，直到?jīng)]有更多的約束可以解決，和相機(jī)的三維幾何計算，以一個令人滿意的精度。通常情況下，收斂迭代求解計算所有的攝影機(jī)和3D位置的方程系統(tǒng)需要在0.05和2秒，1對現(xiàn)場（50至200三維分）和復(fù)雜程度的限制使用。我們所有的約束條件表示為線