机器视觉综述PPT课件

ComputerVision(机器视觉)Imagebykirkh.deviantart.comToday’sTalkWhatisComputerVision?WhyStudyComputerVision?HowVisionisUsedNow?OverviewofComputerVisionAlgorithmChallengesofComputerVisionQuestionsWhatiscomputervision?Terminator2Terminator5EverypicturetellsastoryGoalofcomputervisionistowritecomputerprogramsthatcaninterpretimagesCancomputersmatch(orbeat)humanvision?WhatisComputerVision?AutomaticunderstandingofimagesandvideoComputingpropertiesofthe3Dworldfromvisualdata(measurement)1.VisionformeasurementReal-timestereoStructurefrommotionNASAMarsRoverPollefeysetal.Multi-viewstereoforcommunityphotocollectionsGoeseleetal.Slidecredit:L.LazebnikWhatisComputerVision?AutomaticunderstandingofimagesandvideoComputingpropertiesofthe3Dworldfromvisualdata(measurement)Algorithmsandrepresentationstoallowamachinetorecognizeobjects,people,scenes,andactivities.(perceptionandinterpretation)2.Visionforperception,interpretationskywaterFerriswheelamusementparkCedarPoint12EtreetreetreecarouseldeckpeoplewaitinginlinerideriderideumbrellaspedestriansmaxairbenchtreeLakeEriepeoplesittingonrideObjectsActivitiesScenesLocationsText/writingFacesGesturesMotionsEmotions…TheWickedTwisterWhatisComputerVision?AutomaticunderstandingofimagesandvideoComputingpropertiesofthe3Dworldfromvisualdata(measurement)Algorithmsandrepresentationstoallowamachinetorecognizeobjects,people,scenes,andactivities.(perceptionandinterpretation)Algorithmstomine,search,andinteractwithvisualdata(searchandorganization)3.VisionforsearchandorganizationComponentsofacomputervisionsystemLightingSceneCameraComputerSceneInterpretationSrinivasaNarasimhan’sslideComputervisionvshumanvisionWhatweseeWhatacomputerseesVisionisreallyhardVisionisanamazingfeatofnaturalintelligenceVisualcortexoccupiesabout50%ofbrainMorehumanbraindevotedtovisionthananythingelseIsthataqueenorabishop?VisionismultidisciplinaryFromwikiComputerGraphicsHCIWhycomputervisionmattersSafetyHealthSecurityComfortAccessFunAlittlestoryaboutComputerVisionIn1966,MarvinMinskyatMITaskedhisundergraduatestudentGeraldJaySussmanto“spendthesummerlinkingacameratoacomputerandgettingthecomputertodescribewhatitsaw”.Wenowknowthattheproblemisslightlymoredifficultthanthat.(Szeliski2009,ComputerVision)Ridiculouslybriefhistoryofcomputervision1966:Minskyassignscomputervisionasanundergraduatesummerproject1960’s:interpretationofsyntheticworlds1970’s:someprogressoninterpretingselectedimages1980’s:ANNscomeandgo;shifttowardgeometryandincreasedmathematicalrigor1990’s:facerecognition;statisticalanalysisinvogue2000’s:broaderrecognition;largeannotateddatasetsavailable;videoprocessingstarts2030’s:robotuprising?Guzman‘68OhtaKanade‘78TurkandPentland‘91Whystudycomputervision?MillionsofimagesbeingcapturedallthetimeLotsofusefulapplicationsThenextslidesshowthecurrentstateoftheartSource:S.LazebnikFlickr1billion2billion3billion4billion5billion6billionOtherphotosharingsites10billion20billion50billion30billion40billion…andgrowingFlickr:>1.7millionphotos/dayFacebook:>100millionphotos/dayYouTube:>35hoursofvideoeveryminute~57billionphotoswillbetaken(US)in2010http://windowsteamblog.com/windows_live/b/windowslive/archive/2010/04/09/what-to-do-with-57-billion-photos.aspx(asofNovember2010)(comparewith~17billionnegativesexposedin1996)(asofFebruary2010)HowvisionisusednowExamplesofstate-of-the-art1.Opticalcharacterrecognition(OCR)Digitrecognition,AT&Tlabshttp://www.research.att.com/~yann/TechnologytoconvertscanneddocstotextIfyouhaveascanner,itprobablycamewithOCRsoftwareLicenseplatereadershttp://en.wikipedia.org/wiki/Automatic_number_plate_recognition2.FacedetectionManynewdigitalcamerasnowdetectfacesCanon,Sony,Fuji,…3.SmiledetectionSonyCyber-shot®T70DigitalStillCamera4.3DfromthousandsofimagesBuildingRomeinaDay:Agarwaletal.2009TheoldcityofDubrovnik,4,619images,3,485,717points5.Objectrecognition(insupermarkets)LaneHawkbyEvolutionRobotics“Asmartcameraisflush-mountedinthecheckoutlane,continuouslywatchingforitems.Whenanitemisdetectedandrecognized,thecashierverifiesthequantityofitemsthatwerefoundunderthebasket,andcontinuestoclosethetransaction.Theitemcanremainunderthebasket,andwithLaneHawk,youareassuredtogetpaidforit…“6.Vision-basedbiometrics“HowtheAfghanGirlwasIdentifiedbyHerIrisPatterns”NationalGeographic7.ForensicsSource:NayarandNishino,“EyesforRelighting”Source:NayarandNishino,“EyesforRelighting”Source:NayarandNishino,“EyesforRelighting”8.Loginwithoutapassword…Fingerprintscannersonmanynewlaptops,otherdevicesFacerecognitionsystemsnowbeginningtoappearmorewidelyhttp://www.sensiblevision.com/9.Objectrecognition(inmobilephones)Point&Find,NokiaGoogleGoggles10.VisioninspaceVisionsystems(JPL)usedforseveraltasksPanoramastitching3DterrainmodelingObstacledetection,positiontrackingFormore,read“ComputerVisiononMars”byMatthiesetal.NASA'SMarsExplorationRoverSpiritcapturedthiswestwardviewfromatopalowplateauwhereSpiritspenttheclosingmonthsof2007.11.IndustrialrobotsVision-guidedrobotspositionnutrunnersonwheels12.Mobilerobotshttp://www.robocup.org/NASA’sMarsSpiritRoverhttp://en.wikipedia.org/wiki/Spirit_roverSaxenaetal.2008STAIRatStanford13.MedicalimagingImageguidedsurgeryGrimsonetal.,MIT3DimagingMRI,CT14.Digitalcosmetics15.InpaintingBertalmioetal.SIGGRAPH0016.DebluringFergusetal.SIGGRAPH0617.SportsSportvisionfirstdownlineNiceexplanationonwww.howstuffworks.comhttp://www.sportvision.com/video.html18.SmartcarsMobileyeVisionsystemscurrentlyinhigh-endBMW,GM,VolvomodelsBy2010:70%ofcarmanufacturers.19.GooglecarsOct9,2010. "GoogleCarsDriveThemselves,inTraffic". TheNewYorkTimes.JohnMarkoffJune24,2011."Nevadastatelawpavesthewayfordriverlesscars". FinancialPost.ChristineDobbyAug9,2011,"HumanerrorblamedafterGoogle'sdriverlesscarsparksfive-vehiclecrash". TheStar (Toronto)20.InteractiveGames:KinectObjectRecognition:http://www.youtube.com/watch?feature=iv&v=fQ59dXOo63oMario:http://www.youtube.com/watch?v=8CTJL5lUjHg3D:http://www.youtube.com/watch?v=7QrnwoO1-8ARobot:http://www.youtube.com/watch?v=w8BmgtMKFbYTheMatrixmovies,ESCEntertainment,XYZRGB,NRC21.Specialeffects:shapecapturePiratesoftheCarribean,IndustrialLightandMagic22.Specialeffects:motioncaptureComputerVisionandNearbyFieldsComputerGraphics:ModelstoImagesComp.Photography:ImagestoImagesComputerVision:ImagestoModelsOverviewofComputerVisionAlgorithmSowhatdohumanscareabout?Verification:isthatabus?slidebyFeiFei,Fergus&TorralbaDetection:aretherecars?slidebyFeiFei,Fergus&TorralbaIdentification:isthatapictureofMao?slidebyFeiFei,Fergus&TorralbaObjectcategorizationskybuildingflagwallbannerbuscarsbusfacestreetlampslidebyFeiFei,Fergus&TorralbaSceneandcontextcategorizationoutdoorcitytraffic…slidebyFeiFei,Fergus&TorralbaRough3Dlayout,depthorderingOverviewofComputerVisionAlgorithmImageformationFeaturesGrouping&fittingMulti-viewgeometryRecognition&learningMotion&tracking1.ImageformationHowdoeslightin3dworldprojecttoform2dimages?2.FeaturesandfiltersTransforminganddescribingimages;textures,colors,edges3.Grouping&fitting[figfromShietal]Clustering,segmentation,fitting;whatpartsbelongtogether?4.MultipleviewsHartleyandZissermanMulti-viewgeometry,matching,invariantfeatures,stereovisionFei-FeiLi5.RecognitionandlearningRecognizingobjectsandcategories,learningtechniques6.MotionandtrackingTrackingobjects,videoanalysis,lowlevelmotion,opticalflowChallenges1:viewpointvariationMichelangelo1475-1564Challenges2:illuminationslidecredit:S.UllmanChallenges3:occlusionMagritte,1957Challenges4:scaleslidebyFeiFei,Fergus&TorralbaChallenges5:deformationXu,Beihong1943Challenges6:backgroundclutterKlimt,1913Challenges7:objectintra-classvariationslidebyFei-Fei,Fergus&TorralbaChallenges8:localambiguityslidebyFei-Fei,Fergus&TorralbaChallenges9:theworldbehindtheimageChallenges10:complexityThousandstomillionsofpixelsinanimage3,000-30,000humanrecognizableobjectcategories30+degreesoffreedomintheposeofarticulatedobjects(humans)BillionsofimagesindexedbyGoogleImageSearch18billion+printsproducedfromdigitalcameraimagesin2004295.5millioncameraphonessoldin2005KeepMoving…Ok,clearlythevisionproblemisdeepandchallenging…timetogiveup?Activeresearchareawithexcitingprogress!………………………