Z.Panetal.(Eds.):ICAT2006,LNCS4282,pp.785795,2006.Springer-VerlagBerlinHeidelberg2006RecognitionandLocationofFruitObjectsBasedonMachineVisionHuiGu,YayaLu,JilinLou,andWeitongZhangInformationEngineeringCollege,ZhejiangUniversityofTechnology,310014,Hangzhou,C,oo327,phonixlou,Abstract.Thispaperdiscussedthelowlevelmachinevisiononfruitandvegetableharvestingrobot,introducedtherecognitionandlocationoffruitandvegetableobjectsundernaturescenes,putforwardanewsegmentationmethodcombinedwithseveralcolormodels.Whatsmore,itpresentedanovelconceptionforthedeterminationoftheabscissionpoint,successfullyresolvedthelocationofcenterandabscissionpointwhenthefruitwerepartiallyoccluded.Meanwhile,bythetechniqueofgeometry,itsettledthelocationsoftheabscissionpointwhenthefruitgrewaskew.Itprovedgoodeffectunderthenaturescene.Keywords:Machinevision,fruitobject,recognition,location.1IntroductionDuringtheprocessofhumanconqueringtheNature,rebuildingtheNatureandpromotingthesociety,humansarefacingtheproblemofabilitylimitation.Asaresult,humanshavebeenseekingfortherobotstosubstitutethemantocompletecomplicatedtasks,andtheintelligentrobotisthebestchoice.Asweallknow,visionisthemainwayofhumansapperceivingtheworld.About80%informationisgotthroughvision.So,itisvitaltograntvisionfunctionforintelligentrobots.Here,wecandefinethemachinevisionasfollows:itisabletoproducesomedescriptionaboutthecontentoftheimageafterprocessingtheinputimage1.Therearemanyfieldsrelatedwithmachinevision.So,italsohasawideapplicationinvariousaspects,frommedicalimagetoremotesensedimage,fromindustrialinspectiontoagriculturalareas,etc.Thefruitandvegetableharvestingrobotwhichwearegoingtodiscussisonekindofautomaticmechanicalharvestingsystemspossessingtheperceptiveability,canbeprogrammedtoharvest,transferandpackthecrops2.Duringtheprocessofharvesting,thechiefproblemofthevisionsystemistorecognizeandlocatethefruitobject3.Here,recognitionmeanssegmentationofthefruitobjectsfromthecomplicatedbackground4.Andlocationincludestwoaspects:locationofthefruitcenterandabscissionpoint.786H.Guetal.Recently,thereremanyresearchesaboutfruitandvegetableharvestingrobotbasedonmachinevision56.CaiJian-rongpresentedthemachinevisionrecognitionmethodsunderthenaturescene.UsingtheOtsualgorithm,itgotthesegmentationthresholdautomaticallyandextractedthetarget7.Miyanagaintroducedtheseedinggraftingtechniquebasedonmachinevisionandtherobotinventedbythemhasbeenputintoproduction8.SlaughterD.Csetuponeorangeclassiermodelbyusingthecolorfeatureinthechromaticdigitalimage9.Amongtheseresearches,therehavebeenmanymethodsofextractingthefruitsfromcomplicatednaturescene.Butthebasicconceptionisextractingthefruitobjectbyconvertingonecolormodeltoanotheronewhichiseasiertoprocessormuchmoresuitableforthecase.However,still,therearetwoproblemsremainunsettled:1)Howtodeterminetheabscissionpointwhenthefruitsgrowaskew；2)Howtodeterminethecenterandabscissionpointwhentherearesomanyfruitoverlappedeachotherthatitisimpossibletodetectthewholeedge.Ifbothoftheproblemsremainunsettled,itmeanstheharvestingmaybeafailure.And,whatismoreimportant,thereisonlyabout40%ofthefruitandvegetableisvisibleintheorchard10,whichmeansabout60%objectsarepartiallyoccludedorcompletelyoccluded.Generally,theagriculturalrobotsarefitwithfanssoastoblowtheleavescoveringthefruit.So,forthefruitoccludedcompletely,itmaybepartiallyresolvedinthisway.So,inthepaper,weonlydiscussedtheproblemofthefruitpartiallyoccluded,inparticular,thecasethatonefruitoverlapanotherone.Asawhole,theproblemwearetodiscussbelongstothelowlevelmachinevision,andisoneofthekeystepsinthemachinevision.2MethodologyUsedinthePaper2.1MainIdeaFromtheanalysisabove,weknew,inordertosegmentthefruitfromleavesandbranches,weshouldusecolormodelsuitscertainsituations.TheRGBcolormodelcommonlyusedisnotsuitablefortheorchardimages.BecauseinRGBcolorspace,thetricolor(RGB)notonlyrepresentthehuevalue,butalsorepresentthebrightness.So,thechangeoftheoutwardilluminationmayaddthedifficultyoftherecognition,soRGBisundependableintheprocessofthesegmentation.Inordertomakeuseofthefruitsclusteringfeatureinhuespace,weneedtoseparatethehueandbrightnessinformation.WecanachievethisgoalbytransferringtheRGBtothemodelswhichseparatehueandbrightness.2.2ColorModelsWeusethreetypesofcolormodelsinthepaper.ThefirstoneisLCD(luminanceandcolordifference)model.Therearefourcolorattributesinthismodel,includingbrightnessinformationY,colordifferenceofred,Cr,colordifferenceofgreenCg,colordifferenceofblueCb.Thetransformformulaisasfollows:RecognitionandLocationofFruitObjectsBasedonMachineVision787=+=YBCYGCYRCBGRYbgr114.0587.0299.0.(1)Duringtheprocessofexperiment,wefoundthatthecolordifferenceofredoffruitismuchhigherthanthatofleavesorbranches,eventheunripefruit,suchasunripetomatothatwouldbereferredlater.SoweonlyhavetoconsideraboutthecolordifferenceofredCr.ThesecondmodelweusedisNormalizedRGB.Thediagramwasusedtorepresentthecolorpropertiesofthethreeportions.Thetransformformulaisdefinedasfollows:+=+=+=)/()/()/(BGRBbBGRGgBGRRr.(2)itisobviousitsatisfies:1=+bgr.Combinedtheadvantagesoftheabovetwomodels,wecanconcludethethirdcolormodelcalledLHMinthispaper.ChoosingYandCrfromthefirstcolormodel,randgfromthesecondmodel；wecanconstructtheformulaasfollow:+=+=+=)/()/(114.0587.0299.0BGRGgBGRRrYRCBGRYr.(3)3SegmentationUnderthenaturesceneoftheorchard,thefactorscontainingthenon-uniformillumination,theocclusionoftheleafandbranchallmakeitmoredifficulttosegment.Atpresent,wecanclassifythechromaticimagesegmentationintothreeclasses:(1)Segmentationbasedonthreshold；(2)Segmentationbasedonedgeinspectingandareagrowing；(3)Segmentationbasedoncolorclustering11.3.1ClusteringandClassifierTheprimaryconceptionofclusteringistodistinguishthedifferentobjectswhichincludedifferentclassesofobjectsanddifferentpartsofthesameobject12.Allclassificationalgorithmsarebasedontheassumptionthattheimageinquestiondepictsoneormorefeaturesandthateachofthesefeaturesbelongstooneofseveraldistinctandexclusiveclasses.Thetraditionalwayofclassifiercomprisestwophasesofprocess:trainingandtesting.Intheinitialtrainingphase,characteristicpropertiesoftypicalimagefeaturesareisolatedand,basedonthese,auniquedescriptionofeachclassificationcategory,i.e.trainingclass,iscreated.Inthesubsequenttestingphase,thesefeature-spacepartitionsareusedtoclassifyimagefeatures.788H.Guetal.Intheexperiment,wesampled60pixelsofleaf,branch,andfruitrespectivelyandconstructedaclassifier.Adoptingtwofeaturepatternsmandn,weformedthedecisionfunctions:cbnamnmf+=),(,wherea,b,andcarearbitraryconstantsaslongasthepointsonthelinesatisfiesthecondition0),(=nmf.Here,featurepatternmaybecolor,shape,size,oranypropertiesoftheobjects.Accordingtothedecisionfunctions0),(nmfor0),(nmf,wecandividetheimageintotwopartsasshowninFig1:.Fig.1.Modelofclassifier3.2SegmentationoftheFruitObjectsInthisstudy,weadoptedthesegmentationmethodofseveralthresholds.Thethresholdsarederivedfromtheabovethreemodelsoftheimageusingthedecisionfunctions.Accordingtotheaboveparagraphs,wecouldgetthreedecisionfunctions:thefirstfunction,F1