Agricultural Information Technology in Asia and Oceania 1998 193 - 193 1998 by The Asian Federation for Information Technology in Agriculture- 193 -AbstractAutomation techniques inagricultural field had been developedrapidly. But, the techniques do not stillreached a level of factory automation.There are several problems inagriculture field. For example,agricultural operations are morecomplicate than industrial one. Sinceshapes of each organs are different fromeach other, image sensors andrecognition systems are required to getfruits position well.Until now, 2- and 3-dimentionalimage sensor, image recognition andshape modeling have been investigated.In our laboratory, a 3-dimentional visualsensor system was developed torecognize direction of strawberry fruit.This system is composed of three units.(1) a CCD camera for detecting the fruitsbased on the color image, (2) a rangefinder unit for recognizing direction offruits and measuring the distance and (3)computers for analyzing 2D- and 3D-data. This systemfinds out the 3D coordinates of fruit based on therecognizing data. Using the 3-dimentional visualsensor system, it is found that the shapes of the fruitsselected can be successfully recognized.To support the movement of gantry system, databaseconsidering daily changes in the fruits growth has beenconstructed. This database contains ripening fruits,unripe fruits, harvested fruits, fruit shape, color and 3Dcoordinates, and therefor can perform shipmentprediction.Thus, a database developed here enable us to make aplan for harvesting fruits by robot, and would be effectiveto increase the efficiency operations efficiency ofharvesting.* To whom correspondence should be addressed. Fax.: + 81-89-947 - 8748 ;E-mail: KenjiHATOUagr.ehime-u.ac.jpIntroductionIn greenhouse production, computer support systemsare introducing to control environmental conditions forautomations. Furthermore , many robots were studiedfor automation in the agricultural field. Harvestingrobots have been studied in particular . For theautomatic robot control, the recognizing system for thepositions of fruits and flowers is important. In theprevious paper, we made robot sensor system. But,previous system could not recognize very small fruitswhich colors were white and green. In this paper, wemade flowers and fruits database to find out these smallfruits. And we made recognition system for flowersand small fruits.Strawberry Cultivating SystemFigure 1 shows a 3-dimentional visual sensor systemon a gantry system. This system is composed of CCDcamera for input colors and 2-dimentional image data,range finder for input z-axis data and a work stage for toHarvesting Support Database based on Image RecognitionKenji Hatou*1), H. Matsuura2) and Y. Hashimoto1)1) Dept. of Biomechanical Systems, Ehime University, 790-8566, Japan2) Sikoku Research Institute Inc., Yashima- nishi 2109 8, Takamatsu, 761 0192, JapanCCD cameraRailGantryWheelWork stageTopviewSideviewRange finderX-axisY-axisZ-axisFigure 1 Gantry system and 3-dimentional visual sensor system.Hatou, et. al.- 194 -move on y-axis. The gantry system can move on therails (x-axis). The unit for image processing on thegantry system is a work stage in figure 2.Figure 2 shows a strawberry field. There are threeridges covered with mulching seats and a gantry systemmoving on the rails. These ridges are separated to total27 work stages for image processing. This unitmoves by turns on these ridges.The strawberries are planted as shown in figure1. Fruits and flowers are in center of ridges, andleafs are at near furrows side. Thus, unit ofimage processing can separate flowers and fruitsfrom 2-dimentional image data easily.Database for Harvesting SupportIn figure 3, left side block shows i mageprocessing system. This system is composed ofthree units. (1) a CCD camera for detecting thefruits based on the color image, (2) a range finderunit for recognizing direction of fruits andmeasuring the distance and (3) computers foranalyzing 2D- and 3D-data and controlling x-yrobot and range finder . This system finds out the3D coordinates of fruit based on the recognizingdata.In figure 3, right side block shows databasesystem for harvest support. This system storesdate, numbers and position of flowers and fluits.It is defficult to detect green unripe fruits fromleafs and stems based on color detection program,because of these are same color. Then, thisestimation unit use flowers data, we can to detect greenunripe fruits easily.Image Processing MethodFigure 3 Image processing system and database system for harvest support.Figure 2 Ridges and gantry system in the strawberry fieldComputer roomPlant factoryHarvestestimation unitColor CCD camera Range finderRobot controland imageprocessing unitX-y robotDatabaseDateLabel and position1. Flower2. Unripe fruit (green)3. Unripe fruit (white)4. Mature fruitLANGantryRidgeRail Work stageHatou, et. al.- 195 -The databese is composed of four stage data areflower, green unripe fruit, white unripe fruit or maturefluit. These stages are desided by color conditionsbased on HSI ( hue, saturation, intensity ) data. Figure 4shows relationship of H SI color space and color solid.1. Frowers detection230 I 255 (1)0 S 0.05 (2)If equation (1) and (2) are satisfied then this point isfrower.2. Green unripe fruitsH = 120 o (3)When this point was flower and not white unripe fruit, ifequation (3) is satisfied then this point is green unripefruits.3. White unripe fruits230 I 255 (4)15 o H 50 o (5)When this point was green unripe fruit, if equation (4)and (5) are satisfied then this point is white unripe fruits.4. Mature fruits0 o H 15 o (6)When this point was white unripe fruit, if equation (6) issatisfied then this point is mature unripe fruits.Result and DiscussionWe can detect flowers and fruits data and stored intodatabase. The database developed here enable us tomake a plan for harvesting fruits, and would be effectiveto increase the efficiency operations efficiency ofharvesting.References1) H. Matsuura, K. Hatou, J. Yamashita and Y.Hashimoto: Recognition by robot visual sensor basedon sensor fusion -Fuse range image and 2-dimentional image- , Journal of SHITA: 9 (2):132138 (1997).2) K. Hatou, H. Matsuura, T. Sugiyama and Y.Hashimoto: Range Image Analysis for theGreenhouse Automation in Intelligent Plant Factory,Preprints of the 13th IFAC World Congress (B) :459 464 (1996) .3) T. Morimoto, K. Hatou and Y. Hashimoto:Intelligent control for plant production system ,Control Eng. Practice 4(6): 773-784 (1996).4) Susumu Tac