1、個(gè)人信息網(wǎng)絡(luò)數(shù)據(jù)庫一搜索引擎數(shù)據(jù)服務(wù)器搭建搜索引擎數(shù)據(jù)服務(wù)器框架圖:頊政航備器堡：|隊(duì)數(shù)藍(lán)聽熟而定朋查間心進(jìn)行刨建搜索成引七據(jù)遭常劇 ,Mysq5.1.62只進(jìn)行協(xié)政操作 INSLRT/DLLLTL.- UPDATEWeb !i iffliiid ShpfriK API iJl-JS 案U擎既拆器-.：& 濘搜賣鞍據(jù),集郡 將數(shù)據(jù)化灼品征返 I，.到將曲從數(shù)據(jù)時(shí)器L Myw|J5.1.62 富聽33MPHP5.3.13+ Apache 2.2 +CeiitOS 6.4用到的軟件：Coreseek / Sphinx：Sphinx支持高速建立索引（可達(dá)10 MB/秒，而Lucene建立索引的速度是

2、 1.8MB/秒）高性能搜索（在2-4 GB的文本上搜索，平均0.1秒內(nèi)獲得結(jié)果）高擴(kuò)展性（在單一 CPU上，實(shí)測(cè)最高可對(duì)100GB的文本建立索引，單一 索引可包含1億條記錄）支持分布式檢索支持基于短語和基于統(tǒng)計(jì)的復(fù)合結(jié)果排序機(jī)制支持任意數(shù)量的文件字段（數(shù)值屬性或全文檢索屬性）支持不同的搜索模式（“完全匹配”，“短語匹配”和“任一匹配”）支持作為Mysql的存儲(chǔ)引擎Coreseek是在sphinx的基礎(chǔ)上對(duì)中文進(jìn)行分詞的軟件包LibMMSeg：中文分詞Mysql：存貯數(shù)據(jù)PHP： 程序使用PHP寫的Apache： Web Server安裝軟件前的準(zhǔn)備：設(shè)置服務(wù)器網(wǎng)絡(luò)安裝相關(guān)軟件包yum -y

3、install gcc g+ gcc-c+ libjpeg libjpeg-devel libpng libpng-devel freetype freetype-devel libxml2 libxml2-devel zlib zlib-devel glibc glibc-devel glib2 glib2-devel bzip2 bzip2-devel ncurses ncurses-devel curl curl-devel e2fsprogs e2fsprogs-devel krb5 krb5-devel libidn libidn-devel openssl openssl-deve

4、l openldap openldap-devel nss_ldap openldap-clients openldap-servers patch libtool automake imake mysql-devel expat-devel安裝軟件包：1.安裝 MySQL+SphinxSEtar zxvf mysql-5.1.62.tar.gztar zxvf coreseek-4.1-beta.tar.gzcp -r coreseek-4.1-beta/csft-4.1/mysqlse/ mysql-5.1.62/storage/sphinxcd mysql-5.1.62./BUILD/a

5、utorun.sh./configure -prefix=/usr/local/mysql/ -enable-assembler -with-extra-charsets=complex -enable-thread-safe-client -with-big-tables -with-readline -with-ssl -with-embedded-server -enable-local-infile -with-plugins=partition,innobase,myisammrg,sphinx makemake install。省略若干配置步驟，和平時(shí)配置MySQL沒什么兩樣。安裝

6、完成啟動(dòng)MySQL后查看sphinx存儲(chǔ)引擎是否安裝成功在mysql命令行下執(zhí)行show engines;如果出現(xiàn)如下圖紅色方框內(nèi)的信息說明SphinxSE已經(jīng)安裝成功!+-+1Engine| Support | Comment| Transactions | XA | Sa_1_j11TTTTT1MRG_MYISAM | YES| Collection of identical My I SAM tables| NO| NO | NO1CSV-| YES| CSV storage engine| NO| NO | NO1MyISAM| DEFAULT | Default engine as

7、of MySQL 323 with great performance| NO| NO | NO1.TnncDR| YES|rcTtf-l evel locking, and foreign keys | YES| YES | YE：SPHINX| YES| Sphinx storage engine 0 . 9 .司| NO| NO | NO1MEMORY| YES| Hash basedr, useful for tensorary tables | NO| NO | NO4-+rows in set (0.00 sec)2.安裝Sphinx全文檢索服務(wù)器Sphinx默認(rèn)不支持中文索引及檢

8、索，以前用Coreseek的補(bǔ)丁來解決，目前 Coreseek不單獨(dú)提供補(bǔ)丁文件，而基于sphinx開發(fā)了 Coreseek全文檢索服務(wù)器， Coreseek應(yīng)該是現(xiàn)在用的最多的sphinx中文全文檢索，它提供了為Sphinx設(shè)計(jì) 的中文分詞包LibMMSeg包含mmseg中文分詞，其實(shí)coreseek-3.2.14.tar.gz 中已經(jīng)包含了 sphinx，前面安裝SphinxSE時(shí)也可以使用這個(gè)壓縮包里的 mysqlse。下邊是安裝過程：安裝 autoconftar zxvf autoconf-2.64.tar.gzcd autoconf-2.64./configure 一prefix=/

9、usrmakemake installcd .安裝 Coreseektar zxvf coreseek-3.2.14.tar.gzcd coreseek-3.2.14cd mmseg-3.2.14/./bootstrap./configure -prefix=/usr/local/mmseg3makemake installcd ./csft-3.2.14/sh buildconf.sh./configure -prefix=/usr/local/coreseek -without-python -without-unixodbc-with-mmseg -with-mmseg-includes

10、=/usr/local/mmseg3/include/mmseg/-with-mmseg-libs=/usr/local/mmseg3/lib/ -with-mysql -host=arm（不支持bigint）./configure -prefix=/usr/local/coreseek -without-python -without-unixodbc-with-mmseg -with-mmseg-includes=/usr/local/mmseg3/include/mmseg/-with-mmseg-libs=/usr/local/mmseg3/lib/ -with-mysql -host

11、=arm -enable-id64 （支持 bigint ）makemake install酉己置 Sphinx.conf, 進(jìn)入 cd /usr/local/coreseek/etc example.sql sphinx.conf.dist sphinx-min.conf.dist其中example.sql是示例sql腳本我們將其導(dǎo)入到數(shù)據(jù)庫中的test數(shù)據(jù)庫中作為測(cè)試數(shù)據(jù)（會(huì)創(chuàng)建兩張表documents和tags）vi sphinx.conf輸入以下內(nèi)容source src1（type=mysqlsql_host=localhostsql_user=rootsql_pass=1234567

12、8sql_db= testsql_port = 3306 # optional, default is 3306 sql_sock = /tmp/mysql.sock sql query pre = SET NAMES utf8 sql_querySELECT id, group_id, date_added, title, content FROM documents sql_attr_uint sql_attr_timestamp sql_query_infoid=$idindex test1(source path docinfo charset_type mlock=0morpholo

13、gy=nonemin_word_len = 1 html_strip=0charset_dictpath ngram_lenindexer(mem_limitUNIX_TIMESTAMP(date_added) ASgroup_id=date_added=SELECT * FROM documents WHEREsrc1/usr/local/coreseek/var/data/test1=externzh_cn.utf-8/usr/local/mmseg3/etc/=032Msearchd(port= 9312 TOC o 1-5 h z log=/usr/local/coreseek/var

14、/log/searchd.log query_log=/usr/local/coreseek/var/log/query.log read_timeout=5max_children=30pid_file=/usr/local/coreseek/var/log/searchd.pid max_matches=1000seamless_rotate=1preopen_indexes=0unlink_old說明：代碼段source src1*代表數(shù)據(jù)源里面主要包含了數(shù)據(jù)庫的配置信息， srcl表示數(shù)據(jù)源名字,可以隨便寫。代碼段index test1(*代表為哪個(gè)數(shù)據(jù)源創(chuàng)建索引，與source *是

15、成 對(duì)出現(xiàn)的，其中的source參數(shù)的值必須是某一個(gè)數(shù)據(jù)源的名字。其他參數(shù)可以查看手冊(cè)，這里不再贅述。生成索引/usr/local/coreseek/bin/indexer -c /usr/local/coreseek/etc/sphinx.conf -all其中參數(shù)-all表示生成所有索引當(dāng)然也可以是索引的名字例如：/usr/local/coreseek/bin/indexer -c /usr/local/coreseek/etc/ sphinx.conf test1執(zhí)行后可以在/usr/local/coreseek/var/data目錄中看到多出一些文件，是 以索引名為文件名的不同的擴(kuò)展名

16、的文件在不啟動(dòng)sphinx的情況下即可測(cè)試命令：/usr/local/coreseek/bin/search -c /usr/local/coreseek/etc/sphinx.conf numberro tsphinx data # /usr/1 ocal/coreseek/bin/search -c /msr/1 ocal/coreseek/etc/sphinx. conf nuroiber Coreseek Fulltext 3.2 Sphinx 0.9.9-release (r2117 Copyright (c 2007-2011, Beijing Choice Software Te

17、chnologies Inc ( using config file 1/usr/local/coreseek/etc/sphinx.conf1. index 1testl1: query nuriber 1: returned 3 matches of 3 total in 0.027 sec displaying matches:weight=lf group id=lf date added=Thu Jan 27 07:27:18 2011id=l一一groiip_id=l group_id2=5 date_added=2011-01-27 07:27:18 one content=th

18、is is my test document nunoiber one. also checking search within phrases.docuragntgf weight=lf group_id=lf date_added=Thu Jan 27 07:27:18 2011(id=2) group_id=l group_id2=6 date_added=2011-01-27 07:27:18 two content=this is my test document nuniber twodocumentf weight=lf group id=2f date added=Thu Ja

19、n 27 07:27:18 2011- - group_id=2 group_id2=8 date_added=2011-Ql-27 07:27:18 titledoc nuroiber four content=this is to test groupswords :1. nuuLber1 : 3 docxmients, Q3 Mitsl可以看到將內(nèi)容中含有number數(shù)據(jù)的數(shù)據(jù)查詢出來。過濾查詢/usr/local/coreseek/bin/search -c /usr/local/coreseek/etc/ sphinx.conf number -filter group_id 2限定

20、group_id為2返回一條記錄同樣也可以測(cè)試中文（需將命令行終端編碼調(diào)整為utf-8）/usr/local/coreseek/bin/search -c /usr/local/coreseek/etc/ sphinx.conf研究生創(chuàng)業(yè)root sphinx data # / usr/local/coreseek/bin/search -c / usr/local/coreseek/ etc/sphinx. conf 研究生倉 J業(yè) Coreseek Fulltext 3.2 Sphinx Q.9.9-release (r2117 Copyright (c) 2007-2011Beijing

21、 Choice Software Technologies Inc ( HYPERLINK )using config file 1/usr/local/coreseek/etc/sphinx.conf1.index * testl1 : query ,研充生倉業(yè) 1 : returned 0 matches of 0 total in 0.026 sec&JQ 1.研咒生 0 documents 0 hits2 . (,創(chuàng)亞p documents f hits可以看到我們輸入的查詢文字已經(jīng)被拆分成了兩個(gè)詞，只是因?yàn)槲覀兊?測(cè)試數(shù)據(jù)中沒有中文數(shù)據(jù)查詢結(jié)果為空。我們插入幾條新數(shù)據(jù)。INSERT

22、INTO test.documents （id ,group_id ,group_id2 ,date_added ,title ,content）VALUES （NULL , 2, 3, ,2011-02-01 00:37:12,研究生的故事，研究生自主創(chuàng)業(yè)）,（NULL , 1, 1, 2011-01-28 00:38:22,研究，為 了創(chuàng)業(yè)而研究生命科學(xué)）；我們?cè)賮砜匆韵聰?shù)據(jù)庫中的主要數(shù)據(jù)mysql select title, content from docuraents;+|id|title|content+|1|test one|thisismytestdocuraentnuroib

23、erone. also checking search within phrases. |2|test two|thisismytestdocuraentnuroibertwo|3|another doc|thisisanothergroup|4|doc nuroiberfour |thisistotestgroupsI 5 |研究生的故事|研究生自主創(chuàng)業(yè)I 6 |研究|為了創(chuàng)業(yè)而研究生命科學(xué)+6 rows in set (0.00 sec)插入新數(shù)據(jù)后需要重新生成索引/usr/local/coreseek/bin/indexer -c /usr/local/coreseek/etc/ sphinx.conf testl然后執(zhí)行查詢測(cè)試/usr/local/coreseek/bin/search -c /usr/local/coreseek/etc/sphinx.conf研究生創(chuàng)業(yè)displaying matches:docuiuerit=5f weight =2 f group id=2 f date added=Tue Feb 1 00:37： 12 2