1、考慮表中二元分類問(wèn)題的訓(xùn)練樣本集表4-8練習(xí)3的數(shù)據(jù)集實(shí)例小th目標(biāo)類1TTLQ+2TT6.0+13TF£04FF4Q4-5FT7.06FT3.07FF8.08TF7.0+9FT5gMM-1.整個(gè)訓(xùn)練樣本集關(guān)于類屬性的嫡是多少？2.關(guān)于這些訓(xùn)練集中a1,a2的信息增益是多少？3.對(duì)于連續(xù)屬性a3,計(jì)算所有可能的劃分的信息增益。4.根據(jù)信息增益，a1,a2,a3哪個(gè)是最佳劃分？5.根據(jù)分類錯(cuò)誤率，a1,a2哪具最佳？6.根據(jù)gini指標(biāo)，a1,a2哪個(gè)最佳？答1.ExamplesforcomputingEntropyEntropyt=一工(j|f)1。"p(j/)010C26

2、J.餐P(C1)=0/6=0P(C2)=6/6=1Entropy=-0log0-1Iog1=-0-0=0P(C1)=1/6P(C2)=5/6Entropy=-(1/6)log2(1/6)-(5/6)log2(56)=0.65P(C1»=2/6P(C2)=4/6Entropy=-6)1。的(2/6)-(4/6)loq?(46)=0.92P(+)=4/9andP(-)=5/9-4/9log2(4/9)-5/910g2(5/9)=0.9911.答2:SplittingBasedonINFO.InformationGainGAIN=Entmpy(p)-XEnfropv(i)<1YlPa

3、rentNodepissplitintokpartitions;nisnumberofrecordsinpartitioni- M巳asu巳sReductioninEntropyachievedbecauseofthesplit.Choosethesplitthatachievesmostreduction(maximizesGAIN)- UsedinID3andC4.5- Disadvantage:Tendstoprefersplitsthatresultinlargenumberofpartitions,eachbeingsmallbutpure,（估計(jì)不考）Foraltributeai.

4、thvcurrvspotidingtxn山出andproLabililkriart:<11十-TF3114Tlicentroj)yforaiis41-(3/1)l喉田4)(1")1唯11/4)+-(4/5)g(4/5)=07616.>>Tbrrpforp.thrinformationgainforisQ9911-0,7616=0,2294.ForAttributea2-.th也corrrsotidingcountsundprobfihilid<sarc:di十-JF922Tlirrutropyfbra*is7J-(2/5)1%S)-3/b)1%(3/3+:-(

5、2/4)log3(2/4)-(2/4)fog2(2/4)=U.皎9.Thrrrfbrr.thrinformati-ongainfor做嗯。期-().9839=0W72.ContinuousAttributes:ComputingGiniIndex.Forefficientcomputation:foreachattribute,- Sorttheattributeonvalues- Linearlyscanthesevalues,eachtimeupdatingthecountmatrixandcomputingginiindex- Choosethesplitpositionthathast

6、heleastginiindexCheatSortedValuesYesNo地p|Nn|Ho|YesY&s丫腌|No|H01HoiNoTaxableIncome60|70|75|85I|95|100|120|126|2200.4200.4000.3750.3430.4170.400。迎0,343137504000.420GiniS'Tan.SleinbachXtJinarIntroductiontoDataMining4/18/20Q437agClassLibelSplitpointEntropyInfoGiinL0十2.0仇84M0.14273.03.50.98850.002

7、64X14-4.5o.gisjOXT7285.03。-5.50.98300.00726.0+6.50.972S0.01837.(17.(1+工5088即0.1022aiproducesthebestsplit.答4:Accordingtoinformationgain,答5:ExamplesforComputingErrorCl0C26Forattributeai:errorrate=2/9.Forattributea2：errorrate=4/9.Therefore,accordingtoerrorrate,Error(f)=1maxP(it)P(C1)s0/6S0P(C2)=6/B=1Er

8、rors1-max(0,1)=1-1=0P(C1)=1/6P(C2)=5/6Error=1-max(1/6,5/6)=1-5/6=1/6P(C1)=2/6F(C2)=4/6Error=1-max(2/6,4J6)=1-4/6=1/3a1producesthebestsplit.答6:BinaryAttributes:ComputingGINIIndexSplitsintotwopartitionsEffectofWeighingpartitions:Gini(Children)=7/12*0408+5/12*0.32=0371-LargerandPuerPartitionsaresoughtf

9、or.Gini(N1)=1-(5/7)2-(2/7)2=0.408Gini(N2)=1一(1百尸一(4f5)?=0.32STan.SlenbachKLimarIntrodU3tianteDataMiniraForattribute硝'the疝niindexisJprrr，1-(3/4產(chǎn)一(1/4/+31-(1/5尸一(I*)?=03414.，L向IForartribiitrdi.theginiindrxis54-1-(2/5/-(3/5)241-(2/4產(chǎn)-(2/4)2=0.488&.LiISincetheginiindexfor%is-mailer,itproducesthe

10、bettersplit.二、考慮如下二元分類問(wèn)題的數(shù)據(jù)集AB類標(biāo)號(hào)TTTTTFFF1TTFTTFTFPFTF十-4-+1.計(jì)算a.b信息增益，決策樹(shù)歸納算法會(huì)選用哪個(gè)屬性HhocontingcncptablesAftersplittingonatrribntpsAand13are:4-3115B=TB=FTheoverallentropytx?ioresplittingis;E口ng=G.4log0.4-0-GlogO.C=0.9710Theinformation：ainafter口口Ais:E&=f313-3Lofi3-31Qg-7/10£Ihrin(orniatien：a

11、inaftersplitting0口Ris:E*t=一,1門g一:I%;=E81134444Kr=f=：】促',1但二通6500=Em."10E人T一6/10Eh尸二力.如倘Thrfor*,attribute.4wilXhospnt<isplitthenode.2.計(jì)算a.bgini指標(biāo)，決策樹(shù)歸納會(huì)用哪個(gè)屬性?Th<?overaJIxt口resplittin旦is1Gsy=1-0.12一。6,=0.48ThegaininginiaftersplittingouAis:-A=TY)1G)Gb©-7/10Ga=t-3/IOGf=0.1371Thegaini

12、nginiaftersplittingonRis:_(1)一G)=037502NG舊二F1=(I)-(0=0277SGig4/10G£j=tG/lUGi(=r=0.1G33'Therefore,attributeBwillbechosentosplitthenode.這個(gè)答案沒(méi)問(wèn)題3.從圖4-13可以看出嫡和gini指標(biāo)在0,0.5都是單調(diào)遞增，而0.5,1之間單調(diào)遞減。有沒(méi)有可能信息增益和gini指標(biāo)增益支持不同的屬性？解釋你的理由Yes,eventhoughthesemeasureshavesimilarrangeandmonotonousbehavior,theirre

13、spectivegains,A,whicharescaleddifferencesofthemeasures,donotnecessarilybehaveinthesameway,asillustratedbytheresultsinparts(a)and(b).貝葉斯分類ExampleofNaiveBayesClassifierGiv&naTestRecord:X=(Refond=No,Married,Income=12OK)naiveBayesClasBifier:pReruna=Yes|No)=3/7Petund=Jo|oj=4'?：¥&|¥t

14、b-CPWStaius-SinaNo)=27PiAlarilaISlalut-DivorcedNop1J7PNiamaistams=waTn|No)=4/7P(MaMaisialus=singie|Yes)=2f7P(VIarttaiStalus=Dlvorceq¥es)=1/7P(MariiajStah£=Mamed|Yes)=0Fa吐1幅incomeif0simplermi印一口samplevari3nce=?g75ifciass=Yes-samplemean=sosamplevanance=25FX|CIass-Noj=P(Refund=No|Class=No)kP(

15、Mlarried|Class=No)<P(lnccine-1?0K|Class=No)=4/7h4/7上0.0072=0,0024P(X|Cl8ss=Yes)=P(Refund=ND|Class=Yes)xLiamedClass=Yes)*P(ricome=120K|Class=Yes)二1xQx12x10*=0SiresP(X|No；PNq)>P(X|¥es)PYss)ThereforeP(No|X)>PfYes|X)=>Class=No呂Tmn.5明門1曲二如討麗心口3怕Minin。6$工考慮表5-10中的數(shù)據(jù)集.親門再睡7的數(shù)據(jù)悠記錄ABc類10001+

16、200L1301La0I«1500+610171D1IS110I一911+1010i十(a)估計(jì)條件概率尸+),尸(0+),嚴(yán)(+)，r(A|-)尸煙一)和尸(牛力根據(jù)6)中的條件概率，使用樸素貝葉斯方法預(yù)測(cè)測(cè)試樣本供=Q,B=1,C=0)的類標(biāo)號(hào).©使用m相計(jì)方法5=1/2且a=4)估計(jì)條件概率0d)同心)，使用(c)中的條件概率Ce)比較估計(jì)概率的兩種方法，哪一種更好？為什么？1.P(A=1|-)=2/5=0.4,P(B=1|-)=2/5=0.4,P(C=1|-)=1,P(A=0|-)=3/5=0.6,P(B=0|-)=3/5=0.6,P(C=0|-)=0;P(A=1|

17、+)=3/5=0.6,P(B=1|+)=1/5=0.2,P(C=1|+)=2/5=0.4,P(A=0|+)=2/5=0.4,P(B=0|+)=4/5=0.8,P(C=0|+)=3/5=0.6.Let"=o.B=l.rf=AH+H=z?=i,r-G)_PA=0,Z?=1,C=XF(+)_rA=o,r?=()_-0|+)JT(£?-1|+)F(C=0|+x?(十)一7?=0.-1xCh2x0i6x0.5/CA=(F,A?=EC'=0)r(A=U.白=l.C=0|-)x7?(-)P(A=(I.=T,C=()r(.i=o|-)xp(b=i|-)xr(c=o|-)xr(K=0

18、/AF2.TheclassbibcEdiauldbe4-3. P(A=0|+)=(2+2)/(5+4)=4/9,P(A=0|-)=(3+2)/(5+4)=5/9,P(B=11+)=(1+2)/(5+4)=3/9,P(B=11-)=(2+2)/(5+4)=4/9,P(C=0|+)=(3+2)/(5+4)=5/9,P(C=0|-)=(0+2)/(5+4)=2/9.4. LetP(A=0,B=1,C=0)=K=0.B=17<7=C)_P(_A=1曰=Q=xP(-l-)r(A=o,/?=i,c=o)p(a="+>尸(C=乂尸(+)(4/Q)x(3/9)x(S/9)x0,5=K=0

19、.0412/KA=U,臼=I,=U=0,/?=1.=n|)x)=P(A-U；£J-1,C-0)PA=U|-Jx尸(歸=l|-Jx/J(C=(J|-)x尸(一)K：(5/()x(4/Q)x(2/Q)x0.5二K=O.O2T4-KTheclasslabelshouldbe5.當(dāng)?shù)臈l件概率之一是零，則估計(jì)為使用m-估計(jì)概率的方法的條件概率是更好的，因?yàn)槲覀儾幌Ｍ麄€(gè)表達(dá)式變?yōu)榱恪?.考慮表5-11中的數(shù)據(jù)集.315-11習(xí)題8的數(shù)據(jù)集實(shí)例A8c1類110Q1-2101*3010一14100w|5101;|6001十7110,s-80I009010十101114估計(jì)條件概率P(A=1R),產(chǎn)

20、(B=小)，P(C=lk%P伍=1卜)，P(B=1|-P(C=1H*(b)根據(jù)中的條件概率，使用樸素貝葉斯方法預(yù)測(cè)測(cè)試樣本5=1,的類標(biāo)號(hào)。(c)比較/(月=1),P(B=I)和/An1,8=1),陳述A.3之間的關(guān)系。(d)對(duì)尸俵=】)，P(fl=0)和?(A=LB=0)重復(fù)9)的分析。<c)比較尸(4=1,占=】|類=+)與P(A=1|類=十)和P(B=1|英三十)給定英)，受盤人.B條件獨(dú)立嗎？1. P(A=11+)=0.6,P(B=11+)=0.4,P(C=11+)=0.8,P(A=1|-)=0.4,P(B=1|-)=0.4,andP(C=1|-)=0.22.LetR:(A=1,

21、B=1,C=1)bethetestrecord.Todetermineitsclass,weneedtocomputeP(+|R)andP(-|R).UsingBayestheorem,P(+|R)=P(R|+)P(+)/P(R)andP(-|R)=P(R|-)P(-)/P(R).SinceP(+)=P(-)=0.5andP(R)isconstant,RcanbeclassifiedbycomparingP(+|R)andP(-|R).Forthisquestion,P(R|+)=P(A=1|+)XRB=1|+)*F(C=11+)=0.192P(R|-)=P(A=1|-)XRB=1|-)XIC

22、=11-)=0.032SinceP(R|+)islarger,therecordisassignedto(+)class.3.P(A=1)=0.5,P(B=1)=0.4andP(A=1,B=1)=P(A)xP(B)=0.2.Therefore,AandBareindependent.4.P(A=1)=0.5,P(B=0)=0.6,andP(A=1,B=0)=P(A=1)xRB=0)=0.3.AandBarestillindependent.5.CompareP(A=1,B=11+)=0.2againstP(A=11+)=0.6andP(B=1|Class=+)=0.4.Sincetheprod

23、uctbetweenP(A=11+)andP(A=11-)arenotthesameasP(A=1,B=11+),AandBarenotconditionallyindependentgiventheclass.三、使用下表中的相似度矩陣進(jìn)行單鏈和全鏈層次聚類。繪制樹(shù)狀況顯示結(jié)果，樹(shù)狀圖應(yīng)該清楚地顯示合并的次序。Table8.1.SimilaritymatrixforExercise16.(a)Singlelink.Ib)C'oinpleielink.822*.0oo-=-8-10210210cbd-f)=*bd=0,2"02(18=1。%=25%2.考慮表a22中顯示的數(shù)據(jù)

24、集s裹622購(gòu)物該事劣的例子r成琴a事務(wù)ID煙獎(jiǎng)項(xiàng)-iQOOL位/禺TO242(MJ12口內(nèi)ME210031國(guó)中間3W1530022他“14CO291<44(KM阿融510033Md同51MK38忸由.將每個(gè)事務(wù)I視為一個(gè)購(gòu)物籃亂明他卻書(shū)也d©!計(jì)算項(xiàng)案和他4或的支持度.（W使用g）的計(jì)算結(jié)果，計(jì)算美聯(lián)規(guī)則他df£利伍一伽刈的置信度，置信度是對(duì)稱的度量嗎？（C）將每個(gè)原客TD作為一個(gè)購(gòu)物簸，重復(fù)值）.應(yīng)當(dāng)將每個(gè)項(xiàng)看作一個(gè)二元變量（如果一個(gè)項(xiàng)在欣客的購(gòu)買事務(wù)中至少出現(xiàn)了一次，則為h古則，為G.（d）使用代）的計(jì)算結(jié)果，計(jì)算美聯(lián)規(guī)則比用一色相1打一（也旬的置信度,（fi）

25、假定酊和是將每個(gè)事務(wù)ID作為一個(gè)購(gòu)物籃時(shí)關(guān)聯(lián)規(guī)則，的支持度和置信度，而歙利辦是將每個(gè)顧客ID作為一個(gè)購(gòu)物18時(shí)關(guān)聯(lián)提則r的支持度和置信度*討論物和J,或白和G之間是否存在某種關(guān)系？N(xronHdriircinnorasvmTnrtricniraurc5(H)3M)Therearenoapparentrelationshipsbetweensi,s2,ci,andc2.6.考慮表6-23中顯示的購(gòu)物籃事務(wù).表6城3購(gòu)物籃事務(wù)1浮購(gòu)買項(xiàng)i234567B910（牛初，M酒,尿布）i面包，貨油,牛奶）（牛奶，尿布，加干回包.黃浦,蝌干f理酒.伊干，尿布牛奶.球布.m包，黃濁）i面包.黃袖，尿布】啤泅

26、展布1牛奶，尿布，面包.黃油）曄工餅干Ca）從這基數(shù)據(jù)中,能夠提取出的關(guān)聯(lián)規(guī)則的最大數(shù)量是多少（包括零支持度的規(guī)則）？（b）能夠提取的頻繁項(xiàng)集的最大長(zhǎng)度是多少（假定最小支持度0）?（0寫出從該數(shù)據(jù)集中能夠梃取的務(wù)項(xiàng)集的最大數(shù)量的表達(dá)式“（d）找出一個(gè)具有最大支持度的項(xiàng)集（長(zhǎng)度為2或更大）.（e）找出一對(duì)項(xiàng)日和也使得規(guī)則。一與和可一內(nèi)具有相同的置信度中(a) What,isthrniaxinmmnunitxrofaxsociarionrulesrhatcanIk1extracted6romthisdata(IncludingrillesthathavezerorapportJ?Answer:Th

27、rparpsixitpnisinthedataset.Ttierrforpth-ptotalnirmbci'ofrulesis儀理.(b) Whatisthemaximumsizeoffrequentitemsetethatcanbeextracted；3i<riLiiiiiriininsup>011Answer:Bocaiisrthe口口transacrioncontains1items,tlicmaxi-niuiEtiiAcuEfreqlientitetnsttif4.(c) Writeancxprcsnidii£orthvinaximuiuncuiiibcr

28、ofsine-3itcnisrtstliatcanbederivedfromtinsdat.aset,Aitswcr:(；)=肛(d) Findanitemwt(of哥a2orlargrTrhntliasthelargfFtsupport.Answer:(Bread.Buittr).(e) Findapairofiteuis.uand6.suulithatdierulesa-*葉and|b1ajntiavoth*1saTjirmnfidenraAnswer:：Bccr.Coulics3(Breail：Buller8. 論理算法使用產(chǎn)生一計(jì)數(shù)的策略找出赧繁項(xiàng)集,通過(guò)合并一對(duì)大小為無(wú)的頻繁項(xiàng)集得到一個(gè)大小為阱1的候選項(xiàng)集1稱作候選產(chǎn)生