1、【原創(chuàng)】定制代寫開發(fā)r/python/spss/matlab/WEKA/sas/sql/C+/stata/eviews數(shù)據(jù)挖掘和統(tǒng)計分析可視化調(diào)研報告/程序/PPT等/爬蟲數(shù)據(jù)采集服務(附代碼數(shù)據(jù))，咨詢QQ：3025393450有問題到百度搜索“大數(shù)據(jù)部落”就可以了歡迎登陸官網(wǎng)：/teradatR語言ablone數(shù)據(jù)集數(shù)據(jù)挖掘預測分析報告介紹數(shù)據(jù)集描述檢測異常值并構(gòu)建清潔數(shù)據(jù)集清潔數(shù)據(jù)分析結(jié)論介紹鮑魚是鐵和泛酸的極佳來源，是澳大利亞，美國和東亞地區(qū)的營養(yǎng)食品資源和農(nóng)業(yè)。100克鮑魚每日攝取這些營養(yǎng)素的量超過20。鮑魚的經(jīng)濟價值與年齡呈正相關(guān)。因此，準確檢測鮑魚的年齡對于農(nóng)民和消費者確定其價格

2、非常重要。然而，目前用來決定年齡的技術(shù)是相當昂貴和低效的。農(nóng)民通常通過顯微鏡切割貝殼并計數(shù)環(huán)以估計鮑魚的年齡。這種復雜的方法增加了成本并限制了它的普及。我們的目標是找出預測戒指的最佳指標，然后找出鮑魚的年齡。數(shù)據(jù)集描述數(shù)據(jù)集描述在這個項目中，數(shù)據(jù)集Abalone是從UCIMachineLearningRepository(1995)獲得的。該數(shù)據(jù)集包含1995年12月由澳大利亞塔斯馬尼亞州主要工業(yè)和漁業(yè)部海洋研究實驗室Taroona記錄的4177只鮑魚的物理測量結(jié)果。有9個變量，分別是性別，長度，直徑和身高，體重，體重，內(nèi)臟重量，外殼重量和戒指。隨著年齡等于戒指數(shù)量，變量戒指與鮑魚年齡呈線性相

3、關(guān)加1.5。檢測異常值并構(gòu)建清潔數(shù)據(jù)集library(ggplot2)library(plyr)library(nnet)library(MASS)library(gridExtra)#Loadingrequiredpackage:gridlibrary(lattice)library(RColorBrewer)library(xtable)Data=read.csv(abalone.csv)#ImportDataprint(str(Data)#StructureoftheData#data.frame:4177obs.of9variables:#Sex:Factorw/3levelsF,I,

4、M:3313221131.#Length:num0.4550.350.530.440.330.4250.530.5450.4750.55.#Diameter:num0.3650.2650.420.3650.2550.30.4150.4250.370.44.#Height:num0.0950.090.1350.1250.080.0950.150.1250.1250.15.#Whole.weight:num0.5140.2260.6770.5160.205.#Shucked.weight:num0.22450.09950.25650.21550.0895.#Viscera.weight:num0.

5、1010.04850.14150.1140.0395.#Shell.weight:num0.150.070.210.1550.0550.120.330.260.1650.32.#Rings:int157910782016919.#NULL有4種不同的體重衡量標準，即Whole.weight，Shucked.weight，Viscera.weight和Shell.weight。Whole.weight應該是最容易測量的。所以我放棄了所有其他措施。Data=subset(Data,select=-c(Shucked.weight,Viscera.weight,Shell.weight)繪制不同戒指

6、的鮑魚數(shù)量。ggplot(Data,aes(x=Rings,fill=Sex)+geom_bar(binwidth=1,color=blue,origin=min(Data$Rings),position=identity)+scale_x_continuous(name=Rings,breaks=seq(0,30,by=2)+theme(axis.text.x=element_text(angle=90)+scale_fill_brewer(Location,type=qual,palette=3)+ylab(Numberofabalone)ggtitle(Numberofabalonewi

7、thdifferentRings)+facet_wrap(Sex,ncol=3)NumberofabalonewithdifferentRings6uoesCTB0alquJnN從上面的圖表中，我們可以看到戒指的范圍是從1到29，這可能太多以至于無法衡量。實際上，人們可能不需要這么詳細的類別。因此，我們將小于6環(huán)(14.5歲)的鮑魚進行分組，相應地表明年輕，成年和老年鮑魚，將它們標記為1,2,3。Age=c(rep(0,nrow(Data)for(iin1:nrow(Data)if(Datai,$Rings=7&Datai,$Rings13)Agei=3gData=cbind(Data,Age

8、)粗略地繪制圖表，并獲得關(guān)于數(shù)據(jù)的小嘗試ggplot(gData,aes(x=Height,y=factor(Rings),colour=factor(Sex)+geom_jitter(position=position_jitter(width=0.3)+geom_point()+ggtitle(OriginalData:HeightvsRings)+scale_colour_brewer(typeseq,palette=Set1)OriginalData:HeightvsRings2726-25-24factdrtSeiMl-*-Jn一J-22222t2D19lB171615ld.13_從

9、上面的圖表我們可以看到女性中有一些輪廓。我想擺脫他們。jData=subset(gData,Height0.4)ggplot(jData,aes(x=Height,y=factor(Rings),colour=factor(Sex)+geom_jitter(position=position_jitter(width=0.1)+geom_point()+ggtitle(ReducedData:HeightvsRings)+scale_colour_brewer(type=seq,palette=Set1)【原創(chuàng)】定制代寫開發(fā)r/python/spss/matlab/WEKA/sas/sql/C

10、+/stata/eviews數(shù)據(jù)挖掘和統(tǒng)計分析可視化調(diào)研報告/程序/PPT等/爬蟲數(shù)據(jù)采集服務(附代碼數(shù)據(jù))，咨詢QQ：3025393450有問題到百度搜索“大數(shù)據(jù)部落”就可以了歡迎登陸官網(wǎng)：/teradat【原創(chuàng)】定制代寫開發(fā)r/python/spss/matlab/WEKA/sas/sql/C+/stata/eviews數(shù)據(jù)挖掘和統(tǒng)計分析可視化調(diào)研報告/程序/PPT等/爬蟲數(shù)據(jù)采集服務(附代碼數(shù)據(jù))，咨詢QQ：3025393450有問題到百度搜索“大數(shù)據(jù)部落”就可以了歡迎登陸官網(wǎng)：/teradat【原創(chuàng)】定制代寫開發(fā)r/python/spss/matlab/WEKA/sas/sql/C+/

11、stata/eviews數(shù)據(jù)挖掘和統(tǒng)計分析告/程序/PPT等/爬蟲數(shù)據(jù)采集服務（附代碼數(shù)據(jù)），咨詢QQ：3025393450有問題到百度搜索“大數(shù)據(jù)部落”就可以了歡迎登陸官網(wǎng)：/teradat可視化調(diào)研報ReducedData:HeightvsRings29-=1000,sousinggamwithformula:ys(x,bs=cs).Usemethod=xtochangethesmoothingmethod.2wnoie.wfiightggplot(Data,aes(x=Length,y=Height)+geom_point(aes(colour=Rings)+scale_colour_g

12、radient(low=purple)+stat_smooth(colour=red)+ggtitle(LengthvsHeight)#geom_smooth:method=autoandsizeoflargestgroupis=1000,sousinggamwithformula:ys(x,bs=cs).Usemethod=xtochangethesmoothingmethod.ID；匚石LengthvsHeight04Lengmggplot(Data,aes(x=Length,y=Diameter)+geom_point(aes(colour=Rings)+scale_colour_gra

13、dient(low=purple)+stat_smooth(colour=red)+ggtitle(LengthvsDiameter)#geom_smooth:method=autoandsizeoflargestgroupis=1000,sousinggamwithformula:ys(x,bs=cs).Usemethod=xtochangethesmoothingmethod.LengthveDiameter從上圖可以明顯看出，長度，高度和直徑是相對線性相關(guān)的。事實上，認為鮑魚越大它們越重，所以我想檢查它是否完全相關(guān)Whole.weight和Volume之間是很自然的。構(gòu)建體積和Whole

14、.weight數(shù)據(jù)集yData=ddply(Data,Sex+Age,summarize,Volume=Length*Diameter*Height,Whole.weight=Whole.weight.Rings=Rings)繪制點和回歸線cols-c(Linear=#f04546.Cubic=#3591dl)ggplot(data=yData,aes(x=Whole.weight,y=Volume)+geom_smooth(method=lm,aes(colour=Linear),lwd=1)+geom_smooth(method=lm,formula=ypoly(x,3),aes(colo

15、ur=Cubic),lwd=1)+geom_point(alpha=1/10,colour=purple)+scale_colour_manual(TypeofRegression,values=cols)+ggtitle(VolumevsWhole.weight)+coord_flip()+scale_y_continuous(name=Volume,breaks=seq(0,0.15,by=0.03)【原創(chuàng)】定制代寫開發(fā)r/python/spss/matlab/WEKA/sas/sql/C+/stata/eviews數(shù)據(jù)挖掘和告/程序/PPT等/爬蟲數(shù)據(jù)采集服務(附代碼數(shù)據(jù))，咨詢QQ：3

16、025393450有問題到百度搜索“大數(shù)據(jù)部落”就可以了歡迎登陸官網(wǎng)：/teradat統(tǒng)計分析可視化調(diào)研報【原創(chuàng)】定制代寫開發(fā)r/python/spss/matlab/WEKA/sas/sql/C+/stata/eviews數(shù)據(jù)挖掘和統(tǒng)計分析可視告/程序/PPT等/爬蟲數(shù)據(jù)采集服務（附代碼數(shù)據(jù)），咨詢QQ：3025393450有問題到百度搜索“大數(shù)據(jù)部落”就可以了歡迎登陸官網(wǎng)：/teradat化調(diào)研報VolumevsWhole.weightTypeofRegreBsicinCutoK-Linear(1.03O.d6volume0.090.從上圖可以看出，立方回歸和線性回歸在開始時幾乎是相同的，

17、但當體積較大時分歧很大，這意味著體積和整體重量對于小鮑魚高度相關(guān)。為什么不使用Whole.weight來預測鮑魚的戒指？計算不同性別的Whole.weight的均值和方差。with(Data,do.call(rbind,tapply(Whole.weight,Sex,function(x)c(M=mean(x),SD=sd(x)#MSD#F1.04690.4303#I0.43140.2863#M0.99070.4697看起來F和M比我重。這是否意味著我的團隊沒有大戒指？ggplot(data=Data,aes(x=Rings,group=Sex,colour=Sex)+geom_density

18、()+ggtitle(DensityofRingsofeachSex)ir10囲Rings0.15-DensityofRingsofeachSex10性別我的鮑魚比F和M組的年輕，但它確實有性別I的舊鮑魚。colourggplot(data=Data,aes(x=factor(Age),y=Whole.weight)+geom_boxplot(outlier.=purple,outlier.size=3,aes(fill=Sex)+facet_wrap(Sex,ncol=3)+ggtitle(Whole.weightvsAgefordifferentSex)+xlab(Age)Whole.we

19、ightv&AgefordifferentSexSex申L1i與M和F相比，似乎鮑魚與性別的重量更輕。擬合多國回歸模型logit-multinom(AgeWhole.weight,data=yData)#weights:9(4variable)#initialvalue4586.706305#iter10value2099.141042#finalvalue2088.269847#convergedsummary(logit)#Call:#multinom(formula=AgeWhole.weight,data=yData)#Coefficients:#(Intercept)Whole.we

20、ight【原創(chuàng)】定制代寫開發(fā)r/python/spss/matlab/WEKA/sas/sql/C+/stata/eviews數(shù)據(jù)挖掘和統(tǒng)計分析可視化調(diào)研報告/程序/PPT等/爬蟲數(shù)據(jù)采集服務(附代碼數(shù)據(jù))，咨詢QQ：3025393450有問題到百度搜索“大數(shù)據(jù)部落”就可以了歡迎登陸官網(wǎng)：/teradat【原創(chuàng)】定制代寫開發(fā)r/python/spss/matlab/WEKA/sas/sql/C+/stata/eviews數(shù)據(jù)挖掘和統(tǒng)計分析可視化調(diào)研報告/程序/PPT等/爬蟲數(shù)據(jù)采集服務(附代碼數(shù)據(jù))，咨詢QQ：3025393450有問題到百度搜索“大數(shù)據(jù)部落”就可以了歡迎登陸官網(wǎng)：/terad

21、at#21.8269.362#3-4.90010.551#Std.Errors:#(Intercept)Whole.weight#20.13530.4488#30.17810.4600#ResidualDeviance:4177#AIC:4185pp=fitted(logit)pred-predict(logit)table(pred,Data$Age)#pred123#12681080#21803127489#3021用上面的信息構(gòu)建一個新的數(shù)據(jù)集。newdata=with(yData,data.frame(Weight=Whole.weight,Sex=Sex,Pred_Prob=pp,A

22、ge=Age,Pred_Age=pred)head(newdata)#WeightSexPred_Prob.1Pred_Prob.2Pred_Prob.3AgePred_Age#10.0800F0.73650.25070.0127611#20.1400F0.61360.36640.0200211#30.1885F0.50130.47140.0272811#40.2405F0.38110.58300.0359012#50.2750F0.30780.65050.0417312#60.1700F0.54480.43080.0243911ggplot(newdata,aes(x=Pred_Age,y=

23、Age,colour=factor(Pred_Age=Age)+geom_point(position=jitter)+scale_colour_manual(AgeGroup,values=c(2,3),labels=c(WongClassification,RightClassification)+ggtitle(Classification)Classification2尸red_AgeAgeGroupWangClassirtiGalionRKjhlCnasfirlicaliiori繪制不同年齡組的預測概率cols-c(Young=blue,Adult=#f04546.Old=green

24、)ggplot(newdata,aes(x=Weight)+geom_line(aes(y=Pred_Prob.l,colour=Young)+geom_line(aes(y=Pred_Prob.2,colour=Adult)+geom_line(aes(y=Pred_Prob.3,colour=Old)+ylab(PredictedProbability)+facet_wrap(Sex,ncol=3)+scale_colour_manual(AgeGroup,values=cols)+ggtitle(PredictedProbability)三oPBV一？rL從圖表中我們可以看出，Young

25、組的預測應該是好的，但是當體重很重時，很難區(qū)分老人和成人。似乎更多的性別我鮑魚是錯誤分類為青年和成人組，但更多的性別鮑魚被誤分類為舊摸索。而最大的問題是老組。其中一個原因可能是我們采用多國回歸中的變量線性組合。如何使用線性判別分析？只使用Whole.weightLDA1=lda(AgeWhole.weight,yData)predl=predict(LDAl)$classnewdataL1=with(yData,data.frame(Weight=Whole.weight,Sex=Sex,Age=Age,Pred_Age1=pred1)Data_misL1=subset(newdataL1,A

26、ge!=Pred_Age1)使用Whole.weight和Volume。LDA2=lda(Age.,yData)pred2=predict(LDA2)$classnewdataL2=with(yData,data.frame(Weight=Whole.weight,Sex=Sex,Age=Age,Pred_Age2=pred2)Data_misL2=subset(newdataL2,Age!=Pred_Age2)繪制圖表：p1=ggplot(Data_misL1,aes(x=Age,fill=Sex)+geom_bar(binwidth=1/2,color=darkgrey)+ylab(Num

27、berofmis-classfieddata)+ggtitle(UsingWhole.weightonly)+theme(axis.text.x=element_text(angle=90)+scale_fill_brewer(Sex,type=qual,palette=3)p2=ggplot(Data_misL2,aes(x=Age,fill=Sex)+geom_bar(binwidth=1/2,color=darkgrey)+ylab(Numberofmis-classfieddata)+ggtitle(UsingWhole.weight&Volume)+theme(axis.text.x

28、=element_text(angle=90)+scale_fill_brewer(Sex,type=qual,palette=3)grid.arrange(p1,p2,main=MisClassifiedDataforDifferentSex_LDA)slnpPBUWBBVG一EQJBquunHUsingWhoe.woightonly通過以上分析，我們發(fā)現(xiàn)盡管Whole.weight與Volume有高度相關(guān)性，但Volume對分類問題有顯著影響。這是否意味著Volume對分類更重要？只使用音量LDA3=lda(AgeVolume,yData)pred3=predict(LDA3)$class

29、newdataL3=with(yData,data.frame(Weight=Whole.weight,Sex=Sex,Age=Age,Pred_Age3=pred3)Data_misL3=subset(newdataL3,Age!=Pred_Age3)繪制圖表：ggplot(newdataL2,aes(x=Pred_Age2,y=Age,colour=factor(Pred_Age2=Age)+geom_point(position=jitter)+scale_colour_manual(AgeGroup,values=c(2,3),labels=c(WongClassification,R

30、ightClassification)+ggtitle(ClassificationusingWhole.weight&Volume)AgeGroupWangClaaairticaloonRjt|h1Cnasfli&aliori2Pred_Age2Clas&ificatianusingWhole.weight&Volumeggplot(newdataL3,aes(x=Pred_Age3,y=Age,colour=factor(Pred_Age3=Age)+geom_point(position=jitter)+scale_colour_manual(AgeGroup,values=c(2,3)

31、,labels=c(WongClassification,RightClassification)+ggtitle(ClassificationUsingVolume)ClassificationUsingVolumePred_Agti3AgeGroupWangClassirtiGalionRjgpirtCnasrli&aliiori答案是不。分別使用Volume和Whole.weight不會給出好的分類。幾乎沒有結(jié)論：使用LDA可以更有效地對數(shù)據(jù)進行分類，然后進行多國回歸?？梢钥隙ǖ氖牵褂肳hole.weight和Volume的觀察結(jié)果，我們已經(jīng)可以很好地將鮑魚分類到不同的年齡組。那么，哪

32、個變量更重要，長度，直徑，高度還是Whole.weight?我們現(xiàn)在不想考慮卷,因為這個措施不能直接獲得。所以我們會回到上面四個容易測量的變量。記錄每個變量的最小值和最大值的新數(shù)據(jù)框。jData-ddply(Data,Sex+Rings,function(x)jLevels-c(min,max)Length=range(x$Length)Height=range(x$Height)Diameter=range(x$Diameter)Whole.weight=range(x$Whole.weight)return(data.frame(Length,Height,Diameter,Whole.w

33、eight,stat=jLevels)繪制圖表p_Height=ggplot(jData,aes(x=Rings,y=Height,group=stat,colour=stat)+geom_line()+geom_point()+facet_wrap(Sex,ncol=3)+ggtitle(Height:MinvsMin)+theme(legend.position=none)p_Length=ggplot(jData,aes(x=Rings,y=Length,group=stat,colour=stat)+geom_line()+geom_point()+facet_wrap(Sex,nco

34、l=3)+ggtitle(Length:MinvsMax)+theme(legend.position=none)p_Diameter=ggplot(jData,aes(x=Rings,y=Diameter,group=stat,colour=stat)+geom_line()+geom_point()+facet_wrap(Sex,ncol=3)+ggtitle(Diameter:MinvsMax)+theme(legend.position=none)p_Whole.weight=ggplot(jData,aes(x=Rings,y=Whole.weight,group=stat,colo

35、ur=stat)+geom_line()+geom_point()+facet_wrap(Sex,ncol=3)+ggtitle(Whole.weight:MinvsMax)tmp-ggplot_gtable(ggplot_build(p_Whole.weight)leg-which(sapply(tmp$grobs,function(x)x$name)=guide-box)legend-tmp$grobsleggrid.arrange(arrangeGrob(p_Whole.weight+theme(legend.position=none),p_Length,p_Height,p_Diam

36、eter)Whole.weight:MinvsMax二IHO10203DO2DanTO203DRingsLength:MinvsMaxRingsHeight:MinvsMinFIM左詈HsyInRlDameter:MinvsMaxFIM-Io42o.o.o.巨EQ!如嘗試xyplot中的類似事情px_Height=xyplot(HeightRings|Sex,jData,group=stat,auto.key=list(columns=x=0.35,y=0.85,corner=c(0,1),type=c(p,l),par.settings=list(superpose.line=list(co

37、l=c(red,blue),superpose.symbol=list(col=c(red,blue),grid=h)grid.arrange(arrangeGrob(pHeight+xlab(),pxHeight)從上圖可以看出，Whole.weight和Height分享了相似的范圍趨勢，而Length和Diameter分享了相似的范圍趨勢。這是否意味著Whole.weight與高度更緊密相關(guān),并且Length與Diameter更相關(guān)？回歸。jFun-function(x)name=c()Coefs=c()for(iin1:4)temp=c()temp-rbind(temp,as.numer

38、ic(coef(lm(x,i+1Rings,x)Coefs=rbind(Coefs,temp)name=rbind(name,names(xi+1)return(data.frame(Intercept=Coefs,1,Slope=Coefs,2,Variable=name)jCoefs-ddply(Data,Sex,jFun)foo-ddply(jCoefs,Variable,function(x)lerange-c(which.min(x$Intercept),which.max(x$Intercept)cbind(xlerange,c(Variable,Sex,Intercept,Slo

39、pe),stat=c(min_slope,max_slope)foo=xtable(foo)print(foo,type=html,include.rownames=FALSE)變量性別截距坡統(tǒng)計直徑世0.130.02min_slope直徑F0.390.01max_slope高度世0.040.01min_slope高度F0.120.00max_slope長度世0.190.03min_slope長度F0.510.01max_slopeWhole.weigh0.0min_slopet世0.198Whole.weighF0.640.0max_slopet4從上表可以看出，性別我對所有變量都有min_slope,對于所有變量我們都有max_slope。繪制高度的回歸線ggplot(subset(Data,Sex%in%c(F,I),aes(x=Rings,y=Height)+facet_wrap(Sex,ncol=2)+geom_smooth(method=lm,colour=purple)+geom_point(aes(colour=factor(Age)+ggtitle(Max_