MachineLearningLecture1Introduction1MotivatingProblemsHandwrittenCharacterRecognition2MotivatingProblemsFingerprintRecognition(e.g.,bordercontrol)3MotivatingProblemsFaceRecognition(securityaccesstobuildingsetc)4CanMachinesLearntoSolveTheseProblems?Or,tobemorepreciseCanweprogrammachinestolearntodothesetasks?5DefinitionofLearningAcomputerprogramissaidtolearnfromexperienceEwithrespecttosomeclassoftasksTandperformancemeasureP,ifitsperformanceattasksinT,asmeasuredbyP,improveswithexperienceE

(Mitchell,MachineLearning,McGraw-Hill,1997)6DefinitionofLearningWhatdoesthismeanexactly?HandwritingrecognitionproblemTaskT:RecognizinghandwrittencharactersPerformancemeasureP:percentofcharacterscorrectlyclassifiedTrainingexperienceE:adatabaseofhandwrittencharacterswithgivenclassifications7DesignaLearningSystemWeshallusehandwrittenCharacterrecognitionasanexampletoillustratethedesignissuesandapproaches8DesignaLearningSystemStep0:Letstreatthelearningsystemasablackbox9LearningSystemZDesignaLearningSystemStep1:CollectTrainingExamples(Experience).Withoutexamples,oursystemwillnotlearn(so-calledlearningfromexamples)10236789DesignaLearningSystemStep2:RepresentingExperienceChoosearepresentationschemefortheexperience/examplesThesensorinputrepresentedbyann-dvector,calledthefeaturevector,X=(x1,x2,x3,…,xn)11(1,1,0,1,1,1,1,1,1,1,0,0,0,0,1,1,1,1,1,0,….,1)64-dVector(1,1,1,1,1,1,1,1,1,1,0,0,1,1,1,1,1,1,1,0,….,1)64-dVectorDesignaLearningSystemStep2:RepresentingExperienceChoosearepresentationschemefortheexperience/examplesThesensorinputrepresentedbyann-dvector,calledthefeaturevector,X=(x1,x2,x3,…,xn)Torepresenttheexperience,weneedtoknowwhatXis.SoweneedacorrespondingvectorD,whichwillrecordourknowledge(experience)aboutXTheexperienceEisapairofvectorsE=(X,D)12DesignaLearningSystemStep2:RepresentingExperienceChoosearepresentationschemefortheexperience/examplesTheexperienceEisapairofvectorsE=(X,D)So,whatwouldDbelike?Therearemanypossibilities.13DesignaLearningSystemStep2:RepresentingExperienceSo,whatwouldDbelike?Therearemanypossibilities.Assumingoursystemistorecognise10digitsonly,thenDcanbea10-dbinaryvector;eachcorrespondtooneofthedigits14D=(d0,d1,d2,d3,d4,d5,d6,d7,d8,d9)e.g,ifXisdigit5,thend5=1;allothers=0IfXisdigit9,thend9=1;allothers=0DesignaLearningSystemStep2:RepresentingExperienceSo,whatwouldDbelike?Therearemanypossibilities.Assumingoursystemistorecognise10digitsonly,thenDcanbea10-dbinaryvector;eachcorrespondtooneofthedigits15X=(1,1,0,1,1,1,1,1,1,1,0,0,0,0,1,1,1,1,1,0,….,1);64-dVectorD=(0,0,0,0,0,1,0,0,0,0)X=(1,1,1,1,1,1,1,1,1,1,0,0,1,1,1,1,1,1,1,0,….,1);64-dVectorD=(0,0,0,0,0,0,0,0,1,0)D=(d0,d1,d2,d3,d4,d5,d6,d7,d8,d9)DesignaLearningSystemStep3:ChooseaRepresentationfortheBlackBoxWeneedtochooseafunctionFtoapproximatetheblockbox.ForagivenX,thevalueofFwillgivetheclassificationofX.ThereareconsiderableflexibilitiesinchoosingF16LearningSystemFF(X)XDesignaLearningSystemStep3:ChooseaRepresentationfortheBlackBoxFwillbeafunctionofsomeadjustableparameters,orweights,W=(w1,w2,w3,…wN),whichthelearningalgorithmcanmodifyorlearn17LearningSystemF(W)F(W,X)XDesignaLearningSystemStep4:Learning/AdjustingtheWeightsWeneedalearningalgorithmtoadjusttheweightssuchthattheexperience/priorknowledgefromthetrainingdatacanbelearnedintothesystem:18E=(X,D)F(W,X)=DDesignaLearningSystemStep4:Learning/AdjustingtheWeights19LearningSystemF(W)F(W,X)XDE=(X,D)Error=D-F(W,X)AdjustWDesignaLearningSystemStep5:Use/TesttheSystemOncelearningiscompleted,allparametersarefixed.AnunknowninputXispresentedtothesystem,thesystemcomputesitsansweraccordingtoF(W,X)20LearningSystemF(W)F(W,X)XAnswerRevisionofSomeBasicMathsVectorandMatrixRowvector/columnvector/vectortranspositionVectorlength/normInner/dotproduct內(nèi)積/點(diǎn)積Matrix(vector)multiplicationLinearalgebra線性代數(shù)Euclideanspace歐式空間BasicCalculus基本微積分PartialderivativesGradientChainrule21RevisionofSomeBasicMathsInner/dotproductx=[x1,x1,…,xn]T

,y=[y1,y1,…,yn]TInner/dotproductofxandy,xTyMatrix/Vectormultiplication22RevisionofSomeBasicMathsVectorspace/EuclideanspaceAvectorspaceVisasetthatisclosedunderfinitevectoradditionandscalarmultiplication.Thebasicexampleisn-dimensionalEuclideanspace,whereeveryelementisrepresentedbyalistofnrealnumbersAnn-dimensionalrealvectorcorrespondstoapointintheEuclideanspace.[1,3]isapointin2-dimensionalspace[2,4,6]ispointin3-dimensionalspace23RevisionofSomeBasicMathsVectorspace/EuclideanspaceEuclideanspace(Euclideandistance)Dot/innerproductandEuclideandistanceLetxandyaretwonormalizednvectors,||x||=1,||y||=1,wecanwriteMinimizationofEuclideandistancebetweentwovectorscorrespondstomaximizationoftheirinnerproduct.Euclideandistance/innerproductassimilaritymeasure24RevisionofSomeBasicMathsBasicCalculusMultivariablefunction:Partialderivative:givesthedirectionandspeedofchangeofy,withrespecttoxiGradientChainrule:Lety=f(g(x)),u=g(x),thenLetz=f(x,y),x=g(t),y=h(t),then25FeatureSpaceRepresentingrealworldobjectsusingfeaturevectors26Ellipticalblobs(objects)12345678910111213141516x2(i)x1(i)iX(i)=[x1(i),x2(i)]x1x2x2(i)x1(i)FeatureSpaceFeatureVectorFeatureSpace27FromObjectstoFeatureVectorstoPointsintheFeatureSpacesEllipticalblobs(objects)12345678910111213141516x1x2X(1)X(3)X(7)X(8)X(15)X(16)X(25)X(6)X(12)X(13)X(4)X(14)X(9)X(11)X(10)RepresentingGeneralObjects28FeaturevectorsofFacesCarsFingerprintsGesturesEm