1、Machine Learning：An Overview,石立臣,Outline,What is machine learning (ML) Types of machine learning Work flow Popular models Applications Futures,What is machine learning,Training set (labels known),Test set (labels unknown),f( ) = “apple” f( ) = “tomato” f( ) = “cow”,What is machine learning,Definitio

2、n Machine learning refers to a system capable of the autonomous acquisition and integration of knowledge Machine learning is programming computers to optimize a performance criterion using example data or past experience,Computer,Data,Algorithm,Program,Knowledge,Knowledge (new),What is machine learn

3、ing,Every machine learning algorithm has three components Representation Model (rules, statistics, instance; logic, KNN, SVM, DNN,) Evaluation Performance (accuracy, mse, energy, entropy,) Optimization Parameters Combinatorial optimization Convex optimization Constrained optimization,Types of machin

4、e learning,Supervised learning Training data includes desired outputs Unsupervised learning Training data does not include desired outputs Semi-supervised learning Training data includes a few desired outputs Reinforcement learning Rewards from sequence of actions,Types of machine learning,Supervise

5、d learning Classification: discrete output Regression: continuous output,Bias-variance,Training and Validation Data,Full Data Set,Training Data,Validation Data,Idea: train each model on the “training data” and then test each models accuracy on the validation data,Underfitting & Overfitting,Predictiv

6、e Error,Model Complexity,Error on Training Data,Error on Test Data,Ideal Range for Model Complexity,Overfitting,Underfitting,Types of machine learning,Unsupervised learning Clustering Dimensionality reduction Factor analysis,Types of machine learning,Semi-supervised learning Clustering or classifica

7、tion,Types of machine learning,Reinforcement learning Robot & control,Work flow,Prediction,Training Labels,Training,Training,Image Features,Image Features,Testing,Test Image,Learned model,Learned model,Slide credit: D. Hoiem and L. Lazebnik,Work flow,Features,Work flow,Models Logic, Rules Statistica

8、l, Black box model Static, dynamic model Online learning Ensemble learning,Work flow,Architecture,Model,Feature,Hardware,Popular models,Linear model: logistic regression, linear discriminant analysis, linear regression (with basis function),Popular models,Nearest neighbor Feature & distance,Popular

9、models,Support vector machine,Popular models,Artificial neural network,Popular models,Decision tree,Popular models,Collaborative filtering,Popular models,Hierarchical clustering K-means Spectral clustering Manifold learning,Popular models,Hidden markov model Conditional random fields,Applications,Ap

10、plications,Applications,Applications,Applications,Applications,Applications,Applications,Applications,Attention,Applications,Image classification,Applications,Applications,Brain machine interface,Applications,Applications,Applications,Applications,Applications,Indirect illumination Regression,Applicat