Introduction to Machine Learning
Rob Schapire
Rita Head's Group
Introduction to Machine Learning
Rob SchapireWhat is Machine Learning?Machine learning studies computer algorithms for learning
to do stuff. We might, for instance, be interested in learning to complete a task, or to make accurate predictions, or to behave intelligently. The learning that is being done is always based on some sort
of observations or data, such as examples (the most common ca
se in this course), direct experience, or instruction. So in general, machine learning is about learning to do better in the future based on what was experienced in the past.The emphasis of machine learning is on automatic methods. In other words, the goal is to devise learning algorithms that do the learning automatically without human intervention or assistance. The machine learning paradigm can be viewed as "programming by example."Often we have a specific task in mind, such as spam filtering. But rather than program the computer to solve the task directly, in machine learning, we seek methods by which the computer will come up with its own program based on examples that we provide. Machine learning is a core subarea of artificial intelligence. It is very unlikely that we will be able to build any kind of intelligent system capable of any of the facilities that we
associate with intelligence, such as language or vision, without using learning to get there.These tasks are otherwise simply too difficult to solve. Further, we would not consider a system to be truly intelligent if it were incapable of learning since learning is at the core of
intelligence. Although a subarea of AI, machine learning also intersects broadly with other fields, especially statistics, but also mathematics, physics, theoretical computer science and more.Examples of Machine Learning ProblemsThere are many examples of machine learning problems. Much of this course will focus on classification problems in which the goal is to categorize objects into a fixed set of categories.
Here are several examples:
* optical character recognition: categorize images of handwritten characters by the letters represented
* face detection: find faces in images (or indicate if a face is p
resent)
* spam filtering: identify email messages as spam or non-spam
* topic spotting: categorize news articles (say) as to whether they are about politics, sports, entertainment, etc.
* spoken language understanding: within the context of a limited domain, determine the meaning of something uttered by a speaker to the extent that it can be classified into one of a fixed set of categories* medical diagnosis: diagnose a patient as a sufferer or non-sufferer of some disease
* customer segmentation: predict, for instance, which customers will respond to a particular promotion
* fraud detection: identify credit card transactions (for instance) which may be fraudulent in nature
* weather prediction: predict, for instance, whether or not it will rain tomorrow (In this last case, we most likely would actually be more interested in estimating the probability of rain tomorrow.)In classification, we want to categorize objects into fixed categories. In regression, on the other hand, we are trying to predict a real value. For
instance, we may wish to predict how much it will rain tomorrow. Or, we might want to predict how much a house will sell for.A richer learning scenario is one in which the goal is actually to behave intelligently, or to make intelligent decisions. For instance, a robot needs to learn to navigate through its environment without colliding with anything. To use machine learning to make money on
the stock market, we might treat investment as a classification problem (will the stock go up or down) or a regression problem (how much will the stock go
up), or, dispensing with these intermediate goals, we might want the computer to learn directly how to decide to make investments so as to maximize wealth. A final example is game playing where the goal is for the computer to learn to play well through experience.Goals of Machine Learning ResearchThe primary goal of machine learning research is to develop general purpose algorithms of practical value. Such algorithms should be efficient. As usual, as computer scientists, we care about time and space efficiency. But in the context of learning, we also care a great deal about another precious resource, namely, the amount of data that is required by the
learning algorithm.Learning algorithms should also be as general purpose as possible. We are looking for algorithms that can be easily applied to a broad class of learning problems, such as those
listed above.Of primary importance, we want the result of learning to be a prediction rule that is as accurate as possible in the predictions that it makes.
Occasionally, we may also be interested in the interpretability of the prediction rules produced by learning. In other words, in some contexts (such
as medical diagnosis), we want the computer to find prediction rules that are easily understandable by human experts.As mentioned above, machine learning can be thought of as "programming by example."What is the advantage of machine learning over direct programming? First, the results of using machine learning are often more accurate than what can be created through direct programming. The reason is that machine learning algorithms are data driven, and are able
to examine large amounts of data. On the other hand, a human expert is likely to be guided by imprecise impressions or perhaps an examination of only a
relatively small number of
examples.
Rob SchapireWhat is Machine Learning?Machine learning studies computer algorithms for learning
to do stuff. We might, for instance, be interested in learning to complete a task, or to make accurate predictions, or to behave intelligently. The learning that is being done is always based on some sort
of observations or data, such as examples (the most common ca
se in this course), direct experience, or instruction. So in general, machine learning is about learning to do better in the future based on what was experienced in the past.The emphasis of machine learning is on automatic methods. In other words, the goal is to devise learning algorithms that do the learning automatically without human intervention or assistance. The machine learning paradigm can be viewed as "programming by example."Often we have a specific task in mind, such as spam filtering. But rather than program the computer to solve the task directly, in machine learning, we seek methods by which the computer will come up with its own program based on examples that we provide. Machine learning is a core subarea of artificial intelligence. It is very unlikely that we will be able to build any kind of intelligent system capable of any of the facilities that we
associate with intelligence, such as language or vision, without using learning to get there.These tasks are otherwise simply too difficult to solve. Further, we would not consider a system to be truly intelligent if it were incapable of learning since learning is at the core of
intelligence. Although a subarea of AI, machine learning also intersects broadly with other fields, especially statistics, but also mathematics, physics, theoretical computer science and more.Examples of Machine Learning ProblemsThere are many examples of machine learning problems. Much of this course will focus on classification problems in which the goal is to categorize objects into a fixed set of categories.
Here are several examples:
* optical character recognition: categorize images of handwritten characters by the letters represented
* face detection: find faces in images (or indicate if a face is p
resent)
* spam filtering: identify email messages as spam or non-spam
* topic spotting: categorize news articles (say) as to whether they are about politics, sports, entertainment, etc.
* spoken language understanding: within the context of a limited domain, determine the meaning of something uttered by a speaker to the extent that it can be classified into one of a fixed set of categories* medical diagnosis: diagnose a patient as a sufferer or non-sufferer of some disease
* customer segmentation: predict, for instance, which customers will respond to a particular promotion
* fraud detection: identify credit card transactions (for instance) which may be fraudulent in nature
* weather prediction: predict, for instance, whether or not it will rain tomorrow (In this last case, we most likely would actually be more interested in estimating the probability of rain tomorrow.)In classification, we want to categorize objects into fixed categories. In regression, on the other hand, we are trying to predict a real value. For
instance, we may wish to predict how much it will rain tomorrow. Or, we might want to predict how much a house will sell for.A richer learning scenario is one in which the goal is actually to behave intelligently, or to make intelligent decisions. For instance, a robot needs to learn to navigate through its environment without colliding with anything. To use machine learning to make money on
the stock market, we might treat investment as a classification problem (will the stock go up or down) or a regression problem (how much will the stock go
up), or, dispensing with these intermediate goals, we might want the computer to learn directly how to decide to make investments so as to maximize wealth. A final example is game playing where the goal is for the computer to learn to play well through experience.Goals of Machine Learning ResearchThe primary goal of machine learning research is to develop general purpose algorithms of practical value. Such algorithms should be efficient. As usual, as computer scientists, we care about time and space efficiency. But in the context of learning, we also care a great deal about another precious resource, namely, the amount of data that is required by the
learning algorithm.Learning algorithms should also be as general purpose as possible. We are looking for algorithms that can be easily applied to a broad class of learning problems, such as those
listed above.Of primary importance, we want the result of learning to be a prediction rule that is as accurate as possible in the predictions that it makes.
Occasionally, we may also be interested in the interpretability of the prediction rules produced by learning. In other words, in some contexts (such
as medical diagnosis), we want the computer to find prediction rules that are easily understandable by human experts.As mentioned above, machine learning can be thought of as "programming by example."What is the advantage of machine learning over direct programming? First, the results of using machine learning are often more accurate than what can be created through direct programming. The reason is that machine learning algorithms are data driven, and are able
to examine large amounts of data. On the other hand, a human expert is likely to be guided by imprecise impressions or perhaps an examination of only a
relatively small number of
examples.