From Machine Learning -Tom M. Mitchell
Machine Learning is at the forefront of advancements in Artificial Intelligence. It’s moving fast with new research coming out each and every day. This post is in continuation of important concepts and notes right from the basics to advance, from the book Machine Learning, by Tom M. Mitchell.
For Machine Learning Notes 1, please click the link below.
CHAPTER 2: CONCEPT LEARNING AND THE GENERAL-TO-SPECIFIC ORDERING
2.1 Concept Learning
A problem of searching through a predefined space of potential hypothesis for the hypothesis that best fits the training example.
Inferring a boolean-valued function from training examples of its input and output.
Inductive Learning Hypothesis
Any hypothesis found to approximate the target function well over a sufficiently large set of training examples will also approximate the target function well over the unobserved examples.
2.2 Find-S Algorithm
An algorithm to find the most specific hypothesis or best fit hypothesis out of all hypothesis.
The algorithm works only for Positive instances.
In general, any instance in positive features having more than one value should be replaced with a general term >, which is suitable for both the values.
Most specific hypothesis, not allowing any new value to fit.
h ← <ϕ,ϕ,ϕ,ϕ,ϕ>
Drawback: Defined for positive instances only.
2.3 Candidate Elimination
Provides us with the most specific & the most general hypothesis for a training example.
Takes consideration for both positive & negative instances.
Return the General Hypothesis (G) and the Specific Hypothesis (S)
2.4 Inductive Bias
Inductive bias talks about the unobserved instances, how does the hypothesis react to it.
The reason why a hypothesis might not work perfectly for the unobserved instances faced in testing data is caused because it never faced it.
To overcome that we ’ve: UNBIASED LEARNER
The solution to it is that the hypothesis should able to represent every possible subset of instance X, i.e, for all subset of X, the power set of X. Which makes the learning of hypothesis function V(h) ←h’
where h’ is the hypothesis state after the unobserved instance.