Machine Learning of Inductive Bias

This book is based on the author's Ph.D. dissertation[56]. The the­ sis research was conducted while the author was a graduate student in the Department of Computer Science at Rutgers University. The book was pre­ pared at the University of Massachusetts at Amherst where the author is currently an Assistant Professor in the Department of Computer and Infor­ mation Science. Programs that learn concepts from examples are guided not only by the examples (and counterexamples) that they observe, but also by bias that determines which concept is to be considered as following best from the ob­ servations. Selection of a concept represents an inductive leap because the concept then indicates the classification of instances that have not yet been observed by the learning program. Learning programs that make undesir­ able inductive leaps do so due to undesirable bias. The research problem addressed here is to show how a learning program can learn a desirable inductive bias.

1 Introduction.- 1.1 Machine Learning.- 1.2 Learning Concepts from Examples.- 1.3 Role of Bias in Concept Learning.- 1.4 Kinds of Bias.- 1.5 Origin of Bias.- 1.6 Learning to Learn.- 1.7 The New-Term Problem.- 1.8 Guide to Remaining Chapters.- 2 Related Work.- 2.1 Learning Programs that use a Static Bias.- 2.2 Learning Programs that use a Dynamic Bias.- 3 Searching for a Better Bias.- 3.1 Simplifications.- 3.2 The RTA Method for Shifting Bias.- 4 LEX and STABB.- 4.1 LEX: A Program that Learns from Experimentation.- 4.2 STABB: a Program that Shifts Bias.- 5 Least Disjunction.- 5.1 Procedure.- 5.2 Requirements.- 5.3 Experiments.- 5.4 Example Trace.- 5.5 Discussion.- 6 Constraint Back-Propagation.- 6.1 Procedure.- 6.2 Requirements.- 6.3 Experiments.- 6.4 Example Trace.- 6.5 Discussion.- 7 Conclusion.- 7.1 Summary.- 7.2 Results.- 7.3 Issues.- 7.4 Further Work.- Appendix A: Lisp Code.- A.1 STABB.- A.2 Grammar.- A.3 Intersection.- A.4 Match.- A.5 Operators.- A.6 Utilities.