Artificial Intelligence for Humans, Volume 1: Fundamental Algorithms

by Jeff Heaton

Overview

A great building requires a strong foundation. This book teaches basic Artificial Intelligence algorithms such as dimensionality, distance metrics, clustering, error calculation, hill climbing, Nelder Mead, and linear regression. These are not just foundational algorithms for the rest of the series, but are very useful in their own right. The book explains all algorithms using actual numeric calculations that you can perform yourself. Artificial Intelligence for Humans is a book series meant to teach AI to those without an extensive mathematical background. The reader needs only a knowledge of basic college algebra or computer programming-anything more complicated than that is thoroughly explained. Every chapter also includes a programming example. Examples are currently provided in Java, C#, R, Python and C. Other languages planned.

Product Details

ISBN-13:	9781493682225
Publisher:	CreateSpace Publishing
Publication date:	11/26/2013
Series:	Artificial Intelligence for Humans , #1
Edition description:	New Edition
Pages:	224
Product dimensions:	7.50(w) x 9.25(h) x 0.47(d)

About the Author

Jeff Heaton, PhD, is a computer scientist that specializes in data science and artificial intelligence. Specializing in Python, R, Java and C#, he is an open source contributor and author of more than ten books. His areas of expertise include predictive modeling, data mining, big data, business intelligence, and artificial intelligence. Jeff holds a Master's Degree in Information Management from Washington University and a PhD in computer science from Nova Southeastern University in computer science. He is the lead developer for the Encog Machine Learning Framework open source project, a senior member of IEEE, and a fellow of the Life Management Institute (FLMI).

Table of Contents

Introduction xix

0.1 Series Introduction xix

0.1.1 Programming Languages xx

0.1.2 Online Labs xxi

0.1.3 Code Repositories xxi

0.1.4 Books Planned for the Series xxii

0.1.5 Oilier Resources xxiii

0.2 Fundamental Algorithms Introduction xxiv

0.3 Structure of this Book xxiv

0.4 The Kickstarter Campaign xxvii

1 Introduction to AI 1

1.1 Relationship to Human Brains 2

1.1.1 The Brain and Its World 4

1.1.2 Brain in a Vat 5

1.2 Modeling Problems 7

1.2.1 Data Classification 7

1.2.2 Regression Analysis 9

1.2.3 Clustering 10

1.2.4 Time Series 11

1.3 Modeling Input and Output 12

1.3.1 A Simple Example 15

1.3.2 Miles per Gallon 16

1.3.3 Presenting Images to Algorithms 18

1.3.4 Financial Algorithms 20

1.4 Understanding Training 22

1.4.1 Evaluating Success 22

1.4.2 Batch and Online Training 23

1.4.3 Supervised and Unsupervised Training 23

1.4.4 Stochastic and Deterministic Training 23

1.5 Chapter Summary 24

2 Normalization 27

2.1 Levels of Measurement 27

2.1.1 Quantitative Observations 30

2.2 Normalizing Observations 31

2.2.1 Normalizing Nominal Observations 32

2.2.2 Normalizing Ordinal Observations 33

2.2.3 Denormalizing Ordinal Observations 35

2.2.4 Normalizing Quantitative Observations 37

2.2.5 Denormalizing Quantitative Observations 38

2.3 Other Methods of Normalization 39

2.3.1 Reciprocal Normalization 39

2.3.2 Reciprocal Denormalization 40

2.3.3 Understanding Equilateral Encoding 40

2.3.4 Implementing Equilateral Encoding 43

2.4 Chapter Summary 47

3 Distance Metrics 51

3.1 Understanding Vectors 51

3.2 Calculating Vector Distance 53

3.2.1 Euclidean Distance 53

3.2.2 Manhattan Distance 55

3.2.3 Chebyshev Distance 57

3.3 Optical Character Recognition 59

3.4 Chapter Summary 62

4 Random Number Generation 65

4.1 PRNG Concepts 66

4.2 Random Distribution Types 67

4.3 Roulette Wheels 70

4.4 PRNG Algorithms 71

4.4.1 Linear Congruential Generator 71

4.4.2 Multiply with Carry 73

4.4.3 Mersonne Twister 74

4.4.4 Box Muller Transformation 76

4.5 Estimating PI with Monte Carlo 77

4.6 Chapter Summary 79

5 K-Means Clustering 83

5.1 Understanding Training Sets 85

5.1.1 Unsupervised Training 85

5.1.2 Supervised Training 88

5.2 Understanding the K-Meaus Algorithm 88

5.2.1 Assignment Step 89

5.2.2 Update Step 90

5.3 Initializing the K-Means Algorithm 91

5.3.1 Random K-Means Initialization 92

5.3.2 Forgy K-Means Initialization 95

5.4 Chapter Summary 97

6 Error Calculation 99

6.1 Sum of Squares Error 100

6.2 Root Mean Square 101

6.3 Mean Square Error 101

6.4 Comparison of Error Calculation Methods 102

6.4.1 Partitioning Training Data 102

6.5 Chapter Summary 104

7 Towards Machine Learning 107

7.1 Coefficients of a Polynomial 109

7.2 Introduction to Training 111

7.2.1 Greedy Random Training 111

7.3 Radial Basis Networks 114

7.3.1 Radial Basis Functions 114

7.3.2 Radial Basis Function Networks 118

7.3.3 Implementing an RBF Network 120

7.3.4 Using an RBF Network 125

7.4 Chapter Summary 126

8 Optimization Training 129

8.0.1 Hill Climbing Training 129

8.1 Simulated Annealing 133

8.1.1 Simulated Annealing Applications 134

8.1.2 Simulated Annealing Algorithm 134

8.1.3 Cooling Schedule 139

8.1.4 Annealing Probability 139

8.2 Nelder Mead 140

8.2.1 Reflection 143

8.2.2 Expansion 144

8.2.3 Contraction 145

8.3 Finishing the Nelder Mead Algorithm 147

8.4 Chapter Summary 148

9 Discrete Optimization 151

9.1 The Traveling Salesman Problem 152

9.1.1 Understanding the Traveling Salesman Problem 152

9.1.2 Implementing the Traveling Salesman Problem 153

9.2 Circular TSP 155

9.3 The Knapsack Problem 156

9.3.1 Understanding the Knapsack Problem 156

9.3.2 Implementing the Knapsack Problem 158

9.4 Chapter Summary 160

10 Linear Regression 163

10.1 Linear Regression 164

10.1.1 Least Squares Fitting 165

10.1.2 Least Squares Fitting Example 167

10.1.3 Anscombe's Quartet 169

10.1.4 Abalone Data Set 170

10.2 Generalized Linear Models 170

10.2.1 Reweight Least Squares Training 173

10.3 Chapter Summary 176

A Examples 179

A.1 Artificial Intelligence for Humans 179

A.2 Staying Up to Date 180

A.3 Obtaining the Examples 180

A.3.1 Download ZIP File 181

A.3.2 Clone the Git Repository 181

A.4 Example Contents 182

A.5 Contributing to the Project 186

References 187