Machine Learning for Email: Spam Filtering and Priority Inbox

by Drew Conway, John White

Overview

If you’re an experienced programmer willing to crunch data, this concise guide will show you how to use machine learning to work with email. You’ll learn how to write algorithms that automatically sort and redirect email based on statistical patterns. Authors Drew Conway and John Myles White approach the process in a practical fashion, using a case-study driven approach rather than a traditional math-heavy presentation.

This book also includes a short tutorial on using the popular R language to manipulate and analyze data. You’ll get clear examples for analyzing sample data and writing machine learning programs with R.

• Mine email content with R functions, using a collection of sample files
• Analyze the data and use the results to write a Bayesian spam classifier
• Rank email by importance, using factors such as thread activity
• Use your email ranking analysis to write a priority inbox program
• Test your classifier and priority inbox with a separate email sample set

Product Details

ISBN-13:	9781449314309
Publisher:	O'Reilly Media, Incorporated
Publication date:	11/07/2011
Pages:	142
Product dimensions:	6.90(w) x 9.10(h) x 0.50(d)

About the Author

Drew Conway is a PhD candidate in Politics at NYU. He studies international relations, conflict, and terrorism using the tools of mathematics, statistics, and computer science in an attempt to gain a deeper understanding of these phenomena. His academic curiosity is informed by his years as an analyst in the U.S. intelligence and defense communities.

John Myles White is a Ph.D. student in the Princeton Psychology Department, where he studies how humans make decisions both theoretically and experimentally. Outside of academia, John has been heavily involved in the data science movement, which has pushed for an open source software approach to data analysis. He is also the lead maintainer for several popular R packages, including ProjectTemplate and log4r.

Table of Contents

Preface; Machine Learning for Hackers: Email; How This Book is Organized; Conventions Used in This Book; Using Code Examples; Safari® Books Online; How to Contact Us; Chapter 1: Using R; 1.1 R for Machine Learning; 1.2 Further Reading on R; Chapter 2: Data Exploration; 2.1 Exploration vs. Confirmation; 2.2 What is Data?; 2.3 Inferring the Types of Columns in Your Data; 2.4 Inferring Meaning; 2.5 Numeric Summaries; 2.6 Means, Medians, and Modes; 2.7 Quantiles; 2.8 Standard Deviations and Variances; 2.9 Exploratory Data Visualization; 2.10 Visualizing the Relationships between Columns; Chapter 3: Classification: Spam Filtering; 3.1 This or That: Binary Classification; 3.2 Moving Gently into Conditional Probability; 3.3 Writing Our First Bayesian Spam Classifier; Chapter 4: Ranking: Priority Inbox; 4.1 How Do You Sort Something When You Don’t Know the Order?; 4.2 Ordering Email Messages by Priority; 4.3 Writing a Priority Inbox; Works Cited;