Math for Programmers: 3D graphics, machine learning, and simulations with Python

by Paul Orland

Overview

"A gentle introduction to some of the most useful mathematical concepts that should be in your developer toolbox." - Christopher Haupt, New Relic

Explore important mathematical concepts through hands-on coding. 
Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications.

Filled with graphics and more than 300 exercises and mini-projects, this book unlocks the door to interesting–and lucrative!–careers in some of today’s hottest fields. As you tackle the basics of linear algebra, calculus, and machine learning, you’ll master the key Python libraries used to turn them into real-world software applications.

Summary
To score a job in data science, machine learning, computer graphics, and cryptography, you need to bring strong math skills to the party. Math for Programmers teaches the math you need for these hot careers, concentrating on what you need to know as a developer. Filled with lots of helpful graphics and more than 200 exercises and mini-projects, this book unlocks the door to interesting–and lucrative!–careers in some of today’s hottest programming fields.

About the technology
Skip the mathematical jargon: This one-of-a-kind book uses Python to teach the math you need to build games, simulations, 3D graphics, and machine learning algorithms. Discover how algebra and calculus come alive when you see them in code!

What's inside

    Vector geometry for computer graphics
    Matrices and linear transformations
    Core concepts from calculus
    Simulation and optimization
    Image and audio processing
    Machine learning algorithms for regression and classification

About the reader
For programmers with basic skills in algebra.

About the author
Paul Orland is a programmer, software entrepreneur, and math enthusiast. He is co-founder of Tachyus, a start-up building predictive analytics software for the energy industry. You can find him online at www.paulor.land.

Table of Contents

1 Learning math with code

PART I - VECTORS AND GRAPHICS

2 Drawing with 2D vectors

3 Ascending to the 3D world

4 Transforming vectors and graphics

5 Computing transformations with matrices

6 Generalizing to higher dimensions

7 Solving systems of linear equations

PART 2 - CALCULUS AND PHYSICAL SIMULATION

8 Understanding rates of change

9 Simulating moving objects

10 Working with symbolic expressions

11 Simulating force fields

12 Optimizing a physical system

13 Analyzing sound waves with a Fourier series

PART 3 - MACHINE LEARNING APPLICATIONS

14 Fitting functions to data

15 Classifying data with logistic regression

16 Training neural networks

Product Details

ISBN-13:	9781638357070
Publisher:	Manning
Publication date:	11/30/2020
Sold by:	SIMON & SCHUSTER
Format:	eBook
Pages:	688
File size:	30 MB
Note:	This product may take a few minutes to download.

About the Author

Paul Orland is a programmer, software entrepreneur, and math enthusiast. He is co-founder of Tachyus, a start-up building predictive analytics software for the energy industry. You can find him online at www.paulor.land.
Paul Orland is CEO of Tachyus, a Silicon Valley startup building predictive analytics software to optimize energy production in the oil and gas industry. As founding CTO, he led the engineering team to productize hybrid machine learning and physics models, distributed optimization algorithms, and custom web-based data visualizations. He has a B.S. in mathematics from Yale University and a M.S. in physics from the University of Washington.

Table of Contents

Preface xvii

Acknowledgments xxi

About this book xxiii

About the author xxviii

About the cover illustration xxix

1 Learning math with code 1

1.1 Solving lucrative problems with math and software 2

Predicting financial market movements 2

Finding a good deal 5

Building 3D graphics and animations 7

Modeling the physical world 9

1.2 How not to learn math 11

Jane wants to learn some math 12

Slogging through math textbooks 13

1.3 Using your well-trained left brain 13

Using a formal language 14

Build your own calculator 15

Building abstractions with functions 16

Part 1 Vectors and Graphics 19

2 Drawing with 2D vectors 21

2.1 Picturing 2D vectors 22

Representing 2D vectors 24

2D drawing in Python 26

Exercises 29

2.2 Plane vector arithmetic 32

Vector components and lengths 35

Multiplying vectors by numbers 37

Subtraction, displacement, and distance 39

Exercises 42

2.3 Angles and trigonometry in the plane 51

From angles to components 52

Radians and trigonometry in Python 56

From components back to angles 57

Exercises 60

2.4 Transforming collections of vectors 67

Combining vector transformations 69

Exercises 70

2.5 Drawing with Matplotlib 72

3 Ascending to the 3D world 75

3.1 Picturing vectors in 3D space 77

Representing 3D vectors with coordinates 79

3D drawing with Python 80

Exercises 82

3.2 Vector arithmetic in 3D 83

Adding 3D vectors 83

Scalar multiplication in 3D 85

Subtracting 3D vectors 85

Computing lengths and distances 86

Computing angles and directions 87

Exercises 89

3.3 The dot product: Measuring vector alignment 92

Picturing the dot product 93

Computing the dot product 95

Dot products by example 97

Measuring angles with the dot product 97

Exercises 100

3.4 The cross product: Measuring oriented area 103

Orienting ourselves in 3D 103

Finding the direction of the cross product 106

Finding the length of the cross product 108

Computing the cross product of 3D vectors 109

Exercises 110

3.5 Rendering a 3D object in 2D 114

Defining a 3D object with vectors 114

Projecting to 2D 116

Orienting faces and shading 116

Exercises 119

4 Transforming vectors and graphics 121

4.1 Transforming 3D objects 123

Drawing a transformed object 124

Composing vector transformations 126

Rotating an object about an axis 129

Inventing your own geometric transformations 131

Exercises 134

4.2 Linear transformations 138

Preserving vector arithmetic 138

Picturing linear transformations 140

Way linear transformations? 142

Computing linear transformations 146

Exercises 149

Computing transformations with matrices 158

5.1 Representing linear transformations with matrices 159

Writing vectors and linear transformations as matrices 159

Multiplying a matrix with a vector 161

Composing linear transformations by matrix multiplication 163

Implementing matrix multiplication 166

3D animation with matrix transformations 166

Exercises 169

5.2 Interpreting matrices of different shapes 175

Column vectors as matrices 176

What pairs of matrices can be multiplied? 178

Viewing square and non-square matrices as vector functions 180

Projection as a linear map from 3D to 2D 181

Composing linear maps 184

Exercises 186

5.3 Translating vectors with matrices 191

Making plane translations linear 191

Finding a 3D matrix for a 2D translation 194

Combining translation with other linear transformations 195

Translating 3D objects in a 4D world 196

Exercises 199

6 Generalizing to higher dimensions 205

6.1 Generalizing our definition of vectors 206

Creating a class for 2D coordinate vectors 207

Improving the Vec2 class 208

Repeating the process with 3D vectors 209

Building a vector base class 210

Defining vector spaces 212

Unit testing vector space classes 214

Exercises 216

6.2 Exploring different vector spaces 219

Enumerating all coordinate vector spaces 219

Identifying vector spaces in the wild 221

Treating functions as vectors 223

Treating matrices as vectors 226

Manipulating images with vector operations 227

Exercises 230

6.3 Looking for smaller vector spaces 237

Identifying subspaces 238

Starting with a single vector 240

Spanning a bigger space 240

Defining the word dimension 243

Finding subspaces of the vector space of functions 244

Subspaces of images 245

Exercises 248

7 Solving systems of linear equations 257

7.1 Designing an arcade game 258

Modeling the game 259

Rendering the game 260

Shooting the laser 261

Exercises 262

7.2 Finding intersection points of lines 263

Choosing the right formula for a line 263

Finding the standard form equation for a line 265

Linear equations in matrix notation 267

Solving linear equations with NumPy 268

Deciding whether the laser hits an asteroid 270

Identifying unsolvable systems 271

Exercises 273

7.3 Generalizing linear equations to higher dimensions 278

Representing planes in 3D 278

Solving linear equations in 3D 280

Studying hyperplanes algebraically 282

Counting dimensions, equations, and solutions 283

Exercises 285

7.4 Changing basis by solving linear equations 294

Solving a 3D example 296

Exercises 297

Part 2 Calculus and Physical Simulation 301

8 Understanding rates of change 303

8.1 Calculating average flow rate from volume 305

Implementing an average_flow_rate function 305

Picturing the average flow rate with a secant line 306

Negative rates of change 308

Exercises 309

8.2 Plotting the average flow rate over time 310

Finding the average flow rate in different time intervals 310

Plotting the interval flow rates 311

Exercises 313

8.3 Approximating instantaneous flow rates 315

Finding the slope of small secant lines 315

Building the instantaneous flow rate function 318

Currying and plotting the instantaneous flow rate function 320

Exercises 322

8.4 Approximating the change in volume 323

Finding the change in volume for a short time internal 323

Breaking up time into smaller intervals 324

Picturing the volume change on the flow rate graph 325

Exercises 328

8.5 Plotting the volume over time 328

Finding the volume over time 328

Picturing Riemann sums for the volume function 329

Improving the approximation 332

Definite and indefinite integrals 334

9 Simulating moving objects 337

9.1 Simulating a constant velocity motion 338

Adding velocities to the asteroids 339

Updating the game engine to move the asteroids 339

Keeping the asteroids on the screen 340

Exercises 342

9.2 Simulating acceleration 342

Accelerating the spaceship 343

9.3 Digging deeper into Euler's method 344

Carrying out Euler's method by hand 344

Implementing the algorithm in Python 346

9.4 Running Euler's method with smaller time steps 348

Exercises 349

Working with symbolic expressions 354

10.1 Finding an exact derivative with a computer algebra system 355

Doing symbolic algebra in Python 356

10.2 Modeling algebraic expressions 358

Breaking an expression into pieces 358

Building an expression tree 359

Translating the expression tree to Python 360

Exercises 362

10.3 Putting a symbolic expression to work 365

Finding all the variables in an expression 365

Evaluating cm expression 366

Expanding an expression 369

Exercises 372

10.4 Finding the derivative of a function 374

Derivatives of powers 374

Derivatives of transformed functions 375

Derivatives of some special functions 377

Derivatives of products and compositions 378

Exercises 379

10.5 Taking derivatives automatically 381

Implementing a derivative method for expressions 382

Implementing the product rule and chain rule 383

Implementing the power rule 384

Exercises 386

10.6 Integrating functions symbolically 387

Integrals as antiderivatives 387

Introducing the SymPy library 388

Exercises 389

11 Simulating force fields 392

11.1 Modeling gravity with a vector field 393

Modeling gravity with a potential energy function 394

11.2 Modeling gravitational fields 396

Defining a vector field 396

1 Defining a simple force field 398

11.3 Adding gravity to the asteroid game 399

Making game objects feel gravity 400

Exercises 403

11.4 Introducing potential energy 404

Defining a potential energy scalar field 405

Plotting a scalar field as a heatmap 407

Plotting a scalarfield as a contour map 407

11.5 Connecting energy and forces with the gradient 408

Measuring steepness with cross sections 409

Calculating partial derivatives 411

Finding the steepness of a graph with the gradient 413

Calculating force fields from potential energy with the gradient 415

Exercises 418

12 Optimizing a physical system 422

12.1 Testing a projectile simulation 425

Building a simulation with Euler's method 426

Measuring properties of the trajectory 427

Exploring different launch angles 428

Exercises 429

12.2 Calculating the optimal range 432

Finding the projectile range as a function of the launch angle 432

Solving for the maximum range 435

Identifying maxima and minima 437

Exercises 439

12.3 Enhancing our simulation 440

Adding another dimension 441

Modeling terrain around the cannon 442

Solving for the range of the projectile in 3D 443

Exercises 447

12.4 Optimizing range using gradient ascent 449

Plotting range versus launch parameters 449

The gradient of the range function 450

Finding the uphill direction with the gradient 451

Implementing gradient ascent 453

Exercises 457

13 Analyzing sound waves with a Fourier series 463

13.1 Combining sound waves and decomposing them 465

13.2 Playing sound waves in Python 466

Producing our first sound 467

Playing a musical note 469

Exercises 471

13.3 Turning a sinusoidal wave into a sound 471

Making audio from sinusoidal functions 471

Changing the frequency of a sinusoid 473

Sampling and playing the sound wave 475

Exercises 477

13.4 Combining sound waves to make new ones 478

Adding sampled sound waves to build a chord 478

Picturing the sum of two sound waves 479

Building a linear combination of sinusoids 481

Building a familiar function with sinusoids 483

Exercises 486

13.5 Decomposing a sound wave into its Fourier series 486

Finding vector components with an inner product 487

Defining an inner product for periodic functions 488

Writing a function to find Fourier coefficients 490

Finding the Fourier coefficients for the square wave 491

Fourier coefficients for other waveforms 492

Exercises 494

Part 3 Machine Learning Applications 497

14 Fitting functions to data 499

14.1 Measuring the quality of fit for a function 502

Measuring distance from a function 503

Summing the squares of the errors 505

Calculating cost for car price functions 507

Exercises 510

14.2 Exploring spaces of functions 511

Picturing cost for tines through the origin 512

The space of all-linear functions 514

Exercises 515

14.3 Finding the line of best fit using gradient descent 515

Reseating the data 516

Finding and plotting the line of best fit 516

Exercises 518

14.4 Fitting a nonlinear function 519

Understanding the behavior of exponential functions 519

Finding the exponential function of best fit 521

Exercises 523

15 Classifying data with logistic regression 526

15.1 Testing a classification function on real data 528

Loading the car data 529

Testing the classification function 529

Exercises 530

15.2 Picturing a decision boundary 532

Picturing the space of cars 532

Drawing a better decision boundary 533

Implementing the classification function 534

Exercises 535

15.3 Framing classification as a regression problem 536

Scaling the raw car data 536

Measuring the "BMWness" of a car 538

Introducing the sigmoid function 540

Composing the sigmoid function with other functions 541

Exercises 543

15.4 Exploring possible logistic functions 544

Parameterizing logistic functions 545

Measuring the quality of fit for a logistic function 546

Testing different logistic functions 548

Exercises 549

15.5 Finding the best logistic function 551

Gradient descent in three dimensions 551

Using gradient descent to find the best fit 552

Testing and understanding the best logistic classifier 554

Exercises 555

16 Training neural networks 559

16.1 Classifying data with neural networks 561

16.2 Classifying images of handwritten digits 562

Building the 64-dimensional image vectors 563

Building a random digit classifier 565

Measuring performance of the digit classifier 566

Exercises 567

16.3 Designing a neural network 568

Organizing neurons and connections 568

Data flow through a neural network 569

Calculating activations 572

Calculating activations in matrix notation 574

Exercises 576

16.4 Building a neural network in Python 577

Implementing an MLP class in Python 578

Evaluating the MLP 580

Testing the classification performance of an MLP 581

Exercises 582

16.5 Training a neural network using gradient descent 582

Framing training as a minimization problem 582

Calculating gradients with backpropagation 584

Automatic training with scikit-learn 585

Exercises 586

16.6 Calculating gradients with backpropagation 588

Finding the cost in terms of the last layer weights 589

Calculating the partial derivatives for the last layer weights using the chain rule 590

Exercises 591

Appendix A Getting set up with Python 595

Appendix B Python tips and tricks 607

Appendix C Loading and rendering 3D Models with OpenGL and PyGame 635

Index 645