Programming Interactivity: A Designer's Guide to Processing, Arduino, and OpenFrameworks

by Joshua Noble

Overview

Ready to create rich interactive experiences with your artwork, designs, or prototypes? This is the ideal place to start. With this hands-on guide, you’ll explore several themes in interactive art and design—including 3D graphics, sound, physical interaction, computer vision, and geolocation—and learn the basic programming and electronics concepts you need to implement them. No previous experience is necessary.

You’ll get a complete introduction to three free tools created specifically for artists and designers: the Processing programming language, the Arduino microcontroller, and the openFrameworks toolkit. You’ll also find working code samples you can use right away, along with the background and technical information you need to design, program, and build your own projects.

• Learn cutting-edge techniques for interaction design from leading artists and designers
• Let users provide input through buttons, dials, and other physical controls
• Produce graphics and animation, including 3D images with OpenGL
• Use sounds to interact with users by providing feedback, input, or an element they can control
• Work with motors, servos, and appliances to provide physical feedback
• Turn a user’s gestures and movements into meaningful input, using Open CV

Product Details

ISBN-13:	9781449311445
Publisher:	O'Reilly Media, Incorporated
Publication date:	02/01/2012
Edition description:	Second Edition
Pages:	724
Product dimensions:	7.00(w) x 9.10(h) x 1.60(d)

About the Author

Joshua Noble is an interaction designer and developer. He's the lead author of the Flex 4 Cookbook (O'Reilly, May 2010) and Programming
Interactivity (July 2009). He's interested in designing humane objects and services for the intersection of public spaces, technology, and micro-computing, and exploring how people can participate in their physical and virtual communities.

Table of Contents

Preface; What Is—and Isn’t—in This Book; Conventions Used in This Book; Companion Website; Using Code Examples; Safari® Books Online; How to Contact Us; Acknowledgments; Chapter 1: Introducing Interaction Design; 1.1 What This Book Is For; 1.2 Programming for Interactivity; 1.3 Design and Interaction; 1.4 Art and Interaction; 1.5 Data Exchange and Exploration; 1.6 Working Process; Chapter 2: Programming Basics; 2.1 Why You’ll Read This Chapter More Than Once; 2.2 The Nature of Code; 2.3 Variables; 2.4 Control Statements; 2.5 Functions; 2.6 Objects and Properties; 2.7 Scope; 2.8 Review; Chapter 3: Processing; 3.1 Downloading and Installing Processing; 3.2 Exploring the Processing IDE; 3.3 The Basics of a Processing Application; 3.4 The Basics of Drawing with Processing; 3.5 Capturing Simple User Interaction; 3.6 Importing Libraries; 3.7 Loading External Data into Processing; 3.8 Running and Debugging Applications; 3.9 Exporting Processing Applications; 3.10 Conclusion; 3.11 Review; Chapter 4: Arduino; 4.1 Starting with Arduino; 4.2 Touring an Arduino Board; 4.3 Touring the Arduino IDE; 4.4 The Basics of an Arduino Application; 4.5 Features of the Arduino Language; 4.6 How to Connect Things to Your Board; 4.7 Hello World; 4.8 Debugging Your Application; 4.9 Importing Libraries; 4.10 Running Your Code; 4.11 Review; Chapter 5: Programming Revisited; 5.1 Object-Oriented Programming; 5.2 Classes; 5.3 Public and Private Properties; 5.4 Inheritance; 5.5 Processing: Classes and Files; 5.6 C++: Classes and Files; 5.7 Pointers and References; 5.8 Review; Chapter 6: openFrameworks; 6.1 Your IDE and Computer; 6.2 Taking Another Quick Tour of C++; 6.3 Getting Started with oF; 6.4 Touring an oF Application; 6.5 Creating “Hello, World”; 6.6 Drawing in 2-D; 6.7 Displaying Video Files and Images; 6.8 Compiling an oF Program; 6.9 Debugging an oF Application; 6.10 Importing Libraries; 6.11 Review; Chapter 7: Physical Input; 7.1 Interacting with Physical Controls; 7.2 Thinking About Kinetics; 7.3 Getting Gear for This Chapter; 7.4 Controlling Controls; 7.5 Turning Knobs; 7.6 Using Lights; 7.7 Detecting Touch and Vibration; 7.8 Detecting Proximity; 7.9 Detecting Motion; 7.10 Reading Distance; 7.11 Understanding Binary Numbers; 7.12 Communicating with Other Applications; 7.13 Sending Messages from the Arduino; 7.14 Detecting Forces and Tilt; 7.15 Introducing I2C; 7.16 What’s Next; 7.17 Review; Chapter 8: Programming Graphics; 8.1 The Screen and Graphics; 8.2 Seeing Is Thinking, Looking Is Reading; 8.3 Math, Graphics, and Coordinate Systems; 8.4 Drawing Strategies; 8.5 Transformation Matrices; 8.6 Creating Motion; 8.7 Using Vectors; 8.8 Using Graphical Controls; 8.9 Importing and Exporting Graphics; 8.10 What’s Next; 8.11 Review; Chapter 9: Bitmaps and Pixels; 9.1 Using Pixels As Data; 9.2 Using Pixels and Bitmaps As Input; 9.3 Providing Feedback with Bitmaps; 9.4 Looping Through Pixels; 9.5 Manipulating Bitmaps; 9.6 Analyzing Bitmaps in oF; 9.7 Using Edge Detection; 9.8 Using Pixel Data; 9.9 Using Textures; 9.10 Saving a Bitmap; 9.11 What’s Next; 9.12 Review; Chapter 10: Sound and Audio; 10.1 Sound As Feedback; 10.2 Sound and Interaction; 10.3 How Sound Works on a Computer; 10.4 Audio in Processing; 10.5 Sound in openFrameworks; 10.6 openFrameworks and the FMOD Ex Library; 10.7 Maximilian; 10.8 Physical Manipulation of Sound with Arduino; 10.9 A Quick Note on PWM; 10.10 Creating Interactions with Sound; 10.11 Further Resources; 10.12 Review; Chapter 11: Arduino and Feedback; 11.1 Using Motors; 11.2 Smart Feedback; 11.3 Using Servos; 11.4 Using Household Currents; 11.5 Working with Appliances; 11.6 Introducing the LilyPad Board; 11.7 Using Vibration; 11.8 Using an LED Matrix; 11.9 Using the SPI Protocol; 11.10 Serial LED Matrix; 11.11 Using LCDs; 11.12 Using Solenoids for Movement; 11.13 What’s Next; 11.14 Review; Chapter 12: Protocols and Communication; 12.1 Communicating over Networks; 12.2 Using XML; 12.3 Understanding Networks and the Internet; 12.4 Handling Network Communication in Processing; 12.5 Understanding Protocols in Networking; 12.6 Using the ofxNetwork Add-on; 12.7 Creating Networks with the Arduino; 12.8 Wireless Internet on the Arduino; 12.9 Communicating with Bluetooth; 12.10 Communicating Using MIDI; 12.11 Review; Chapter 13: Graphics and OpenGL; 13.1 What Does 3-D Have to Do with Interaction?; 13.2 Understanding 3-D; 13.3 What Is OpenGL?; 13.4 Working with 3-D in Processing; 13.5 Making Custom Shapes in Processing; 13.6 Using Coordinates and Transforms in Processing; 13.7 3-D in openFrameworks; 13.8 Lighting in OpenGL; 13.9 Blending Modes in OpenGL; 13.10 Creating 3-D Objects in oF; 13.11 Using Textures and Shading in Processing; 13.12 Using Another Way of Shading; 13.13 Using ofShader; 13.14 Using Shaders in Processing; 13.15 What to Do Next; 13.16 Review; Chapter 14: Motion and Gestures; 14.1 Computer Vision; 14.2 OpenCV; 14.3 Using Blobs and Tracking; 14.4 Detecting Features with oF; 14.5 Using OpenCV in Processing; 14.6 Feature Tracking in Processing; 14.7 Using Blob Tracking with Physics; 14.8 Exploring Further in OpenCV; 14.9 Detecting Gestures; 14.10 Capturing iOS gestures with oF; 14.11 Touch with oF; 14.12 What’s Next; 14.13 Using the Microsoft Kinect; 14.14 Review; Chapter 15: Movement and Location; 15.1 Using Movement As and In Interaction; 15.2 Using Software-Based Serial Ports; 15.3 Understanding and Using GPS; 15.4 Storing Data; 15.5 Logging GPS Data to an Arduino; 15.6 Sending GPS Data; 15.7 Getting Location on a Mobile Device; 15.8 What to Do Next; 15.9 Review; Chapter 16: Spaces and Environments; 16.1 Using Architecture and Space; 16.2 Sensing Environmental Data; 16.3 Using an XBee with Arduino; 16.4 Placing Objects in 2-D; 16.5 Using the X10 Protocol; 16.6 Setting Up an RFID Sensor; 16.7 Reading Heat and Humidity; 16.8 Determine Position of an Object; 16.9 What’s Next; 16.10 Review; Chapter 17: Further Resources; 17.1 What’s Next?; 17.2 Bibliography; 17.3 Conclusion; Colophon;