Making Things See: 3D vision with Kinect, Processing, Arduino, and MakerBot

by Greg Borenstein

Overview

This detailed, hands-on guide provides the technical and conceptual information you need to build cool applications with Microsoft’s Kinect, the amazing motion-sensing device that enables computers to see. Through half a dozen meaty projects, you’ll learn how to create gestural interfaces for software, use motion capture for easy 3D character animation, 3D scanning for custom fabrication, and many other applications.

Perfect for hobbyists, makers, artists, and gamers, Making Things See shows you how to build every project with inexpensive off-the-shelf components, including the open source Processing programming language and the Arduino microcontroller. You’ll learn basic skills that will enable you to pursue your own creative applications with Kinect.

• Create Kinect applications on Mac OS X, Windows, or Linux
• Track people with pose detection and skeletonization, and use blob tracking to detect objects
• Analyze and manipulate point clouds
• Make models for design and fabrication, using 3D scanning technology
• Use MakerBot, RepRap, or Shapeways to print 3D objects
• Delve into motion tracking for animation and games
• Build a simple robot arm that can imitate your arm movements
• Discover how skilled artists have used Kinect to build fascinating projects

Product Details

ISBN-13:	9781449327781
Publisher:	Make Community, LLC
Publication date:	01/13/2012
Sold by:	Barnes & Noble
Format:	eBook
Pages:	440
File size:	13 MB
Note:	This product may take a few minutes to download.

About the Author

After a decade as a musician, web programmer, and startup founder, Greg Borenstein recently moved to New York to become an artist and teacher. His work explores the use of special effects as an artistic medium. He is fascinated by how special effects techniques cross the boundary between images and the physical objects that make them: miniatures, motion capture, 3D animation, animatronics, and digital fabrication. He is currently a grad student at NYU’s Interactive Telecommunications Program.

Table of Contents

PrefaceChapter 1: What Is the Kinect? Chapter 2: Working with the Depth ImageChapter 3: Working with Point CloudsChapter 4: Working with the Skeleton DataChapter 5: Scanning for FabricationChapter 6: Using the Kinect for RoboticsChapter 7: Conclusion: What’s Next? AppendixColophon

Interviews

After spending a decade as a musician, web programmer, and startup founder, what caused you change directions and focus on the Kinect?
I've been obsessed with movie visual effects for a long time and as soon as I saw the Kinect I couldn't help but imagine all the ways it could be used to reproduce traditional effects techniques such as background subtraction and compositing, 3D model integration, camera tracking, etc. It felt like the beginning of a world where the amazing image-making tools of Hollywood blockbusters would become available to everyone. Also, I was blown away by the energy, positivity, and creativity of the open source community that was rapidly forming around the project.

How is the Kinect changing the future of making?
The Kinect makes it radically easier to detect the position and movement of people than ever before. This has had a huge impact on interactive design and art. Cool interaction techniques that used to be so difficult that only experts could implement them are now available to even casual coders. These techniques include detecting people's presence to trigger graphics or sound, tracking their movements and gestures to give them control of digital objects or hardware, and providing, and bringing spaces to life by making them interactive.

Also, the Kinect heralds a future of 3D scanning for fabrication and media. Just like the invention of scanners and digital cameras the ability to capture a three dimensional image of an object promises to unleash new whole new forms of design and image making, from 3D printing to point cloud cinema.

Who is this book for?
My book is aimed at beginning creative coders. This group includes artists and designers looking to use more technology in their work as well as programmers looking to branch out into graphical and interactive work.

Making Things See uses Processing for all its examples. It assumes you've seen Processing (or something like it) before but that you're far from an expert. I designed the book to proceed from beginning topics to more advanced techniques as it goes through the material about the Kinect, simultaneously taking you from beginning creative coding into intermediate and advanced topics.

A healthy curiosity about working in 3D and an excitement about getting up and moving around to control your computer with your body would also really help.

How do you think projects like these will change the future of the Make community?
On the grand scale I think the chief effect of the Kinect (and other related projects such as the Makerbot) will be to increase the importance of working in 3D for Makers. Programming and creating things in 3D involves a different set of technical and mathematical concepts than working on the web or with 2D graphics. Right now it's hard for beginners to learn these concepts, but hopefully we'll get better at teaching them and make software toolkits like Processing even more powerful for working with them.