Read an Excerpt
Mining Amazon Web Services
By John Paul Mueller
John Wiley & SonsISBN: 0-7821-4307-5
Chapter One* Learning about Amazon Web Services
Understanding Web Services
Working with Amazon Web Services
Using Amazon Web Services to Your Advantage
Getting the Amazon Web Services Kit
Installing the Kit on a Machine
Considering the Setup of the Development System
You might have already heard quite a bit about Amazon Web Services without really understanding what the phrase means. Amazon is an amazing company. Unlike many companies that created an online presence and are no longer in business, Amazon is still around today and thriving. In fact, the company's owner, Jeff Bezos, was recently quoted as saying the Amazon Services subsidiary (the one that provides your online experience among other things) might eventually become their main business. Given all that Amazon is doing, it's easy to understand how all of these new services could become confusing even to seasoned professionals.
This chapter helps you understand what Amazon Web Services can do for you and how it fits within your business or research plans. It also discusses what you need to know in order to best use Amazon Web Services for your particular needs. Although you might think that Amazon Web Services is only good for people who want to act as associates and sell goods through their Web site, Amazon Web Services is capable of performing myriad other interesting tasks. For example, many individuals use Amazon Web Services as a means for performing research without ever engaging in buying or selling products.
Once the chapter gets past the basic concepts behind the Amazon Web Service, it demonstrates how to obtain the required resources and set them up on your machine. For example, you need both the Amazon Web Services Kit and a developer token to perform basic tasks. In fact, the chapter describes setup scenarios that you should consider to make your machine better reflect what the user of the Web site, document, or application you build will see. Some situations might require you to set up a system that also includes alternative devices such as a Personal Digital Assistant (PDA). In short, when you complete this chapter, you'll have a copy of the Amazon Web Services Kit installed on a machine configured to meet specific requirements.
Understanding Amazon Web Service
Whenever a new technology appears on the scene, it's important to compare it with other technologies you might have used in the past. The comparison process often helps you decide how this new technology differs from what you used in the past and reduces problems caused by hype. The media might try to convince you that a new product or service is something completely different, when in fact it's merely an update of an existing technology. Currently, there's a lot of hype about Web services that makes them sound like something new and very complex. This section of the chapter defines Web services generally, examines Amazon Web Services specifically, and compares this technology to older technologies. What you'll find might surprise you because Web services are really a new implementation of an old technique.
What Is a Web Service?
You can look at a Web service from a number of perspectives. The easiest way to view a Web service is as a means of obtaining access to information. Essentially, you ask the server for information and the server returns that information in some form. The request and the returned information normally appear in eXtensible Markup Language (XML) form. Using XML preserves the meaning behind the information, regardless of the diversity of the platforms involved, so that you receive not only the information, but understand the context in which the information is used. The "Understanding XML Basics" section of Chapter 3 tells you more about XML. All you need to know now is that you receive information in XML format.
From an Amazon Web Services perspective, you request information on one or more products, vendors, or other search criteria. The request defines the kind of information you want to know and how detailed that information will be. Amazon Web Services returns the information you request (when available) in a standardized format.
A Web service also performs some type of useful work. The useful work might be something as simple as interpreting your request, calculating the answer, and sending the result back. In the case of the Amazon Web Service, the Web service accepts your request (normally some search criteria), interacts with the database through a search engine to obtain the information you requested, and sends the information back to you. You can also perform other tasks using the Amazon Web Service, such as making a purchase or selling goods to others. The rest of the book shows how to perform all of these tasks.
The final consideration for a Web service (at least from the Web service user perspective) is that it executes on the remote machine, not on your machine. In short, this means you're using resources on that other machine with the permission of the machine's owner. The remote machine can set requirements for using the Web service, as well as require you to perform specific setup and security checks as part of your request. In the case of the Amazon Web Service, you need to obtain this permission by requesting it as part of the Amazon Web Services Kit download process. The "Getting a Developer Token" section of this chapter tells how to obtain the required permission and what this permission means to you.
How Do Web Services Work?
Many people fear new technology because they don't understand how it works, and many of those who do know how it works enjoy the mystique of knowledge too much to share it with anyone else. Web services are actually quite easy to understand if you look at them in a way that relates the task to everyday occurrences. For example, you might compare the operation of a Web service to making a withdrawal at the bank-the process really is the same. The one thing to remember is that the process a Web service uses to perform a task is always the same. No matter what technology you use to make a request or receive a response, the steps are still the same. Here are the steps that most Web services, including the Amazon Web Service, use to complete a transaction.
1. The client discovers the Web service. During the act of discovery, the client might do things like download a file that tells how to interact with the Web service. This step is the same as someone walking into the bank. The person knows the bank exists and the bank teller might have noticed the person. The bank posts the rules for making a withdrawal or the teller might help a first-time banker understand the rules.
2. The client makes a request based on the rules delivered during the discovery phase. The rules might specify that the request has to appear in a certain form and the client must provide specific data. This step is the same as the person walking up to the teller's window with a withdrawal request. The request must contain the person's account number, the amount they wish to withdraw, and other identifying information. The bank specifies the format of the request and the information it must contain.
3. The server might ask the client for credentials depending on the openness of the Web service. Amazon Web Services is public but still requires that you supply a developer token as identification. This step is the same as the bank teller asking you for a driver's license or other form of identification before honoring your withdrawal request.
4. The Web service performs the work required to honor your request. In most cases, the Web service accesses a database for information, it could enter an order, and it might even provide some level of formatting. Amazon Web Services performs a number of tasks depending on the request you make. The easiest request is a product search. This step equates to the bank teller getting the money from the drawer and counting it.
5. The Web service sends the data to the client. The content of the information depends on the Web service. Amazon Web Services provides data in a very specific format based on the content of the associated database and the nature of the request. This step equates to the teller handing the person their money. In general, the teller orders the money in a specific way and counts it out to the person, rather than simply handing the money over.
6. The client logs out of the Web service or the Web service disconnects the client after some period of inactivity. This step equates to the person leaving the bank, money in hand. If the person doesn't leave the bank (they just hang out in the lobby), you can be sure that someone will ask them to leave.
7. The client does something with the data it receives. In many cases, it formats the data and presents it on screen for the user. This step equates to the person spending the money they receive from the bank.
You can add any amount of complexity needed to the individual steps, but these seven steps define the process every Web server follows. When you break a Web service down into these seven steps, the process that used to appear as magic suddenly becomes quite doable. Chapters 5 through 9 are essentially options you can use to perform these seven steps using different technologies. This book explores the seven steps using various languages and platforms -Amazon Web Services makes information available to just about anyone who needs it. However, it's important to remember that everything comes down to a client making a request and the Web server returning data.
Considering the Usage Requirements
There's no free lunch. Some people would have you believe that the Web service does everything for you and that the client does nothing at all. However, the client interacts with the Web service, which means the client must possess some intelligence to perform the task. To use a Web service, you must understand the usage requirements.
From a client perspective, the type of device you use to access the Web service determines the access speed, as well as what you can do with the data once you receive it. Although a PDA such as the Pocket PC can access Amazon Web Services just fine, most people wouldn't want to use it to place orders. About the best you can hope for is to search for individual products or to perform other kinds of simple research. On the other hand, a desktop or laptop machine has all of the processing power, screen real estate, and functionality to perform any task. Amazon Web Services hasn't changed, but the capability of the client has.
Amazon Web Services also has some usage requirements and these requirements might change the way that you use your client. For example, according to the license agreement (see Appendix B for details) you can't make more than one request per second. In addition, you can't send a file larger than 20KB to Amazon Web Services. Some requests let you supply information to Amazon, so it's possible to exceed the size limitation. These two limitations ensure the Amazon servers won't become overloaded, but they also mean you must provide some type of monitoring in your application to prevent abuse of the licensing terms.
Amazon is interested in protecting its intellectual property. A Web service is essentially a borrowed resource-you're using information supplied by someone else with resources on the remote server. Consequently, you'll find some restrictions on what you can do with the data that Amazon supplies through its Web service. Although you can display the data in any format desired, you can't take credit for the data or modify it (except to shorten it as necessary). You also have to display any copyright or other notices provided with the data. These provisions ensure that someone viewing the data you obtain from Amazon sees the same information that they'll find on the Amazon site.
Old data is a problem. The sales rank for a book might change or Amazon could change the price of a product. Consequently, when using Amazon Web Services you must guarantee that you'll refresh the data every 24 hours when you don't provide prices or availability, or every hour when you do. The licensing agreement provides an email address you can use to request changes to this policy. Data update requirements become a problem when you choose to store the data locally, rather than obtain it from Amazon each time. Developers will store the data locally to obtain a performance benefit-it takes less time to use the local copy of a product description than to request it from Amazon. Obviously, this particular requirement won't be a problem for most Pocket PC users because local storage is at a premium, but it's something you do need to consider for every other environment.
Discovering Uses for Amazon Web Service
The main reason to use Amazon Web Services is to gain personal, company, or monetary benefit. Many people view Amazon Web Services as a way to earn extra money by becoming an associate. However, there are other ways to use this Web service and many of them don't involve an exchange of money. Although this book will provide a lot of examples on how to leverage Amazon Web Services on a Web site or as part of an application to act as an associate, you'll also see other examples that show other uses. (See the "Developing a Shopping Cart Application" section in Chapter 12 for an in-depth explanation of how to develop a Web application that allows users to buy and sell products from Amazon through your Web site.)
Using Overviews and Details
No matter what you plan to do with Amazon, you need to know how to search for the products and services you want. It might be tempting to say that you could use the search engine interface provided by Amazon to perform this task, but the Amazon display is somewhat limiting in that you need to perform several searches to obtain the information you want in most cases. For example, suppose you want to find a book on working with exotic fish. To ensure the book contains information on tetras, you need to look through the table of contents for each book. Although the search engine provides this information, getting to it online can be cumbersome because you have to look up all of the available books that discuss exotic fish, select an individual book from the list, locate the table of contents, review the table of contents one page at a time, and hope that you find what you need. If the first book doesn't contain the information you need, you must perform the same process with the second and subsequent books. This search process could take all day.
Using an application to hunt down the information for you is much simpler. A program that relies on Amazon Web Services can download all of the books that discuss exotic fish, including the table of contents for each book. (Not every publisher supplies a table of contents for online viewing, so you might not be able to perform this search on every book.) The program can then do the searching for you. Your program would only display the books that actually mention tetras in the table of contents.
Searching for information on tetras is one example of a far more complex process, but it gives you a good understanding of how you can use custom search programs to your benefit. In this case, the custom program saves substantial time, lets you locate the precise book you need quickly, and still garners a sale for Amazon. It is the perfect example of how Web services can pay off for everyone involved. Amazon could never design a search engine interface that meets such specific needs-the developers at Amazon have to create a search engine interface that works for the majority of people who visit the site. In sum, a generic search engine can serve most needs, but it might not be the most efficient choice for everyone.
Amazon doesn't require you to know specific product names or any specifics for that matter. Searching for information lets you refine the criteria for locating specific products. You can specify searches using any of the criteria in the following list (Amazon might add other criteria in a future update).
Amazon Standard Item Number (ASIN) Amazon assigns a unique number to every product in their stock called an ASIN. You can use the ASIN, if you know it, to obtain information about a specific book. For books, Amazon uses the ISBN number without any punctuation the book publisher might provide. For example, an ISBN of 0-7821-4134-X becomes an ASIN of 078214134X. The correlation isn't always direct, but you can learn more about the ASIN topic essentials in the kit at \AmazonWebServices\API Guide\search_asin_isbn.htm.
Excerpted from Mining Amazon Web Services by John Paul Mueller Excerpted by permission.
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.