First Chapter

Search Engine Optimization For Dummies

John Wiley & Sons

Chapter One

In This Chapter

* Discovering where people search

* Understanding the difference between search sites and search systems

* Distilling thousands of search sites down to about a dozen search systems

* Preparing your search strategy

You've got a problem. You want people to visit your Web site; that's the purpose, after all, to bring people to your site to buy your products, or learn about them, or hear about the cause you support, or for whatever other purpose you've built the site. So you've decided you need to get traffic from the search engines - not an unreasonable conclusion, as you find out in this chapter. But there are so many search engines! There are the obvious ones, the Googles, AOLs, Yahoo!s, and MSNs of the world, but you've probably also heard of others - HotBot, Dogpile, Inktomi, Ask Jeeves, Netscape, EarthLink, LookSmart ... even Amazon provides a Web search on almost every page. There's Lycos and InfoSpace, Teoma and WiseNut, Mamma.com, and Web-Crawler. To top it all off, you've seen advertising asserting that for only $49.95 (or $19.95, or $99.95, or whatever sum seems to make sense to the advertiser), you too can have your Web site listed in hundreds, nay, thousands of search engines. You may have even used some of these services, only to discover that the flood of traffic you were promised turns up missing.

Well, I've got some good news. You can forget almost all the names I just listed - well, at least you can after you've read this chapter. The point of this chapter is to take a complicated landscape of thousands of search sites and whittle it down into the small group of search systems that really matter. (Search sites? Search systems? Don't worry, I explain the distinction in a moment.)

If you really want to, you can jump to the end of the chapter to see the list of search systems you need to worry about and ignore the details. But I've found that, when I give this list to people, they look at me like I'm crazy because they've never heard of most of the names, and they know that some popular search sites aren't on the list. This chapter explains why.

What Are Search Engines and Directories?

The term search engine has become the predominant term for search system or search site, but before reading any further, you need to understand the different types of search, um, thingies, you're going to run across. Basically, you need to know about four thingies:

•   Search indexes or search engines: These are the predominant type of search tools you'll run across. Originally, the term search engine referred to some kind of search index, a huge database containing information from individual Web sites. Google's vast index (google.com) contains over 3 billion pages, for instance. Large search-index companies own thousands of computers that use software known as spiders or robots (or just plain bots - Google's software is known as Googlebot) to grab Web pages and read the information stored in them. These systems don't always grab all the information on each page or all the pages in a Web site, but they grab a significant amount of information and use complex algorithms to index that information. Google, shown in Figure 1-1, is the world's most popular search engine.

•   Search directories: A directory is a categorized collection of information about Web sites. Rather than containing information from Web pages, it contains information about Web sites. The most significant search directories are owned by Yahoo! (dir.yahoo.com) and the Open Directory Project (dmoz.org). (You can see an example of the Open Directory Project, displayed in Google, in Figure 1-2.) Directory companies don't use spiders or bots to download and index pages on the Web sites in the directory; rather, for each Web site, the directory contains information such as a title and description. The two most important directories, Yahoo! and Open Directory, have staff members who examine all the sites in the directory to make sure they are placed into the correct categories and meet certain quality criteria. Smaller directories often allow people submitting sites to specify which category should be used.

TIP

Here's how to see the difference between Yahoo!'s search results and the Yahoo! directory. Go to yahoo.com, type a word into the Search box, and click the Search button. The list of Web sites that appears is what Yahoo! calls the Yahoo! Search results, which are currently provided by Google. But notice the Directory tab at the top of the page; or, underneath some of the search results, you see a line that says something like More Sites about: Arthritis. Click either the tab or link, and you end up in the Yahoo! Directory. (You can go directly to the directory by using dir. yahoo.com.)

•   Non-spidered indexes: I wasn't sure what to call these things, so I made up a name: non-spidered indexes. A number of small indexes, less important than the major indexes such as Google, don't use spiders to examine the full contents of each page in the index. Rather, the index contains background information about each page, such as titles, descriptions, and keywords. In some cases, this information comes from the meta tags pulled off the pages in the index. (I tell you about meta tags in Chapter 2.) In other cases, the person who enters the site into the index provides this information. A number of the smaller systems discussed in Chapter 10 are of this type.

•   Pay-per-click systems: Some systems provide pay-per-click listings. Advertisers place small ads into the systems, and when users perform their searches, the results contain some of these sponsored listings, typically above and to the right of the free listings. Pay-per-click systems are discussed in more detail in Chapter 15.

Keeping the terms straight

Here are a few additional terms that you see scattered throughout the book:

•   Search site: A Web site at which you can search through some kind of index or directory of Web sites, or perhaps both an index and directory. (In some cases, search sites allow you to search through multiple indices.) Google.com, AOL.com, and EarthLink.com are all search sites.

•   Search system: An organization that possesses a combination of software, hardware, and people that is used to index or categorize Web sites - they build the index or directory you search through at a search site. Google is a search system, but AOL.com and EarthLink.com are not. In fact, if you go to AOL.com or EarthLink.com and search, you actually get Google search results.

Google and the Open Directory Project provide search results to hundreds of search sites. In fact, most of the world's search sites get their search results from elsewhere (see Figure 1-3).

•   Search results: The information returned to you (the results of your search) when you go to a search site and search for something. Remember that in many cases, the search results don't come from the search site you're using, but from some other search system.

Why bother with search engines?

Why bother using search engines? Because search engines represent the single most important source of new Web site visitors.

You may have heard that most Web site visits begin at a search engine. Well, this is not true. It was true several years ago, and many people continue to use these outdated statistics because they sound good - "80 percent of all Web site visitors reach the site through a search engine," for instance. However, in 2003, that claim was finally put to rest. The number of search-originated site visits dropped below the 50-percent mark. Most Web site visitors reach their destinations by either typing a URL - a Web address - into their browsers and going there directly or by clicking a link on another site that takes them there. Most visitors do not reach their destinations by starting at the search engines.

However, search engines are still extremely important for a number of reasons:

•   At the time of writing, almost 50 percent of site visits begin at the search engines. Sure, it's not 80 percent, but it's still a lot of traffic.

•   Of the over 50 percent of visits that don't originate at a search engine, a large proportion are revisits - people who know exactly where they want to go. This is not new business; it's repeat business. Most new visits come through the search engines, making search engines the single most important source of new visitors to Web sites.

•   Some studies indicate that a large number of buyers begin at the search engine. That is, of all the people who go online planning to buy something or looking for product information while planning a purchase, perhaps over 80 percent start at the search engines.

•   The search engines represent a cheap way to reach people. In general, you get more bang for your buck going after free search engine traffic than almost any other form of advertising or marketing.

Where Do People Search?

You can search for Web sites at many places. Literally thousands of sites, in fact, provide the ability to search the Web. (What you may not realize, however, is that all these sites search only a small subset of the World Wide Web.)

However, most searches are carried out at just a small number of search sites. How do the world's most popular search sites rank? That depends on how you measure popularity: the percentage of Internet users who visit a site (audience reach); the total number of visitors; the total number of searches carried out at a site; or the total number of hours that visitors spend searching at the site. Each measurement provides a slightly different ranking, though all provide a similar picture, with the same sites appearing on the list, though some in slightly different positions.

The following list runs down the world's most popular search sites, based on the total search hours at each site during a one-month period, as compiled in a 2003 Nielsen/NetRatings study:

Google.com 18,700,000 hours

AOL.com 15,500,500 hours

Yahoo.com 7,100,000 hours

MSN.com 5,400,000 hours

AskJeeves.com 2,300,000 hours

InfoSpace.com 1,100,000 hours

AltaVista.com 800,000 hours

Overture.com 800,000 hours

Netscape.com 700,000 hours

EarthLink.com 400,000 hours

LookSmart.com 200,000 hours

Lycos.com 200,000 hours

Remember, this is a list of search sites, not search systems. In some cases, the sites have own their own systems. Google provides its own search results, but AOL and MSN do not. (AOL gets its results from Google, and MSN's results come from Inktomi, a company owned by Yahoo! - at least at the time of this writing.)

The fact that some sites get results from other search systems means two things. First, the numbers in the preceding list are somewhat misleading. They suggest that Google has around a third of all the search hours. But Google also feeds AOL its results - add AOL's hours to Google's, and you've got almost two thirds of all search hours. Clearly the Google search system is far more important than the Google search site. In fact, the Google search system also feeds four more systems on this list - Yahoo!, Ask Jeeves, Netscape, and EarthLink - and many smaller sites that don't appear on this list. Some estimates put Google's share of the Web's search results as high as 75 or 80 percent. (That statistic will change soon, perhaps even by the time you read this, as you find out a little later in this chapter - Yahoo will stop using Google results soon.)

The second thing to understand is that you can ignore some of these systems. At present, for example, and for the foreseeable future, you don't need to worry about AOL.com. Even though it's probably the world's second most important search site, you can forget about it. Sure, keep it in the back of your mind, but as long as you remember that Google feeds AOL, you need to worry about Google only.

When you get to the search sites that appear below Lycos in the preceding list, the sites become dramatically less important. Google, according to this chart, has almost 100 times the search hours spent at Lycos. And the first 11 sites on this list combined have 265 times the search hours of Lycos. (However, as I explain in a moment, this list doesn't include some important search systems.)

Now reexamine the list of the world's most important search sites and see what you remove so you can get closer to a list of sites you care about. Check out Table 1-1 for the details.

Based on the information in Table 1-1, you can whittle down your list of sites to four: Google, Yahoo!, Ask Jeeves, and AltaVista. These four search sites are all important, and Google is also an important search system, feeding three quarters of the world's search results to AOL, Yahoo!, Netscape, EarthLink, and many other search sites. Teoma/Ask Jeeves is an important search-system feeder, too, providing results to many smaller search sites.

TIP

Okay, so you visited one or two of the sites that you just crossed off and found that you can submit your Web site to the index at that site. What's going on here is that the search site is selling paid inclusion into the search system that feeds it. (I talk about paid inclusion in Chapter 9.) When you pay Lycos to submit your site, for example, Lycos takes your money and then places your site into FAST/AlltheWeb - which isn't a Lycos search system. Lycos is simply acting as a reseller.

REMEMBER

Some important systems are not important sites. For example, MSN, one of the world's most important search sites, gets its search results from Inktomi and LookSmart. To take this into account, make the following changes to your list:

•   Add Inktomi to the list. It's not a search site itself - you can't search Inktomi's index at Inktomi.com - but it's an important search system, feeding not only MSN but also Overture and LookSmart.
  
  Continues...
  

---

Excerpted from Search Engine Optimization For Dummies by Peter Kent Excerpted by permission.
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.

---