A web author's guide to search engines

Part 1: how to improve your ranking

Introduction: how do search engines work?

When you use a search engine - maybe Google or Ask Jeeves - you're actually searching a database that contains information about billions of web pages. How is all this information gathered?

Search engines build their databases automatically by sending out software agents across the web. These agents, variously known as 'robots', 'web crawlers' or 'spiders', are programs that follow links from page to page, gathering information about those pages on the way.

If you understand a little about how these robots work and the kind of information they gather, you can influence how your web pages are indexed and so improve the chances that you reach the right audience.

Search engines are constantly changing, both to improve the services they provide and to counteract the efforts of web authors who learn how to mislead them. You should never expect to be able to leap dramatically up the rankings in major search engines - that's what everyone else is trying to do, too. Instead, you should be doing all you can to ensure that users who want to find a site like yours can do so.

There are a number of measures you can take to help users find your web pages: these principles of good practice are set out in this guide.