December 2003

Searching for the right search engines?

Sam Steane

In the second of her search engine two-parter, Sam Steane explains how to submit to the search engines, what to look out for and a shot at what the future may hold.

- How do I submit to the search engines?

Google likes to find your website via incoming links, so if possible get links to your site from others that are already indexed by Google. The Googlebot is generally quick to come out and index websites, so you shouldn't have to wait long before you start to appear in the Google database. The link for submission to the Google search engine (no charge) is http://www.google.co.uk/addurl.html

AlltheWeb has its own 'submit site' link for the FAST search engine, which is free, but you can also pay for inclusion. Paid inclusion means that your website will be indexed much more quickly, and refreshed every 48 hours. Ironically, FAST can be slow to index new websites that haven't paid for submission. The link for free submission to FAST is http://addurl.alltheweb.com/add_url
For paid submission, go to Position Technologies at http://www.positiontech.com

Lycos accepts free and paid submissions, even though the search results are primarily fed by FAST. Lycos is very slow to index free submissions. The link to the submissions page at Lycos is https://secure.lycos.de/countries/uk/

Ask Jeeves has its own search engine, Teoma, supplying its results. To submit to Ask Jeeves, you will need to pay for inclusion - an annual fee which guarantees your website will be indexed every week for 12 months. Position Technologies administer the submission for Ask Jeeves/Teoma via their website at http://www.positiontech.com/

The alternative is to wait for the Teoma search engine to find your website and index it for free, which it should eventually do, but it's difficult to say how long this will take.

To submit to Inktomi, you will need to pay for inclusion in the same way as Ask Jeeves, via Position Technologies. However, if you're prepared to wait, Inktomi will eventually follow links to your website and index it for free.

If you have a large website of over 500 pages, particularly an e-commerce website with dynamic pages, you can submit to Inktomi via their Trusted Feed programme to ensure the dynamic pages are included in the search results. Payment is on a 'per click' basis. Ask Jeeves and AltaVista also have a Trusted Feed programme via Position Technologies.

AltaVista again can be slow to index web pages, particularly if they have been submitted for free via their Basic Submit. To submit to AltaVista, follow this link: http://addurl.altavista.com/, where you have a variety of options.

Open Directory ( DMOZ)

Open Directory is free and edited by volunteers. It supplies results to Google, and numerous other websites. Open is a good place to start if you need to generate incoming links to your website. Submitting your website is quite easy. Getting it listed, however, is another matter as most of the editors are overwhelmed with submissions and some categories have no editor at all. Therefore you may have to submit several times over a few months before your website is listed. Before submitting, have ready a short, objective description of your website (around 25 words), and use the 'submit site' link when you have found the category of your choice.

- Which search engines are worth paying for?

Paying for inclusion, in our opinion, is worth every penny if a) you really want quick results and want to attract traffic to your website fast, or b) you have content that changes on a regular basis or key dynamic pages you want indexed regularly. For low-traffic sites or those where the content changes infrequently, paying for inclusion may not be worth it. The search engines will eventually index your website by following links to it anyway. If you do want to pay for inclusion then submit to Inktomi, FAST and Ask Jeeves/Teoma, via the Position Technologies website.

Other quick results can be gained from the pay-per-click search engines. This, however, is a whole different ball game - see future issues of Marketing Karma for more on pay-per-click.

- What should I look out for?

Before you submit your website to the search engines, it really is worthwhile making sure it has been optimised properly for your key phrases so that the search engines will be able to index your website without any problems. Poor navigation, dynamic pages, frames, and Flash can all hinder the indexing of your website, so check this out first. If you are not paying for inclusion, don't expect quick results. Generally it takes 3-6 months before significant results are seen for a website in the search engines.

To help you get going, here's a very useful link with free tools to help you find out if your website is ready to be submitted.

Other things to bear in mind are:

1. When re-designing a website that's already ranking well in the search engines, ensure the file names are kept the same, eg index.htm, aboutus.htm, contactus.htm etc. If you change the file names or endings of your web pages, then the search engines will need to re-index the entire website and you will be starting again from scratch, as the web pages that are in the search engine's database will no longer exist.
   
   
2. Actively gain incoming links from quality, relevant websites. For example, if the subject matter is engineering, get the site listed in relevant engineering directories and ask for links from related websites. If you produce good content and opinion on a regular basis, getting it published on other websites will further increase your link popularity.
   
   
3. Search engines gain their reputations through relevant results for their users, so make sure your web pages are relevant to any searches visitors are doing that relate to your products or services. Supply good quality content on your website to make your visitors want to come back.

- And finally, what might the future hold ...?

Latest rumours have been that Microsoft is in talks with Google for a takeover. If this happens, it will create a huge monopoly for Microsoft in search as well as in most other things computer-related. It appears to be unlikely this will happen though, and Bill Gates has denied speaking to Google.

However, it is very likely that MSN will launch its own search engine to compete aggressively with Google and Yahoo! Perhaps in 2004?

Another rumour concerning Google is the expectation that it is to go public next year - is this good or bad? Will Google be at the whim of its investors or continue to dominate as it has done, with added cash? We may even see developments in search with more search products being launched across all the engines to gain an edge. Google has already launched Froogle, amongst other search products, and other search engines now have news searches and image searches - this trend is likely to continue.

Much of the search media is now dominated by Yahoo!. The internet giant now owns Inktomi, Overture, AltaVista and FAST. It is expected that Yahoo! will replace the Google listings that currently feature in its web results with those from Inktomi. Is it going to become harder for newcomers to compete?

Another competitor in the search engine scene would certainly add more variety to our searches. The choice is definitely becoming more limited with all the takeovers that have taken up much of the search engine news in 2003. Perhaps a new player will appear on the scene with a big enough budget to take on the leaders. Or will one of the smaller less-known search engines, such as Gigablast for example, become more of an alternative for users who are fed up with the monopoly of the big search media?

At the moment there really are only a handful of search engines that power the majority of searches across the web. How you get good search results for your website is becoming more and more dependent upon how much money you have in your pocket or how much time you have in your hands. 2004 should prove to be interesting times in the search engine scene.


Related link: Searching for the right search engine? Part 1 by Sam Steane, Marketing Karma November 2003