From bounce-webpronews-84909@listsub.com Fri Oct 27 19:02:28 2000 received: by salzo.cary.nc.us with uucp (FX UUCP) for dave; Fri, 27 Oct 00 22:07:24 EDT Received: from listsub.com [64.23.2.56] by mercury.birdsong.org with smtp (Exim 2.05 #1 (Debian)) for dave51@burtonsys.com id 13pLKF-00031b-00; Fri, 27 Oct 2000 19:02:27 -0700 X-Mailer: Lyris Web Interface Date: Fri, 27 Oct 2000 12:23:43 -0400 Subject: Understanding Search Engine Robots To: From: WebProNews List-Unsubscribe: Reply-To: Message-Id: ==================================================================== WebProNews October 27th, 2000 Issue Understanding Search Engine Robots http://www.WebProNews.com An iEntry Network Newsletter ==================================================================== IN THIS ISSUE: 1.)... Editor's Note 2.)... Today's Feature Article 3.)... Today's Quick Click Specials ==================================================================== ===================================================================== ARE YOU LOSING 88% OF YOUR SALES? A recent Gallup poll found that 88% of online shoppers use a credit card for their Internet purchases. If your site doesn't accept credit cards, then you are losing most of your sales. CHARGE.COM lets you take MasterCard, Visa, American Express, and Discover. They have the best rates, over 95% of applicants are accepted, and there's no application fee. To apply right now, visit http://www.charge.com/wpn ===================================================================== ===================================================================== 1.) EDITOR'S NOTE Hello Webmasters, I hope that yesterday's article by Adam King helped those of you who had 404 problems. On Tuesday, I published an article by Mr. David Notestein called "The Truth about Search Engines." Mr. Notestein has been using computers for many, many years. He is a search engine expert as well as an awesome developer. He is the developer of the Zeus Robot which has been gaining popularity for its ease of use and ability to automate previously difficult webmaster/web marketer's task (Link reciprocity and your own Search Engine Directory). Although a marketer wrote me saying that the article threatened his business, it should be noted that website marketing, like most Internet ventures, should give allowance to the changing technology. It would be wisest to use the best tools available to do the best job in the easiest and most efficient way possible. Most reader's loved Mr. Notestein's article and Tina Maples who builds really nice looking websites with her partners of webmistresses in the company Web Design With TLC ( http://www.webdesignwithTLC.com ) said, Mr. Notestein was: "..right on target with how we feel based on our over 10,000 hours of dealing with the web specifically, and the Internet in general." Today's WebProNews is a great one on search engine robots. These webots are getting to be so important in Internet culture, automating tasks that previously took a heck of a long time. It is a good idea to learn about these very useful and important tools. If you would like to visit David's site: http://cyber-robotics.com/c/c.pl?webrobots I hope you enjoy this issue. Pete mailto:pete@ientry.com ===================================================================== ===================================================================== Join the Tel3.com Affiliate Network Today ! Give away FREE Long Distance and earn up to $10 per Referral (plus residual) in only 4 simple steps! Tel3.com was designed to provide your referrals with all their calling card needs, the best rates and quality service. Call 1-305-914-3365 or click below to Create your Account Now. http://www.tel3.com/affmembersignup.asp?member=2211 If you have any questions, mailto:swilliams@tel3.com ===================================================================== ===================================================================== 2.) ~~~~Understanding Search Engine Robots~~~~~~ Robot Travel If there is one thing I have learned about robots, it is that there is absolutely no pattern to them. Most robots are stupid and wander randomly. For example, 50% of robot hits to my sites, ask for the robots.txt page and then go away never asking for anything else. Then they come back a week later, ask for the same thing and then go away, again. This happens over and over again for months. I have never figured it out. What are they doing? If they wanted to see if the website was really a web site, they could just Ping it. This would be much faster and much more efficient. They seldom visit another page and if they do, they ask for one other page every visit or so. Some come in and issue rapid-fire requests for every page in the website. How rude! You have to quit worrying so much about robots. It takes 6 months before they request enough pages to do you any good. I really quit thinking about them a long time ago. Build a lot of pages correctly and, if you have reciprocal links to them, the robots will find them someday. Try this: Go to AltaVista and type into the search box link:YourSite.com (Leave off the www). This will list the reciprocal links to your web site. Try link:crownjewels.com and you get 136 links to it. Think about this now: The robots say to themselves, "Here is a site that must be popular or why would so many websites SIMILAR to it have it's link on their pages?" Remember that only SIMILAR sites with SIMILAR THEMES would probably have a link to your site. They give more importance to this than you submitting your link to them. Wouldn't you? Go to heavily trafficked sites matching your web site's Themes and use AltaVista to find out how many reciprocal links they have. This will prove to you I am right. Search engines are nothing more than reciprocal links to your site. The problem is, you are constantly having to fight for your positioning in the search query listings. Forget about that. Leave the fighting to people who are able to spend 24 hours a day trying to trick everybody. Quit trying to compete with the large organizations pouring millions into their marketing. Completely forget about Search Engines after submitting to them and go after the reciprocal links. The Search Engines will then believe you are a heavily visited site because you will be. You will now be getting the traffic you so richly deserve. Search engine visitors to your site, are often-times not qualified visitors. Too many visitors pop into your home page for 2 seconds and then leave. You know how it is. We all do it when we are using the search engines. Either it wasn't the information we were looking for, or they had this huge graphic on this stupid portal page, which just took forever to load. These visitors shouldn't even count, but they get counted as 12-18 hits in your server logs. Hits are requests to the server. One page request can incur a lot of hits: requests to the page itself plus the graphics, each count as a hit. Reciprocal links bring in qualified visitors. These are visitors who were already on a web site which had matching Themes to yours. They already have a good idea of what type of site you are. They will come into your site and actually stay awhile. These visitors should count as double credit, they are so good. I know which type of visitor I would rather have. How do you get people to WANT to put your link on their web sites? Why would a similar site put a link to your site on theirs? Simple, you have similar Themes. You are similar, but not competition. There is one very important lesson to be learned from this crazy robot behavior. You need to make the navigation in your web site so easy that a visitor can find any page within 2 clicks of your home page. One way of doing this is installing hidden DotLinks [Dotlinks are little periods that are linked to other pages which are not really noticeable on your page if you put it as a period. Although, they are not easily seen by the human eye, they are a link that a robot can follow] in your web site. When you do this, robots can find your pages faster and more easily. <> Giving The Robots What They Want. So how do you make the search engine robots give your site a better rating than all the other millions of websites trying to do the same thing? Simple, give them what they want. You can't trick them or make them think that you are better than you are. Think about a visit from the eyes of a robot. He finds a site, usually from links embedded in web pages, then loads the text from the first page. He looks for the META tags and pulls out the keywords and description. If not there he takes the first 200 or so characters of text and uses them as a description. The Title is extracted. He extracts the pure text from the page (strips out the HTML coding). He takes out the common words leaving what he feels may be keywords. (Most do not do this last step.) He now extracts the hyperlinks collating them into those that belong to this website and those that don't (He visits these later as this is how he finds new websites). He may do the same with the email addresses. He goes on to the next page and so on until he has visited all of the pages in your web site. Now they store all of this information. He now knows how many pages you have, how many 'outside hyperlinks in your site', and can give your site a score based on how it is set up. These are the basics. What do they do with the info? When someone comes to search a phrase or keyword, another search routine program takes over using the information the robot found. A person types in the keywords and the search program returns the 256,000 pages matching their keywords. BUT they also consider the following: How old is the website or how long has the engine known about it? How large is the website? Was it properly constructed? How many hyperlinks are there to outside websites? VERY IMPORTANT! How many hyperlinks are located on other websites to this site. The older and better the website the more links to it. These robots know when you are cheating. You can't trick them. It is so simple for the robot developer to incorporate code to negate the tricks. What about scoring keywords only once or twice per page or area like meta, title, etc? Is this page close in size to all the other portal pages? How many web pages in the same directory have the word "index" in them? Does this site have a lot of content? Is any text the same color as the background? Are there links to outside sites? Each page can be checked and compared against what the robot feels is a statistically normal page. These are computers you know. You need a lot of pages with normal content. Instead of spending the time to make fake pages, give the real ones content. This will also give your visitors something to come back to. CONTENT ==================================================================== David Notestein is the inventor of Zeus, an easy-to-use, intelligent, internet robot that builds a link directory and creates reciprocal links for your web site. To learn more or download your free copy of Zeus: http://cyber-robotics.com/c/c.pl?webrobots ==================================================================== QuickClicks A recent Gallup poll found that 88% of online shoppers use a credit card for their Internet purchases. If your site doesn't accept credit cards, then you are losing most of your sales. http://www.charge.com/wpn Give away FREE Long Distance and earn up to $10 per Referral (plus residual) in only 4 simple steps! Tel3.com was designed to provide your referrals with all their calling card needs, the best rates and quality service. http://www.tel3.com/affmembersignup.asp?member=2211 ==================================================================== Advertise to 500,000+ internet entrepreneurs and more! For ad details & prices, click... mailto:susan@WebProNews.com ==================================================================== We at the Editorial Team would like to thank all our readers for reading WebProNews. We hope you find this information useful. Sincerely, Peter Thiruselvam mailto:pete@WebProNews.com Editor Here are some other great newsletters: http://www.ientry.com/aboutus/signup.html ==================================================================== You are currently subscribed as: *** dave51@burtonsys.com *** ==================================================================== --- You are currently subscribed to webpronews as: dave51@burtonsys.com Removing your email address from this list is easy. Simply click the email address below and send: