Tags |
|
|
Links |
|
|
Casual Articles - Reverse Engineering Search Engine Ranking Algorithms
When forming a joint venture with another business, your most important step is the approach you use when contacting your potential business partner.Participating in a joint venture with another prominent business can be extremely beneficial to ones business. The biggest reason why most webmasters do not get a reply back when sending a proposal, is the approach they use.I cannot begin to tell you how many proposals I personally have received that get deleted almost instantly. There are some that address the proposal with “Dear Webmaster” or as simple as just “Hi”. Some do not tell me anything about themselves or their business (I guess I’m suppose to guess at this one). While others may be run-on, have misspellings, or just plainly unprofessional.A joint venture is not as simple as lets say a link exchange. It requires a little more research and commitment by both parties.Although you can send a joint venture through standard mail, consider sending your proposal by email in your first approach. If they have provided their phone number in their contact information, let them know that after a f
Another drawback is that if you are doing this for a corporate web site, it might not look so good.
For example, you might recall a few weeks ago when Google was caught using shadow pages, and of course claimed they were only "test" pages. Right. Does Google have no dev servers? No staging servers? Are they smart enough to make shadow pages hidden from normal users but not smart enough to hide dev or test pages from normal users? Have they not figured out how a URL or IP filter works? Those pages must have served a purpose, and they didn't want most people to know about it. Maybe they were just weather balloon pages?
I recall discovering some pages that were placed by a hot online & print tech magazine (that wired us into the digital world) on search engines. They had placed numerous blank landing pages using font colors matching the background, which contained large quantities of keywords for their largest competitor. Perhaps they wanted to pay digital homage to CNET? Again, this was probably back in 1998. In fact, they were running articles at the time about how it is wrong to try and trick search engines, yet they were doing it themselves.
Conclusion
While this methodology is good for learning a few things about search engines, on the whole I would not recommend making this the basis for your web site promotion. The quant
Hiring the right person for the right position is not an easy task for companies around the world even in Canada. This can be a painstaking process especially if there are many candidates vying for the position. All the documents submitted by each candidate should be scrutinized to check their authenticity. And their character should be determined by means of a background check as part of employment screening in Canada.What is the purpose of pre-employment screening?Pre-employment screening is a robust tool that a company relies on to authenticate documents submitted by applicants in Canada. Some of these documents include education certificates and experience certificates. There are applicants who submit fake documents just to be qualified for the position. And in order to determine the authenticity of these documents, the company can hire employment screening companies to conduct this screening.What employment screening companies do?The primary job of employment screening companies is to do thorough background check in order to verify the authenticity of the documents presented by the appli
I was able to largely declare my research a success. In fact, it was so accurate that in one case I was able to write a program that produced the exact same search results as one of the search engines. This article explains how I did it, and how it is still beneficial today.
Step 1: Determine Rankable Traits
The first thing to do is make a list of what you want to measure. I came up with about 15 different possible ways to rank a web page. They included things like:
- keywords in title
- keyword density
- keyword frequency
- keyword in header
- keyword in ALT tags
- keyword emphasis (bold, strong, italics)
- keyword in body
- keyword in url
- keyword in domain or sub-domain
- criteria by location (density in title, header, body, or tail) etc
Step 2: Invent a New Keyword
The second step is to determine which keyword to test with. The key is to choose a word that does not exist in any language on Earth. Otherwise, you will not be able to isolate your variables for this study.
I used to work at a company called Interactive Imaginations, and our site was Riddler.com and the Commonwealth Network. At the time, Riddler was the largest entertainment web site, and CWN was one of the top trafficed sites on the net (in the top 3). I turned to my co-worker Carol and mentioned I needed a fake word. She gave me "oofness". I did a quick search and it was not found on any search engine.
Note that a unique word can also be used to see who has copied content from your web sites onto their own. Since all of my test pages are gone (for many years now), a search on Google shows some sites that did copy my pages.
Step 3: Create Test Pages
The next thing to do was to create test pages. I took my home page for my now defunct Amiga search engine "Amicrawler.com" and made about 75 copies of it. I then numbered each file 1.html, 2.html... 75.html.
For each measurement criteria, I made at least 3 html files. For example, to measure keyword density in title, I modified the html titles of the first 3 files to look like this:
1.html:
2.html:
3.html:
The html files of course contained the rest of my home page. I then logged in my notebook that files 1 - 3 were keyword density in title files.
I repeated this type of html editing for about 75 or so files, until I had every criteria covered. The files where then uploaded to my web server and placed in the same directoty so that search engines can find them.
Step 4: Wait for Search Engines to Index Test Pages
Over the next few days, some of the pages started appearing in search engines. However a site like AltaVista might only show 2 or 3 pages. Infoseek / Ultraseek at the time was doing real time indexing so I got to test everything right away. In some cases, I had to wait a few weeks or months for the pages to get indexed.
Simply typing the keyword "oofness" would bring up all pages indexed that had that keyword, in the order ranked by the search engine. Since only my pages contained that word, I would not have competing pages to confuse me.
Step 5: Study Results
To my surprise, most search engines had very poor ranking methodology. Webcrawler used a very simple word density scoring system. In fact, I was able to write a program that gave the exact same search engine results as Webcrawler. That's right, just give it a list of 10 urls, and it will rank them in the exact same order as Webcrawler. Using this program I would make any of my pages rank #1 if I wanted to. Problem is of course that Webcrawler did not generate any traffic even if I was listed number 1, so I did not bother with it.
AltaVista responded best with the most number of keywords in the title of the html. It ranked a few pages way at the bottom, but I don't recall which criteria performed worst. And the rest of the pages ranked somewhere in the middle. All in all, AltaVista only cared about keywords in the title. Everything else didn't seem to matter.
A few years later, I repeated this test with AltaVista and found it was giving high preference to domain names. So I added a wildcard to my DNS and web server, and put keywords in the sub-domain. Voila! All of my pages had #1 ranking for any keyword I chose. This of course led to one problem... Competiting web sites don't like losing their top positions and will do anything to protect their rankings when it costs them traffic.
Other Methods of Testing Search Engines
I am going to quickly list some other things that can be done to test search engines algorithms. But these are all lengthy topics to discuss.
I tested some search engines by uploading large copies of the dictionary, and redirecting any traffic to a safe page. I also tested them by indexing massive quantities of documents (in the millions) under hundreds of domain names. I found in general that there are very few magic keywords found in most documents. The fact still remains that a few keyword search times like "sex", "britney spears", etc brought in traffic but most do not. Hence, most pages never saw any people traffic.
Drawbacks
Unfortunately there were some drawbacks to getting listed #1 for a lot of keywords. I found that it ticked off a lot of people who had competing web sites. They would usually start by copying my winning methodology (like placing keywords in the sub-domain), and then repeat the process themselves, and flood the search engines with 100 times more pages than the 1 page I had made. It made it worthless to compete for prime keywords.
And second, certain data cannot be measured. You can use tools like Alexa to determine traffic or Google's site:domain.com to find out how many listings a domain has, but unless you have a lot of this data to measure, you won't get any useable readings. What good is it for you to try and beat a major web site for a major keyword if they already have millions of visitors per day, you don't, and it is part of the search engine ranking?
Bandwidth and resources can become a problem. I have had web sites where 75% of my traffic was search engine spiders. And they slammed my sites every second of every day for months. I would literally get 30,000 hits from the Google spider every day, in addition to other spiders. And contrary to what THEY believe, they aren't as friendly as they claim.
Another drawback is that if you are doing this for a corporate web site, it might not look so good.
For example, you might recall a few weeks ago when Google was caught using shadow pages, and of course claimed they were only "test" pages. Right. Does Google have no dev servers? No staging servers? Are they smart enough to make shadow pages hidden from normal users but not smart enough to hide dev or test pages from normal users? Have they not figured out how a URL or IP filter works? Those pages must have served a purpose, and they didn't want most people to know about it. Maybe they were just weather balloon pages?
I recall discovering some pages that were placed by a hot online & print tech magazine (that wired us into the digital world) on search engines. They had placed numerous blank landing pages using font colors matching the background, which contained large quantities of keywords for their largest competitor. Perhaps they wanted to pay digital homage to CNET? Again, this was probably back in 1998. In fact, they were running articles at the time about how it is wrong to try and trick search engines, yet they were doing it themselves.
Conclusion
While this methodology is good for learning a few things about search engines, on the whole I would not recommend making this the basis for your web site promotion. The quanti
Tent caterpillars create a cocoon for an entire colony. Within one colony there are two types of caterpillar. One type, pioneers, weave long threads forming the skeleton of the tent. The others, the workers, weave cross threads filling in the structure. If there are too many pioneers, the tent is too big and collapses. If there are not enough pioneers, the tent is too dense and the colony suffocates. Viva la difference.Within the colony of human beings we can find similar differences. Some are explorers, entrepreneurs extending the boundaries of our environment in art, science, or business. The rest fill in the details as accountants or managers.As a manager you know that creativity is the quality which enables your company to expand and improve products, services and processes. The best salespeople, strategic planners and inventors are creative people who generate a steady flow of ideas. However, creativity may come with other features which are not so desirable such as impulsiveness, lack of organization of ideas, paper, and time, and social problems in communication and relationships at work or at hom
Note that a unique word can also be used to see who has copied content from your web sites onto their own. Since all of my test pages are gone (for many years now), a search on Google shows some sites that did copy my pages.
Step 3: Create Test Pages
The next thing to do was to create test pages. I took my home page for my now defunct Amiga search engine "Amicrawler.com" and made about 75 copies of it. I then numbered each file 1.html, 2.html... 75.html.
For each measurement criteria, I made at least 3 html files. For example, to measure keyword density in title, I modified the html titles of the first 3 files to look like this:
1.html:
2.html:
3.html:
The html files of course contained the rest of my home page. I then logged in my notebook that files 1 - 3 were keyword density in title files.
I repeated this type of html editing for about 75 or so files, until I had every criteria covered. The files where then uploaded to my web server and placed in the same directoty so that search engines can find them.
Step 4: Wait for Search Engines to Index Test Pages
Over the next few days, some of the pages started appearing in search engines. However a site like AltaVista might only show 2 or 3 pages. Infoseek / Ultraseek at the time was doing real time indexing so I got to test everything right away. In some cases, I had to wait a few weeks or months for the pages to get indexed.
Simply typing the keyword "oofness" would bring up all pages indexed that had that keyword, in the order ranked by the search engine. Since only my pages contained that word, I would not have competing pages to confuse me.
Step 5: Study Results
To my surprise, most search engines had very poor ranking methodology. Webcrawler used a very simple word density scoring system. In fact, I was able to write a program that gave the exact same search engine results as Webcrawler. That's right, just give it a list of 10 urls, and it will rank them in the exact same order as Webcrawler. Using this program I would make any of my pages rank #1 if I wanted to. Problem is of course that Webcrawler did not generate any traffic even if I was listed number 1, so I did not bother with it.
AltaVista responded best with the most number of keywords in the title of the html. It ranked a few pages way at the bottom, but I don't recall which criteria performed worst. And the rest of the pages ranked somewhere in the middle. All in all, AltaVista only cared about keywords in the title. Everything else didn't seem to matter.
A few years later, I repeated this test with AltaVista and found it was giving high preference to domain names. So I added a wildcard to my DNS and web server, and put keywords in the sub-domain. Voila! All of my pages had #1 ranking for any keyword I chose. This of course led to one problem... Competiting web sites don't like losing their top positions and will do anything to protect their rankings when it costs them traffic.
Other Methods of Testing Search Engines
I am going to quickly list some other things that can be done to test search engines algorithms. But these are all lengthy topics to discuss.
I tested some search engines by uploading large copies of the dictionary, and redirecting any traffic to a safe page. I also tested them by indexing massive quantities of documents (in the millions) under hundreds of domain names. I found in general that there are very few magic keywords found in most documents. The fact still remains that a few keyword search times like "sex", "britney spears", etc brought in traffic but most do not. Hence, most pages never saw any people traffic.
Drawbacks
Unfortunately there were some drawbacks to getting listed #1 for a lot of keywords. I found that it ticked off a lot of people who had competing web sites. They would usually start by copying my winning methodology (like placing keywords in the sub-domain), and then repeat the process themselves, and flood the search engines with 100 times more pages than the 1 page I had made. It made it worthless to compete for prime keywords.
And second, certain data cannot be measured. You can use tools like Alexa to determine traffic or Google's site:domain.com to find out how many listings a domain has, but unless you have a lot of this data to measure, you won't get any useable readings. What good is it for you to try and beat a major web site for a major keyword if they already have millions of visitors per day, you don't, and it is part of the search engine ranking?
Bandwidth and resources can become a problem. I have had web sites where 75% of my traffic was search engine spiders. And they slammed my sites every second of every day for months. I would literally get 30,000 hits from the Google spider every day, in addition to other spiders. And contrary to what THEY believe, they aren't as friendly as they claim.
Another drawback is that if you are doing this for a corporate web site, it might not look so good.
For example, you might recall a few weeks ago when Google was caught using shadow pages, and of course claimed they were only "test" pages. Right. Does Google have no dev servers? No staging servers? Are they smart enough to make shadow pages hidden from normal users but not smart enough to hide dev or test pages from normal users? Have they not figured out how a URL or IP filter works? Those pages must have served a purpose, and they didn't want most people to know about it. Maybe they were just weather balloon pages?
I recall discovering some pages that were placed by a hot online & print tech magazine (that wired us into the digital world) on search engines. They had placed numerous blank landing pages using font colors matching the background, which contained large quantities of keywords for their largest competitor. Perhaps they wanted to pay digital homage to CNET? Again, this was probably back in 1998. In fact, they were running articles at the time about how it is wrong to try and trick search engines, yet they were doing it themselves.
Conclusion
While this methodology is good for learning a few things about search engines, on the whole I would not recommend making this the basis for your web site promotion. The quant
You’re struggling to write an ezine, an e-book and a handful of articles. Even if you’re a good writer, it’s tempting to think, “Wouldn’t it be nice to hire someone to do this? A ghostwriter?! Then my To Do List will stop haunting my dreams.”Let’s consider 3 sources of help for the time-challenged (and it’s possible that some readers would rather scrub floors than write, too).(1) Ghostwriters complete the project from start to finish. You never see their names on the final product. Ideally, you created your contract to establish that your Ghost was writing “for hire,” i.e., you own all the rights to the finished product.Good ghostwriters don’t come cheap. For published books, they can get 50% of the advance and royalties – sometimes more. After all, they get no glory. Often they can’t list your project on their resumes and you won’t give them testimonials. That’s the whole idea: they’re invisible! So they must be compensated for the lack of track record. They’re known by word of mouth.You can hire low-cost ghostwriters at eLance. But you have to develop the assignment, wri
Simply typing the keyword "oofness" would bring up all pages indexed that had that keyword, in the order ranked by the search engine. Since only my pages contained that word, I would not have competing pages to confuse me.
Step 5: Study Results
To my surprise, most search engines had very poor ranking methodology. Webcrawler used a very simple word density scoring system. In fact, I was able to write a program that gave the exact same search engine results as Webcrawler. That's right, just give it a list of 10 urls, and it will rank them in the exact same order as Webcrawler. Using this program I would make any of my pages rank #1 if I wanted to. Problem is of course that Webcrawler did not generate any traffic even if I was listed number 1, so I did not bother with it.
AltaVista responded best with the most number of keywords in the title of the html. It ranked a few pages way at the bottom, but I don't recall which criteria performed worst. And the rest of the pages ranked somewhere in the middle. All in all, AltaVista only cared about keywords in the title. Everything else didn't seem to matter.
A few years later, I repeated this test with AltaVista and found it was giving high preference to domain names. So I added a wildcard to my DNS and web server, and put keywords in the sub-domain. Voila! All of my pages had #1 ranking for any keyword I chose. This of course led to one problem... Competiting web sites don't like losing their top positions and will do anything to protect their rankings when it costs them traffic.
Other Methods of Testing Search Engines
I am going to quickly list some other things that can be done to test search engines algorithms. But these are all lengthy topics to discuss.
I tested some search engines by uploading large copies of the dictionary, and redirecting any traffic to a safe page. I also tested them by indexing massive quantities of documents (in the millions) under hundreds of domain names. I found in general that there are very few magic keywords found in most documents. The fact still remains that a few keyword search times like "sex", "britney spears", etc brought in traffic but most do not. Hence, most pages never saw any people traffic.
Drawbacks
Unfortunately there were some drawbacks to getting listed #1 for a lot of keywords. I found that it ticked off a lot of people who had competing web sites. They would usually start by copying my winning methodology (like placing keywords in the sub-domain), and then repeat the process themselves, and flood the search engines with 100 times more pages than the 1 page I had made. It made it worthless to compete for prime keywords.
And second, certain data cannot be measured. You can use tools like Alexa to determine traffic or Google's site:domain.com to find out how many listings a domain has, but unless you have a lot of this data to measure, you won't get any useable readings. What good is it for you to try and beat a major web site for a major keyword if they already have millions of visitors per day, you don't, and it is part of the search engine ranking?
Bandwidth and resources can become a problem. I have had web sites where 75% of my traffic was search engine spiders. And they slammed my sites every second of every day for months. I would literally get 30,000 hits from the Google spider every day, in addition to other spiders. And contrary to what THEY believe, they aren't as friendly as they claim.
Another drawback is that if you are doing this for a corporate web site, it might not look so good.
For example, you might recall a few weeks ago when Google was caught using shadow pages, and of course claimed they were only "test" pages. Right. Does Google have no dev servers? No staging servers? Are they smart enough to make shadow pages hidden from normal users but not smart enough to hide dev or test pages from normal users? Have they not figured out how a URL or IP filter works? Those pages must have served a purpose, and they didn't want most people to know about it. Maybe they were just weather balloon pages?
I recall discovering some pages that were placed by a hot online & print tech magazine (that wired us into the digital world) on search engines. They had placed numerous blank landing pages using font colors matching the background, which contained large quantities of keywords for their largest competitor. Perhaps they wanted to pay digital homage to CNET? Again, this was probably back in 1998. In fact, they were running articles at the time about how it is wrong to try and trick search engines, yet they were doing it themselves.
Conclusion
While this methodology is good for learning a few things about search engines, on the whole I would not recommend making this the basis for your web site promotion. The quant
I just bought a product from a pretty well known Internet expert, and I can tell you that all the bells and whistles were there – CDs, DVDs, manuals, interview transcripts, graphs and mind maps. But a key element was missing – an element that would have unlocked the success of the product. But the “guru” didn’t include that info.Not even a short mention. That was pretty harsh.It’s About the Product – THEN it’s about the LaunchGranted, launching the product is where the money is – but who wants to be stuck with massive returns. Money can go out the door just as quickly as it came in. Instead, concentrate on creating a truly useful product – then focus on the launch. Got it?That makes the launch easier anyway. You’ll have an easier time stating the benefits of the product to your target audience.You know, when it comes to generating leads, it’s okay to leave something to the imagination. In fact, that’s the only way to go. After all, you don’t want to give away the whole store.
I am going to quickly list some other things that can be done to test search engines algorithms. But these are all lengthy topics to discuss.
I tested some search engines by uploading large copies of the dictionary, and redirecting any traffic to a safe page. I also tested them by indexing massive quantities of documents (in the millions) under hundreds of domain names. I found in general that there are very few magic keywords found in most documents. The fact still remains that a few keyword search times like "sex", "britney spears", etc brought in traffic but most do not. Hence, most pages never saw any people traffic.
Drawbacks
Unfortunately there were some drawbacks to getting listed #1 for a lot of keywords. I found that it ticked off a lot of people who had competing web sites. They would usually start by copying my winning methodology (like placing keywords in the sub-domain), and then repeat the process themselves, and flood the search engines with 100 times more pages than the 1 page I had made. It made it worthless to compete for prime keywords.
And second, certain data cannot be measured. You can use tools like Alexa to determine traffic or Google's site:domain.com to find out how many listings a domain has, but unless you have a lot of this data to measure, you won't get any useable readings. What good is it for you to try and beat a major web site for a major keyword if they already have millions of visitors per day, you don't, and it is part of the search engine ranking?
Bandwidth and resources can become a problem. I have had web sites where 75% of my traffic was search engine spiders. And they slammed my sites every second of every day for months. I would literally get 30,000 hits from the Google spider every day, in addition to other spiders. And contrary to what THEY believe, they aren't as friendly as they claim.
Another drawback is that if you are doing this for a corporate web site, it might not look so good.
For example, you might recall a few weeks ago when Google was caught using shadow pages, and of course claimed they were only "test" pages. Right. Does Google have no dev servers? No staging servers? Are they smart enough to make shadow pages hidden from normal users but not smart enough to hide dev or test pages from normal users? Have they not figured out how a URL or IP filter works? Those pages must have served a purpose, and they didn't want most people to know about it. Maybe they were just weather balloon pages?
I recall discovering some pages that were placed by a hot online & print tech magazine (that wired us into the digital world) on search engines. They had placed numerous blank landing pages using font colors matching the background, which contained large quantities of keywords for their largest competitor. Perhaps they wanted to pay digital homage to CNET? Again, this was probably back in 1998. In fact, they were running articles at the time about how it is wrong to try and trick search engines, yet they were doing it themselves.
Conclusion
While this methodology is good for learning a few things about search engines, on the whole I would not recommend making this the basis for your web site promotion. The quant
Search Engine OptimizationSearch engine optimization (or SEO as it's popularly known) is a set of tools and techniques, which, when applied to a Web site lead to higher (that is better) ranking in the search engines. These tools and techniques are often divided into two main categories - on-page factors and off-page factors. Off-page factors refers to backlinks - links from other sites to individual pages on your site.On-Page FactorsBasically on-page factors and those things over which the Webmaster has direct control. These factors include such items as meta tags, alt tags, and page content. When it comes to page content many sites leave out the one tag that can lead to a dramatic improvement in search engine rankings - that is the H1 tag.The H1 TagThe H1 tag stands for heading 1. There are also tags for heading 2 (H2), heading 3 (H3), and so on. H1 tags are typically used toward the top of the page and most likely (but not necessarily) include the page title (and of course your target keywords or phrases). Be default, anything i
Another drawback is that if you are doing this for a corporate web site, it might not look so good.
For example, you might recall a few weeks ago when Google was caught using shadow pages, and of course claimed they were only "test" pages. Right. Does Google have no dev servers? No staging servers? Are they smart enough to make shadow pages hidden from normal users but not smart enough to hide dev or test pages from normal users? Have they not figured out how a URL or IP filter works? Those pages must have served a purpose, and they didn't want most people to know about it. Maybe they were just weather balloon pages?
I recall discovering some pages that were placed by a hot online & print tech magazine (that wired us into the digital world) on search engines. They had placed numerous blank landing pages using font colors matching the background, which contained large quantities of keywords for their largest competitor. Perhaps they wanted to pay digital homage to CNET? Again, this was probably back in 1998. In fact, they were running articles at the time about how it is wrong to try and trick search engines, yet they were doing it themselves.
Conclusion
While this methodology is good for learning a few things about search engines, on the whole I would not recommend making this the basis for your web site promotion. The quantity of pages to compete against, the quality of your visitors, the shoot-first mentality of search engines, and many other factors will prove that there are better ways to do web site promotion.
This methodology can be used for reverse engineering other products. For example, when I worked at Agency.com doing stats, we used a product made by a major micro software company (you might be using one of their fine operating system products right now) to analyze web server logs. The problem was that it took more than 24 hours to analyze 1 days worth of logs, so it was never up to date. A little bit of magic and a little bit of perl was able to generate the same reports in 45 minutes simply by feeding the same logs into both systems until the results came out the same and every condition was accounted for.
HTTP = HTML link (for blogs, profiles,phorums):
<a href="http://www.casualarticles.com/article/78413/casualarticles-Reverse-Engineering-Search-Engine-Ranking-Algorithms.html">Reverse Engineering Search Engine Ranking Algorithms</a>
BB link (for phorums):
[url=http://www.casualarticles.com/article/78413/casualarticles-Reverse-Engineering-Search-Engine-Ranking-Algorithms.html]Reverse Engineering Search Engine Ranking Algorithms[/url]
Related Articles:
Doing Business In the Age of Information Technology
Internet Marketing on a Budget
Press Releases - Do You Need It? Heck - What Is It Anyway?
Bookmark it:
|