No more visitors from Google. What has happened?

Last week, many webmasters observed that all of their websites with an .info domain name disappeared from Google’s search results.

Some websites removed traffic drops from several hundreds of visitors per day to zero visitors per day. It seemed that all websites that used the .info top level domain had been removed from Google’s index.

A few days later, the websites with the .info domains reappeared in Google’s search results.

Why did this happen?

It looks as if Google updated its filters for special domain names and went a little too far. Earlier this year, the head of Google’s anti-spam team made the following statement:

“A top-level domain (TLD registry) will offer domains for under $4. The result will be another TLD blighted by spammy domain registrations.”

Domain names with a .info ending have been available for 99 Cent for some time. It’s likely that very many .info domain names have been purchased for spamming purposes.

Google might have intended to block .info domains that spam and a bug in the algorithm wiped all .info domains from Google’s results. Fortunately, Google’s engineers fixed the bug within days.

What does this mean for your website?

Filtering all .info domains just because many of them are used for spamming is a very drastic measure. Although Google doesn’t do this, it’s clear that there is some kind of filter for these domains.

If you want to succeed with your online business, it might be better to use a .com domain or the local top level domain of your country instead of a .info domain.

However, one method is likely to get your website banned on search engines and recent developments indicate that websites that use that method will be in trouble soon.

What is white hat SEO?

White hat SEO means that the webmaster doesn’t try to trick search engines. White hat SEO means playing by the rules. Web pages that are created with white-hat SEO methods are beneficial to web surfers, search engines and webmasters.

What is black hat SEO?

Black hat SEO attempts to improve rankings in ways that are disapproved of by the search engines, or involve deception. These methods include cloaking, doorway pages, hidden text, etc.

Google and other search engines have made it clear that they penalize websites that use black hat SEO methods when they detect them.

Black-hat SEO methods seem to work. So why not use them?

Some black-hat SEO methods can lead to good results. There are quite a few webmasters who obtained high rankings for their web pages although they optimized them with methods that were not approved by Google and the other search engines.

You have probably also seen some web pages in the search results that looked strange or hardly related to what you’ve actually searched. So do black-hat SEO methods seem to work? Should you use them?

Nearly all black-hat SEO methods have been detected by search engines sooner or later. Javascript redirects or doorway pages used to work in the past but nowadays, these methods are usually the ticket to the land of banned websites.

While some cloaking methods continue to work at this time (if your competitors don’t peach on you), it’s likely that Google can detect them soon. The same is true for paid links. Some paid links can still not be detected by Google but it’s only a matter of time until Google has the algorithms that can.

You might get in trouble even if you used black-hat methods years ago

The problem is that things that cannot be detected by Google now might be detected by Google tomorrow. And Google might also be able to find out what you did in the past.

A good example for a spam filter that also considers things that have been done in the past is the WikiScanner. WikiScanner can find manipulations that have been made in the past and it can also associate anonymous changes to the people and companies who made these “anonymous” changes.

Combine such a spam scanner which a web page archive like Archive.org and you have an easy way to track the spam history of a web page.

Things that you have done in the past might backfire on you.

Don’t use black-hat SEO methods. As technical possibilities evolve, it’s very likely that these methods will be detected even if you don’t use them anymore. It’s better to use tools that focus on white-hat SEO methods.

Follow these steps to ensure you are targeting the right keywords for your blog.

List Your Blog’s Appropriate Keywords

First off, you need to decide what keywords do fit. This can be a nice master list to consult when you’re writing a blog post. Do a little keyword research, either through a keyword research program or Web sites.

I like the Google Adwords Keyword Tool, which includes the ability to scan your blog and suggest keywords. You can also see which terms on your site have the most Adwords competition and search volume.

Examine Current Inappropriate Keywords

You should look at your site’s statistics to examine the keyword phrases people are using to get there.

If you see phrases that have nothing to do with your site’s topics, figure out which article is pulling people and revise the title and the targeted keyword phrase. If you find keywords that are a good fit, look for alternative keyword phrases or similar subjects that fit for future posts.

If you see a particular keyword phrase is driving a lot of traffic to your site or a particular post, edit that post to include more internal links. If this is a popular landing page, you want to give people a reason to stay.

Look for Indirect Keyword Phrases

This can be a little trickier, but it isn’t always bad to get traffic from a keyword phrase that isn’t the precise topic you cover. You may want to leave it alone if it’s working.

What you should consider first is whether the people seeking that information would also be interested in your information. If you run a blog that appeals to teens but is about MTV, you could be introducing readers to your blog even if they found you while searching for information on MySpace.

Why Keywords are Important

You want your blog read by people who are interested. You want people who will visit again, and who will comment and link to it. You might get a quick hit out of a popular phrase that isn’t related, but those won’t be quality visits.

How do Google, Yahoo and MSN/Live create the descriptions and snippets that are used in the search results?

How Google creates descriptions and snippets

Google seems to use the description from the meta description tag if you search for a page by its URL, or if the searched keywords do not appear within the found page.

If the found web page doesn’t have a meta description tag then Google seems to use the sentence that contains the searched keyword as the description.

If a web page is listed in the Open Directory Project (DMOZ.org) then Google might also use the description that is used in the DMOZ directory.

How Yahoo creates descriptions and snippets

Yahoo seems to use only the first part of the meta description which is complemented by a text snippet from the searched page that contains the searched keyword.

If a web page doesn’t have a meta description, Yahoo will use the description of the web page from Yahoo’s directory (if the page is listed there).

If a web page has no meta description and is not listed in the Yahoo directory, then Yahoo will display sentences from the found web page that contain the searched keywords.

How MSN/Live creates descriptions and snippets

MSN/Live seems to use the first sentence that contains the searched keyword as the description. If the searched keyword does not appear on the page, MSN/Live seems to use the first sentence that appears on the page.

What does this mean for your web pages?

If you want to make sure that your web pages are listed with an appealing description in the search results, you should use meta descriptions on your web pages. If you don’t want to use the description that is used in the Yahoo directory and on DMOZ.org you should use the corresponding tags that prevent search engines from using these descriptions.

Since one the best free ways to drive traffic to your site is trough search engines it’s really important to have high ranks for your desired keyword(s).

This post is going to inform you on how you can easily archive your desired keywords by doing the right thing at the right moment.

Is the search volume high enough to drive traffic?

Some keywords are not worth ranking for since they are not popular and never searched for, thus those keywords are not going to benefit you any way or form. A perfect example would be “Pros and Cons of Modern Economics”.

On the other hand, there are keywords, which are almost impossible to archive, and thus you should not target them as well. For example, “Economics”.

The perfect choice would be something that is not difficult to archive and yet people are actually search for it. So the perfect choice in our example, would be “Modern Economics”

So what’s the best way to check for the keywords search volume? There are a number of tools to do so. However, most of them seem not to work now due to unknown reasons.

The best alternative would be, this Google Tool, that perfectly displays keywords search volume.

Learn from your successful competitors

Do a search for the keyword on a number of popular search engines and see who are your competitors. Once, this is done, analyze the sites. In other words determine what those sites have and you don’t. Check their backlinks, anchor texts, places where they are promoted and so on.

There are various tools available on the net to do so. Some of them are Iwebtool tools and SEO tools by SEO company.

Establish linkbacks, as your desired keywords and anchor text

The most important thing you should do whenever you desire to archive high ranks for the specific keywords you should use anchor text to help you.

“Linked Text” Anchor text is the text that you click on to activate and follow a hyperlink to another web page or another web site.

Also remember that if you are promoting your website using banners, it doesn’t help your SEO ranking it helps only your traffic stats. So the ultimate choice would be to place anchor text right under your promotional banner.

There are a lot of places to promote your resource and I am sure that you are fully aware of them. All I want to bring up to your attention is List of Forums to Announce Your Directory, which is going to help in your promotion.

In conclusion, I would like to mention that you should try to be as much ethical as possible so don’t use black hat SEO and link spamming techniques.

What is the X-Robots-Tag?

Google introduced the new X-Robots-Tag directive in 2007 to allow webmasters to control access to non-web page documents, such as Adobe PDF files, video and audio files.

The X-Robots-Tag is included in the HTTP header of a document. The HTTP header is the initial reply of a server to a query. It contains information about the document that follows, including content type, creation date, character set, encodings, etc.

The new X-Robots-Tag allows webmasters to add information about search engine indexing to the HTTP header.

Which commands are supported by the X-Robots-Tag?

At this time, the X-Robots-Tag supports the following commands:

• X-Robots-Tag: NOINDEX (Use this tag if you don’t want to show the URL in Google’s search results)
• X-Robots-Tag: NOARCHIVE (Use this tag if you don’t want to see a Cache link in the search result pages for the document)
• X-Robots-Tag: NOSNIPPET (Use this tag if you don’t want to display a summary in the search result pages.)
• X-Robots-Tag: NOFOLLOW (Use this tag if you don’t want Googe and Yahoo to index the links in the page.)

The X-Robots-Tag is currently supported by Google and Yahoo. Other search engines don’t support the tag yet.

How to add the X-Robots-Tag to your documents

It depends on your server and on the method you use to create your web pages how you can add additional tags to the HTTP header. If your web server uses Apache, you can use the htaccess file to modify your HTTP headers.

In general, you shouldn’t mess with the HTTP header if you’re not 100% sure what you’re doing. A broken HTTP header can keep search engines away so that no search engine will index your website.

Instead of using the X-Robots-Tag, you can also add meta tags with the same functionality to your web pages:

<meta name=”robots” content=”noarchive”>
<meta name=”robots” content=”nosnippet”>
<meta name=”robots” content=”nofollow”>
<meta name=”robots” content=”noindex”>

You can edit these tags directly in the head part of your web pages. They have the same effect as the corresponding X-Robots-Tags explained above.

Do you need the X-Robots-Tag or the corresponding meta tags?

If you don’t want to limit access to your documents of if you only have normal web pages on your website then you don’t need the X-Robots-Tag or the meta tags.

The X-Robots-Tag has been designed to restrict access to your documents. That means that it does not help to increase search engine rankings.

The new X-Robots-Tag and the meta robots tag allow you to restrict access to your documents. If you want to make sure that search engines index all of your web pages then you have to optimize your web pages.

What is Pagerank (PR?)

Before I provide some tips on improving the internal linking of your own site/blog in order to not only increase the “indexibility” (I know, that’s not a word, I like it though) of your own site/blog, but also to increase the PR of your “money pages,” I’ll explain the basics of PR for those who don’t already know.

Every page that is found on the World Wide Web (WWW) is given a set authority. Google calls this authority “PageRank,” while other Search Engines (SE’s) give it their own title.

This authority, or “link juice” is normally calculated by the authority of the page(s) in which it is linked and by the number links on that page in particular. For instance, page.html is assigned a Google PR of 6. This page contains 20 links. Each of the 20 pages that have been linked to will receive a small portion of that PR6. In other words, the higher the PR of the page linking to your page and the fewer the links, the more authority your page will receive.

It’s also important to understand the difference between Google Toolbar PageRank and the actual PageRank that is assigned to each page since Google is the biggest player in the SE game. Toolbar PR is the reading (usually false) that is shown to us via the Google Toolbar or any other tool that draws this data from the Google API, while the actual Google PR is not shown to anyone, outside a select few at Google, and does directly contribute to your pages Search Engine Result Pages (SERPS) positions.

So, although many will say PageRank doesn’t matter, it actually does matter! Next time someone tells you “PageRank doesn’t mean anything,” do me a favor and tell them that they need to understand what PageRank is before they make a statement like that. Google toolbar PR doesn’t mean anything these days, but the actual PR that we can’t see does in fact mean quite a bit!

How to Improve your Internal Linking Structure

As I stated before, authority is passed through to each link on each page of the WWW that has been indexed by any given SE. With that being said, consider all the different types of links you harbor on your own site/blog. Most likely you have one or more of the following:

1. Affiliate links
2. Contact us links
3. About us links
4. Social media links
5. Links to sites that don’t need your help ie. Google, Wikipedia etc.
6. Links to other internal pages that contain the same content as your “money pages” ie. archives and navigation etc.

None of these links deserve any authority, yet they get it on many sites across the net. There are 3 ways to prevent this.

1. robots.txt
2. META nofollow
3. rel=”nofollow” attribute

The robots.txt option is the best for restricting internal pages, as outlined in #2,3 and 6 above. The nofollow META tag is useless, as you can do the same thing with robots.txt, which is honored by more major SE’s and the rel=”nofollow”, which unfortunately is only honored by Google and a few other SE’s, is what you should be using to restrict affiliate links, social network links etc. from being assigned authority or “PR.”

To restrict links from being indexed and assigned authority via the robots.txt method, just insert the following in your robots.txt file (replacing “page.php” and “directory/” with your own pages and/or directories:)

user-agent: Googlebot
disallow: /page.php
disallow: /directory/

You can also use a wildcard to restrict everything after a certain parameter ie.

/directory/*

To restrict SE’s from indexing and/or assigning authority to links via the rel=”nofollow” (some SE’s treat it differently) just insert the following into the link(s) you want to restrict:

rel=”nofollow”

To restrict SE’s from indexing and/or assigning authority to pages via the META nofollow option, just insert the following META tag into the page you want to restrict:

By using one or all of the above options you will be increasing the PR of your site/blog by taking the authority that was once assigned to these pages that should have been ignored in the first place and letting it flow to the pages that you want to rise in the SE’s.

Once you’ve loaded up your robots.txt file and restricted all the static pages on your site/blog it is relatively easy to continue this positive trend. Also keep it in mind and remember to insert a rel=”nofollow” into every link you publish that doesn’t need to receive any authority.

Brand new domain names are often used by spammers to make a quick buck. These spammers buy hundreds of domains, fill them with automatically created scraper content and hope to make some money with the ads that appear on these sites.

In addition, some webmasters use new domains to test new search engine spamming techniques.

As it is difficult for Google to find out whether a new domain can be trusted or not, Google invented a set of filters that downranks new websites until Google thinks that they can be trusted.

What can you do to overcome Google’s filters for new websites?

It’s very difficult to get high rankings before Google trusts your website. For that reason, do things that make your website trustworthy:

1. Start with the right keywords

   It’s not possible to get a top 10 ranking for highly competitive general search term such as “cars” for a new website. However, it is possible to get high rankings for terms such as “used car dealer atlanta”.

   It’s not just easier to get high rankings for more specific search terms, these terms are also much more likely to convert to sales. Take some time to find the right keywords for your site.

2. Get links to your website

   It is not possible to get high rankings on Google without good incoming links. Try to get as many links from related websites as possible. If the right websites link to your site then Google will trust your website more quickly.

3. Optimize your web pages

   While more links to your website greatly increase your chance of getting high search engine rankings, you must also tell search engines for which search terms you want to have high rankings. Optimize the content of your web pages to make sure that Google lists your website for the right search terms.

   Search engines should be able to parse the content of your web pages easily. Consider this when creating a new website from scratch.

4. Wait

   A website that has been online for several years is much less likely to game Google’s ranking algorithms than newer sites. For that reason, your Google rankings will also increase just by waiting (given that you followed the steps 1 to 3).

If you do it correctly, getting high search engine rankings for brand new websites is possible. It’s important that you do the right things in the right order.

1. Google doesn’t like cloaking.
2. It doesn’t matter if a small or a big company uses cloaking.
3. Websites that use cloaking will be removed from Google’s index.
4. Google will detect all cloaking attempts sooner or later.

What is cloaking?

Cloaking is a search engine optimization technique in which the web page content presented to search engine spiders is different from that presented to the normal web surfers.

This is done by delivering web page content based on the IP address or the User-Agent of the user requesting the page.

When a user is identified as a search engine spider, a script on the server delivers a different version of the web page. The purpose of cloaking is to deceive search engines.

All major search engines consider cloaking to be a violation of their guidelines. For that reason, websites that use cloaking will be banned from the search results.

Google’s opinion on “undetectable” cloaking

In his recent blog post, Google’s Matt Cutts commented on a Danish company that offered ‘undetectable’ cloaking to its customers.

Google tried to check if this claim was true and they quickly found a website that used the services of the company. It turned out that their cloaking wasn’t undetectable at all:

“If someone is trying to manipulate Google by deceptive cloaking, it means that a webserver is returning different content to Googlebot than to users.

Thatâ€^TMs a condition that can be checked for by algorithms or manually, and such cloaking is certainly not ‘undetectable.’”

Don’t risk your search engine rankings

You might get short term results with shady SEO techniques such as cloaking but it is very likely that your site will be banned from search engines if you use them. You’ll put your web business at severe risk if you use black-hat SEO methods.

First thing’s first, on page optimization (then we’ll finish with link building). While many SEO’s delve into a bunch of meta information (keywords, index and follow, etc.) and keyword density, we’re not going to worry with that. Not  that I’m saying it’s unecessary or unimportant, only that there is little (if any) benefit to worrying about that. Let’s just make sure the pages on your website are crawlable and able to be fully indexed. There are a few factors here that are quite important:

Page Title – in my experience this is the most important part of your web page. The page title (found in your head tags and is visible in your browser bar) and should briefly describe exactly what your page is about. Less is more here, keep it short and sweet. My advice is to avoid using your site’s domain name in the title unless it contains your main keyword, or you’re concerned with branding. Regardless of what you decide, at least place you page’s actual title in front of anything else.

For example, this page’s title would display as “Basic Search Engine Optimization Anyone Can Do” or if I wanted to I could append “- Joe hayes {dot} Org” to the end of that or maybe even the category the post was placed in.

Another important thing is to ensure all pages titles are different. Duplicate page titles will likely land you’re pages deep into the supplemental index – and alot of sites do this. Every page on their site will be “DomainName.com” or whatever. Really bad, fix this quick.

Heading Tags – When at all possible, include your page title somewhere on the page in a heading tag, preferably H1 though H2 will do if you’re using that for you’re site name (common for blogs). Don’t repeat your site’s domain name here though, your visitors already know where they are. If you have sub-sections for longer posts, feel free to use H3 or H4 tags there as well. It’s good practice for copywriting.

Your Content – Yea that’s important too. When writing it’s important to use the keywords your wanting to rank for in your content, preferably starting early (in your first paragraph). Nothing spammy here, just make it a point to ensure the key phrases can be found on your page.

Interlink Where Possible – When referencing things that can be found other places on your website, link to them when possible. In content links are perhaps the most important links you can have. Not only that, it helps your readers with references to things you’re talking about. Also, don’t be afraid to link to other sources of information – even your competition.

Site Map – Good idea to have one of these as well, especially helpful in a portal or blog type platform where articles or posts can get lost deep in an archive. Notice I’m referring to a physical site-map here, not an xml machine readable format. Also good for visitors who may be looking for something in particular, in absense of a search tool. And on xml sitemaps while we’re on the subject, I’ve found they’re great for getting pages indexed fast but doesn’t seem to make a difference when it comes to rankings.

Friendly URLs – Not quite as important as it used to be but still a good idea if you can. While both www.example.com/topic.php?=123 and www.example.com/the-topic.html should both be indexed equally (used to be having multiple parameters in the url would prevent the page from being indexed, now it doesn’t seem to be a problem as long as it’s not excessive), having keywords in the url does seem to have some weight when comes to serps. It’s not a dramatic difference, which is why it’s near the bottom of the list.

Build Relevant Backlinks – While this is posted last, it’s certainly not last when it comes to ranking well and is is one of the more misunderstood concepts. All links, quite simply, are not the same. Now, let me be clear – you don’t have to buy links to be successful – in this day and age the social media sites out there are a great avenue for getting exposure and building natural backlinks to compelling content. With that said, however, getting powerful websites in your niche linking back to you is a great way to bolster search rankings and drive targetted traffic to your website. For many, this means purchasing advertising or trading reciprocal links.

Some basic backlink strategies include submitting to web directories (free and paid), Submitting articles to article sites, exchanging links with other sites in your niche (this is important, don’t link spam with unrelated websites), purchase links on related websites (please don’t be one of those page rank huggers that blindly go after after any website just because of PR and also know that Google doesn’t like link buying), leaving valuable comments on blogs related to your niche – especially if they dofollow (blogs that don’t use a rel=”nofollow” on outgoing links), participate in forums related to your niche (most allow a link back in your signature), along with many other ways to get people linking to your website – be creative!

In Summary, remember that any search engines goal is to return the most relevant results for the user’s query. Your site must be more relevant for that particular search than any of your competition. Once your site is crawlable or otherwise search engine friendly, it pretty much comes down to backlinks. All things being equal, the site with the most (relevant, powerful, and in quantity)  backlinks will win. Things like the age of the backlinks, sudden surges in recent backlinks, relevancy (have I mentioned relevancy?), and strength of the backlinks among the things that play a part in how your website ranks today. Once you get top 10 exposure, writing the best content should get you to the top eventually.

Hopefully these tips can help get you on a level playing field with your compeition. Remember to have patience and don’t expect results overnight. It can often take months to bring a site from zero to hero in the serps – depending on site age, competition, and many other factors. Build a website of value for your visitors, make sure it’s crawlable, and gain exposure from other websites – then rankings will come.