I'm sure it is my mistake in mod_rewrite rules, but I can't seem to figure it out. I've set up the rules for my new site and all works perfect, unless - I try to create a sitemap. The URLs get duplicated/triplicated/etc... pretty much ten URLs which open the same page. Here are the rewrite rules: RewriteRule sport/training/article/(.*)$ ./article.php?q=$1RewriteRule sport/eating/article/(.*)$ ./article.php?q=$1RewriteRule sport/track/article/(.*)$ ./article.php?q=$1RewriteRule sport/training/(.*)$ ./list.php?l=1p=$1RewriteRule sport/eating/(.*)$ ./list.php?l=2&p=$1RewriteRule sport/track/(.*)$ ./list.php?l=3&p=$1When my site gets crawled, the result is something like this: http://www.blah.com/sport/training/article/article-about-sporthttp://www.blah.com/sport/training/sport/training/article/article-about-sportSometimes it is even like this: http://www.blah.com/sport/training/sport/training/article/article/article-about-sportAll of the above open the correct page BTW.Internal generated link structure design seems to be ok. What am I missing here?
After mod_rewrite the internal URLs are duplicated many times when crawling
mod rewrite;links;url rewriting;canonical url
I just figured it out. It actually turns out to be a mistake in URL generation. It was an idea about crawler sees what it sees that helped me to solve this.