What is a sitemap.xml file?
In short, it's a list of pages. But unlike a sitemap that's designed for human readers - a nicely laid out list of the various pages on your website - it's designed for search engine spiders. They're the little programs that go from page to page, and site to site, building up an index of what's out there on the website so that Google, Bing or whomever can return relevant search results.
Aside from telling them what pages make up your site, and where it can find them, it adds in various other bits of information as well - when each page was last modified, how often each page should be checked for updates, and so on. It makes life easier for the search engines - and for you - because instead of relying on the spiders to find all your pages, it tells them where to find them. And from the spiders' point of view, it saves them time - they will know when they last indexed a particular page, and if the sitemap tells them that it's not been changed since that time, they don't need to expend time and effort in reindexing it.
What does a sitemap.xml look like?
A sitemap will look something like the below :
<?xml version="1.0" encoding="UTF-8">
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http:www.w3.org/1999/xhtml">
<url>
<loc>https://status218.com</loc>
<lastmod>2017-10-06</lastmod>
<changefreq>weekly</changefreq>
<priority>0.9</priority>
<xhtml:link rel="alternate" hreflang="en" href="https://status218.com" />
<xhtml:link rel="alternate" hreflang="fr" href="https://status218.com/fr" />
</url>
...lots more <url></url> tags...
In the above example, one single URL (or page) on the website is represented by the content between the opening
Your robots.txt file and your sitemap
As we saw in the previous post in this series, one of the things that your robots.txt file may contain is a pointer to search engine spiders telling them where your sitemap is.
By default search engines will automatically assume that there's a sitemap.xml file at https://yoursite.com/sitemap.xml - but it doesn't have to be at that location. If it's not, and if you've not told them (via Google's Search Console, or Bing's Webmaster Tools) where they can find it, then they won't necessarily find it.
This isn't necessarily fatal - the spiders will work their way through your site from page to page via internal links, and will build up a pretty decent picture of your site that way. But if there's on theme that will hopefully become clear through this series, it's that the more you can tell spiders things, and the less you can rely on them figuring out for themselves, the better.
So, if your site's sitemap is at a non-standard location, be sure to tell the spiders, in your robots.txt file, where that location is.
Sitemap indexes (indices?)
You don't have to have one sitemap - you can have multiple ones, particularly if you find it more practical to separate things out and have, for example, one sitemap for blog posts, one for blog categories, one for tags, one for pages, and so on.
In that case, your main sitemap becomes an index - it doesn't contain any page details itself, but instead is a list of the individual sitemaps - the spiders then work their way through each individual sitemap in turn.
A sitemap index would look like the below - instead of listing pages, it lists the individual sitemaps. It still tells the spiders when each sitemap was last updated (again, saving the spiders time).
<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<sitemap>
<loc>http://www.example.com/sitemap1.xml</loc>
<lastmod>2004-10-01T18:23:17+00:00</lastmod>
</sitemap>
<sitemap>
<loc>http://www.example.com/sitemap2.xml</loc>
<lastmod>2005-01-01</lastmod>
</sitemap>
</sitemapindex>
Note that sometimes sitemaps may show with sitemap.xml.gz as the address - all that means is that the sitemap is zipped up using Gzip, and it doesn't impact how the spiders will see it.
How can I check my sitemap?
The first thing to do is to try and find it - visit your site and then add sitemap.xml to the end of the address - so https://yoursite.com/sitemap.xml, for example - and see if anything comes back. If it does, then you should see a list of pages on your site.
You may see a list of other sitemaps, in which case you're viewing a sitemap index file, and can then see the addresses of all the other sitemaps it's pointing to in turn.
What if I cannot find a sitemap?
This isn't necessarily fatal - first thing to do is see if you've got a robots.txt file and if so, whether that lists a non-standard address for your sitemap.
If you're running Wordpress, then try looking at /wp-sitemap.xml, as Wordpress should automatically generate a sitemap index for posts, pages, categories, products, etc.
If you've tried that, and no joy, then get in touch with us - we're happy to take a look and get you on track.
What should I do with my sitemap?
If you've got one and it's at sitemap.xml, then not much - as long as it looks like it's up to date, and is being generated regularly, leave it.
If you've got one and it's not at sitemap.xml, then make sure it's pointed to from your robots.txt.
And finally, make sure (wherever it is) that it's added to Google Search Console and Bing Webmaster Tools, as a very minimum.
If you missed it, catch up with Day 2: The Importance of a robots.txt.
Next, we tackle broken links in Day 4 of our 10 Days to a Healthier Website series.