.NET code to help with XML sitemap generation

I created some code which might save some time for others who want to generate XML sitemaps from .NET code. It is written in C#, but would likely be easily ported to other .NET languages. This code is used by my ASP.NET MVC blog engine to generate the sitemap for my blog entries. It could just as easily be used in ASP.NET WebForms, a console application, or from any .NET code framework.

The code for the helper class can be found here. You might want to change the namespace in the file.

The class SiteMapData is used to hold the data for each sitemap node.

[Serializable]
public class SiteMapData
{
   public string Loc { get; set; }
   public DateTime? Lastmod { get; set; }
   public string Changefreq { get; set; }
   public decimal? Priority { get; set; }
}

A list of SiteMapData instances is passed into the SiteMapHelper class function GenerateSiteMap, which will transform the list of SiteMapData objects into an XDocument.

public XDocument GenerateSiteMap(List<SiteMapData> dataRows)
{
   var xmlNodes =
      (from x in dataRows
         select CreateSiteMapUrlNode(x));

   XDocument siteMap = new XDocument(
      new XDeclaration("1.0", "utf-8", "yes"),
      new XElement(_xmlns + "urlset",
         new XAttribute(XNamespace.Xmlns + "xsd", "http://www.w3.org/2001/XMLSchema"),
         new XAttribute(XNamespace.Xmlns + "xsi", "http://www.w3.org/2001/XMLSchema-instance"), xmlNodes));

   return siteMap;
}

The resulting XDocument can be converted to a GZip'ed stream of bytes by calling the SiteMapHelper function GZipSiteMap.

public MemoryStream GZipSiteMap(XDocument siteMap)
{
   using (MemoryStream ms = new MemoryStream())
   {
      siteMap.Save(ms);
      ms.Seek(0, SeekOrigin.Begin);
      using (MemoryStream compressed = new MemoryStream())
      {
         using (GZipStream zip = 
            new GZipStream(compressed, System.IO.Compression.CompressionMode.Compress))
         {
            ms.CopyTo(zip, (int)ms.Length);
         }

         return new MemoryStream(compressed.ToArray());
      }
   }
}

The following code is an example of how to use the SiteMapHelper class to generate a sitemap from an ASP.NET MVC controller action. This is the code used to generate the sitemap for my blog.

public ActionResult SiteMap()
{
   EntityCollection<BlogEntryEntity> entities = 
      ServiceFactory.Beget<IEntityGenericService>().FetchAllEntities<EntityCollection<BlogEntryEntity>>();

   var mapData =
      from e in entities
      orderby e.DateInserted descending
      select new SiteMapData { 
         Loc = FormattingUtils.FormatUrlStart(Request.Url) + 
            Url.RouteUrl("Blog entry with id and title", new { BlogEntryId = e.BlogEntryId, blogEntrytitle = FormattingUtils.NormalizeTitle(e.Title) }), 
            Lastmod = e.LastTimeModified, 
            Priority = 0.5m, 
            Changefreq = "Weekly" };

   SiteMapHelper helper = new SiteMapHelper();

   XDocument siteMap = helper.GenerateSiteMap(mapData.ToList());
   MemoryStream compressedBytes = helper.GZipSiteMap(siteMap);

   FileStreamResult result = new FileStreamResult(compressedBytes, "gzip");
   result.FileDownloadName = "sitemap.xml.gz";
   return result;
}

The code used to fill in the SiteMapData list would change according to each specific situation. The use of the SiteMapHelper to the end of the function would be the same for most situations. Hopefully this illustrates the general idea.

One thing to keep in mind is that XML sitemaps can have at most 50,000 URLs and can be at most 10MB in size. So, if the list of SiteMapData nodes would generate a file that is larger than the given limits, then the list would have to be processed in chunks to generate separate sitemap files.

I share this with hope that it helps save someone a bit of time :-)

The entire SiteMapHelper code is shown below. It can be downloaded from the link at the top of this blog entry as well.

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.IO.Compression;
using System.Linq;
using System.Xml.Linq;

namespace FoxBlog.Shared
{
   [Serializable]
   public class SiteMapData
   {
      public string Loc { get; set; }
      public DateTime? Lastmod { get; set; }
      public string Changefreq { get; set; }
      public decimal? Priority { get; set; }
   }

   public class SiteMapHelper
   {
      XNamespace _xmlns = "http://www.sitemaps.org/schemas/sitemap/0.9";

      public XDocument GenerateSiteMap(List<SiteMapData> dataRows)
      {
         var xmlNodes =
            (from x in dataRows
               select CreateSiteMapUrlNode(x));
   
         XDocument siteMap = new XDocument(
            new XDeclaration("1.0", "utf-8", "yes"),
            new XElement(_xmlns + "urlset",
               new XAttribute(XNamespace.Xmlns + "xsd", "http://www.w3.org/2001/XMLSchema"),
               new XAttribute(XNamespace.Xmlns + "xsi", "http://www.w3.org/2001/XMLSchema-instance"), xmlNodes));

         return siteMap;
      }

      public MemoryStream GZipSiteMap(XDocument siteMap)
      {
         using (MemoryStream ms = new MemoryStream())
         {
            siteMap.Save(ms);
            ms.Seek(0, SeekOrigin.Begin);
            using (MemoryStream compressed = new MemoryStream())
            {
               // The zip stream has to be closed to write out all the bytes. The underlying stream
               // is closed as well. To get the bytes of the the closed stream ToArray must be called.
               using (GZipStream zip = new GZipStream(compressed, System.IO.Compression.CompressionMode.Compress))
               {
                  ms.CopyTo(zip, (int)ms.Length);
               }

               return new MemoryStream(compressed.ToArray());
            }
         }
      }

      private XElement CreateSiteMapUrlNode(SiteMapData data)
      {
         XElement node = new XElement(_xmlns + "url", new XElement(_xmlns + "loc", data.Loc));

         if ( data.Lastmod.HasValue )
         {
            node.Add(new XElement(_xmlns + "lastmod", data.Lastmod.Value.ToString("yyyy-MM-dd")));
         }
         
         if ( data.Changefreq != null )
         {
            node.Add(new XElement(_xmlns + "changefreq", data.Changefreq));
         }

         if ( data.Priority.HasValue )
         {
            node.Add(new XElement(_xmlns + "priority", data.Priority.ToString()));
         }

         return node;
      }
   }
}