Sitemap and Robots Generator in Episerver CMS

There are several plugins in the market that can help you with the creation of sitemaps and robots in Episerver, but we decided some time ago to create a new one to solve our own needs in our company and at the same time provide some added value to the current sitemaps plugins. In this post, we are going to talk about the Sitemap and Robots Generator plugin by Verndale, how to install it, configure it and use it. It is important to acknowledge that the plugin only works for Episerver CMS 11 or greater.

To begin, we are going to install the latest package 1.0.0.4 at the time of writing this blog using the Package Management Console and this command

Install-Package Verndale.Sitemap.Robots.Generator -Version 1.0.0.4

Please read carefully the Readme file. Some of the new changes we brought to this version are mentioned in there, but most important, it explains how to configure several parameters of the plugin which we will also explain meticulously later in this post.

After installation, we will copy the following lines to your base page, this is not required, but it is needed if you want more control over the pages that are going to appear in the sitemap, its frequency and priority. Please pay special attention to the comments in the code.

[Display(GroupName = "Navigation",
		Name = "Exclude from Xml Site Map",
		Order = 400)]
public virtual bool ExcludeFromSiteMap { get; set; } // Exclude this page from the sitemap
       
[Display(GroupName = "Navigation",
    Name = "Priority in Site Map",
    Order = 500)]
public virtual string Priority { get; set; } // The priority applied to this page in the sitemap
     
[Display(GroupName = "Navigation",
    Name = "Change Frequency of Page in Site Map",
    Order = 600)]
public virtual string ChangeFrequency { get; set; } // The frequency applied to this page in the sitemap

In addition, if you are using Web Api in your Episerver project, you need to add these lines to your Global.asax.cs.

protected override void RegisterRoutes(System.Web.Routing.RouteCollection routes)
{
    base.RegisterRoutes(routes);
    routes.Ignore("sitemap.xml");
    routes.Ignore("robots.txt");
}

With these changes in your code, it is enough to make the plugin work as expected. Now we are going to explain the app settings configurations which you can modify to fit your needs inside the web.config file.

    <add key="siteMapConfig" value="App_Configsitemap.config" />
    <add key="SiteMap_MAX_URLS" value="50000" />
    <add key="Show_Localized_URLS" value="true" />
    <add key="Display_Localized_URLS_Separated" value="false" />
    <add key="Show_Trailing_Slash" value="false" />
    <add key="SiteMap_OutPutRobots_txt" value="1" />

The sitemap.config file has the following configurations

Ke<?xml version="1.0"?>
<sitemapVariables>
  <sitemapVariable name="xmlNsTpl" value="http://www.sitemaps.org/schemas/sitemap/0.9" />
  <sitemapVariable name="xmlNsLangTpl" value="http://www.w3.org/1999/xhtml" />
  <sitemapVariable name="productionEnvironment" value="false" />
</sitemapVariables>

Now we will compile the code and go to the admin interface in the Episerver CMS editor, config tab and then to Sitemap and Robots Manager inside the Tool Settings section.

In this page there are 3 tabs that allows you to configure even more the plugin.

The first tab “Restricted types” allows you to restrict all pages in your site which belong to a specific type instead of doing it one by one. When you finish, click the update restricted types to save your changes.

The second tab “Sites” allows you to restrict sites. If the restricted check is set in here the sitemap will not be generated for that site. In addition, there is a text area field were you can set the information for the robots file also per site. When you finish, click the update sites to save your changes. It is important to mention that the Robots file does not need to be generated. If you saved your changes here, you will see those changes reflected in the robots.txt file immediately.

The last tab “Search engines” allows you to create links to engines that will be notified after sitemap generation. (This is a work in progress)

Finally, to generate the sitemap you must go to the admin interface in the Episerver CMS editor, admin tab and then to Sitemap Generation inside the Schedule Jobs section.

Click the start manually button to run the job and if you want to stop it you can click the stop button any time. Just remember, that before running this job for the first time you must save your configurations as explained above at least once.

To test the results just go your site url and at the end add sitemap.xml or robots.txt

That is it. If you have any questions or suggestions please let me know in the comments. I hope this plugin can help someone and as always keep learning !!!