Sitemaps: A Quick SEO Win for New Websites

I recently added sitemaps to a few Rails-based marketing sites I manage for clients, and within weeks, we saw a noticeable boost in search traffic. It reminded me that not many people know about them or have totally forgotten about them (it's quite an old but effective SEO technique). Even fewer know about Google Search Console (not Google Analytics), the official Google tool where you can submit sitemaps and request indexing for new pages as soon as you publish them.

So wanted to give a quick primer about what a sitemap is, how you can create it yourself in Rails, and how to submit it to Google. Sometimes it's these small technical things that can make a big difference.

SEO is all about helping search engines better understand your website's content, which helps your target readers find your website faster. If you're selling anything online, better SEO = more traffic to your website = more sales for whatever you're selling.

Most search engines like Google automatically crawl the Internet looking for new pages to add to their index. The thing is, there is not a single database of all the web pages in the world, so Google must constantly look for new pages to index them. Most often, Google discovers a website because another website that's already indexed links to a page from that website.

So the obvious next question is: how can a new website which doesn't have any external links can tell Google to index itself?

Of course, you can use SEO agencies that spam reputed websites asking them to backlink to the new website, or ask to submit junk guest posts that link back to the website. I get probably dozens of such emails every week to get backlinks from this blog. Thanks to HEY, most of those emails go straight to spam, and I never even see them (not an ad, just a happy user and big fan).

Anyway, the most effective SEO approach that works is to play the long term game, and write content that people want to read. Write content that shows experience, expertise, authoritativeness, and builds trust, also known as E-E-A-T framework in SEO circles.

Here's the Google Console report for Write Software, Well for the past year. The wheels of SEO turn slowly, but they grind exceedingly fine.

Got it, but what about a new blog?

When you start a new blog or a new marketing website for your company, there's one simple thing you can do to tell Google about which pages your site has so its crawlers can index them. Use a sitemap.

What's a sitemap?

A sitemap is an XML file that lists the pages, videos, documents and other files on your website and the relationships between them. It tells Google which pages to index and when they were last updated.

Do I need a sitemap?

Yes, you do. Although Google will discover most of the pages eventually, why wait? If your site is new, it won't have any external links to it. Google searches the web by crawling URLs found in previously crawled pages. Hence it might take too long if there're no pages linking to your site.

I'm sold. How can I build a sitemap?

Most likely, the platform you're using already generates the sitemap. For example, I've been using the excellent Ghost platform for the past three years to write this blog (again, not an ad, just a happy user). It automatically creates the sitemap based on the latest entries. You can check it out at /sitemap.xml (click the links to explore further). This is how it looks:

If you have a custom-built CMS, check out the gems or libraries provided by your web framework. For example, in Rails you can use the sitemap_generator gem. If you're using Jekyll, there's a jekyll-sitemap plugin, and so on. If you're the non-technical owner of the website, talk to your developers about building one.

I have built and maintain a few simple marketing websites with Rails for some of my clients, and have found it easier to just build the sitemap in plain Ruby. Here's a quick demo of how you can do it in Rails for a simple blog.

How to Generate Sitemap in Rails

When creating a sitemap, you're essentially telling the crawlers about all the URLs on your site you want to show in the search results. Also note that these are the canonical URLs, i.e complete URLs and not paths.

First, you'll need a route for the /sitemap.xml endpoint.

get "sitemap", to: "sitemap#index", defaults: { format: "xml" }

Next, add the following SitemapController:

class SitemapController < ApplicationController
  # GET /sitemap.xml
  def index
    # Cache the sitemap for 1 hour
    expires_in 1.hour, public: true

    @posts = Post.published.includes(:tags).order(published_on: :desc)
    @pages = Page.all.order(:title)
    @tags = Tag.joins(:posts).merge(Post.published).distinct.order(:name)
    @static_pages = static_pages_data

    respond_to do |format|
      format.xml { render layout: false, content_type: "application/xml" }
    end
  end

  private

  def static_pages_data
    [
      {
        url: root_url,
        lastmod: @posts.first&.updated_at || Time.current,
        changefreq: "weekly",
        priority: "1.0"
      },
      {
        url: blog_url,
        lastmod: @posts.first&.updated_at || Time.current,
        changefreq: "daily",
        priority: "0.9"
      },
      {
        url: testimonials_url,
        lastmod: 6.months.ago,
        changefreq: "monthly",
        priority: "0.7"
      }
    ]
  end
end

With help from Cursor, or any other AI tools, you can pass the above template along with your routes.rb file for context, and it will update the controller to include all other endpoints in your site for complete sitemap.xml file.

Finally, add the XML view template to generate the final sitemap.

xml.instruct! :xml, version: "1.0", encoding: "UTF-8"
xml.urlset xmlns: "http://www.sitemaps.org/schemas/sitemap/0.9" do
  # Static pages (home, blog, testimonials, etc.)
  @static_pages.each do |page_data|
    xml.url do
      xml.loc page_data[:url]
      xml.lastmod page_data[:lastmod].iso8601
      xml.changefreq page_data[:changefreq]
      xml.priority page_data[:priority]
    end
  end

  # Dynamic pages (about, contact, etc.)
  @pages.each do |page|
    xml.url do
      xml.loc page_url(page.slug)
      xml.lastmod page.updated_at.iso8601
      xml.changefreq "monthly"
      xml.priority "0.6"
    end
  end

  # Published blog posts
  @posts.each do |post|
    xml.url do
      xml.loc post_url(post)
      xml.lastmod post.updated_at.iso8601
      xml.changefreq "monthly"
      xml.priority "0.8"
    end
  end

  # Tag pages (only those with published posts)
  @tags.each do |tag|
    xml.url do
      xml.loc tag_url(tag)
      xml.lastmod tag.updated_at.iso8601
      xml.changefreq "monthly"
      xml.priority "0.5"
    end
  end
end

That's it, now restart your Rails app and visit the sitemap.xml endpoint. You'll be presented with a sitemap.xml file, filled with all the pages of your website.

I have a sitemap, now what?

Now you can either wait for Google to read that sitemap next time it decides to visit your website, or you can be proactive and submit it to Google. For this, you will need a Google Search Console account. Not Google Analytics, but Google Search Console. These are two completely separate things.

First, get set up on Search Console. You'll have to verify that you're the owner of the site using your domain registrar (takes less than a minute) and then go to the dashboard. Click on "Sitemaps".

You'll be prompted to enter the sitemap URL here. Just make sure you can access the URL in a separate browser tab, and then enter it in the input field. Click submit.

That's it. You've now told Google what links it should crawl and have also provided additional metadata which helps with crawling and indexing. Now keep writing helpful and useful posts and let the wheels of SEO grind.

But wait, there one other bonus tip I wanted to share. Every time you publish a new post, tell Google.

Submit New Posts on Search Console

Let's say you've written a new article or added a new page on your website. You can either wait for Google to crawl it, or you can actively tell Google. For this, go to Search Console, and at the top you'll see this input field: "Inspect any URL".

Enter the URL of your new post, and it will show a screen similar to this.

Click on Request Indexing, and you're good to go. Google will add the link to its list of pages to be crawled.

Now go write some awesome posts and share them with the world.