Fermin Perdomo

Senior Full Stack Engineer | PHP | JavaScript

Creating Large Dynamic Sitemaps with Next.js, Drupal, and next-sitemap

Fermin Perdomo
December 6, 2023

A sitemap is an XML file containing the list of indexable URLs of a domain. When sitemaps become large, they are split into 1 sitemap index file that point to multiple sitemap files. Learn more about splitting sitemaps with Google’s documentation. next-sitemap is a library that conveniently generates the sitemap XML document after reading the Next.js build manifests or when given a list of URLs. Check out some real examples, like the Google sitemap index or the sitemap of this website.

Static Sitemaps (Build Time)

Static routes generated at build time are automatically picked up by next-sitemap.
 This works for both:

  • Static pages
  • Paths generated by getStaticPaths

It works out of the box!

You may also add other options, such as:

  • exclude (paths to ignore)
  • additionalPaths
  • generateRobotsTxt

To automate generation after building, add it to the postbuild step in package.json:

{
  "scripts": {
    "build": "next build",
    "postbuild": "next-sitemap"
  }
}

Dynamic Sitemaps (Runtime)

For user-generated URLs, pulling everything at build time means your sitemaps only update on deployment.
 Instead, you can generate them on demand at runtime.

Required Routes

You’ll need routes to render:

  • A sitemap index
  • Individual sitemap pages

Example URLs:

/server-sitemap.xml
/server-sitemap-0.xml
/server-sitemap-1.xml
...

Since Next.js doesn’t allow dynamic filenames like server-sitemap-[page].ts, use rewrites.

Steps to Implement

1. Create Pages

  • pages/server-sitemap.xml/index.ts
  • pages/server-sitemap/[page].ts

2. Add Rewrites

In next.config.js:

async rewrites() {
  return [
    {
      source: "/server-sitemap-:page.xml",
      destination: "/server-sitemap/[page].ts",
    },
  ];
}

Using next-sitemap APIs

next-sitemap provides two APIs for server-side generation:

  1. getServerSideSitemapIndex – Generates the sitemap index file.
    • Fetch how many sitemap pages exist
    • Pass their URLs to getServerSideSitemapIndexLegacy
  2. getServerSideSitemap – Generates a single sitemap file.
    • Fetch the corresponding page
    • Pass its URLs to getServerSideSitemapLegacy

Caching Dynamic Sitemaps

Dynamic sitemaps usually hit your API or DB for many items — avoid overloading them.

Options:

  • Use the Cache-Control header (supported by Next.js SSR)
  • On Vercel, caching works automatically
  • For self-hosted setups, configure Redis or similar

⚠️ Keep in mind: response size cannot exceed 10 MB.

Learn More


Reactions

Loading reactions...
Log in to react to this post.

Comments

Please login to leave a comment.

Newsletter