Generate sitemap for static site with dynamic routes

[helloworld]

Hey all,

I'm trying to build a sitemap for my static site generated by Next.js.

I currently don't explicitly define a exportPathMap, because I want to rely on the functionality of the CLI build and export commands to automatically read my /pages directory and build all my pages.

This functionality is really helpful because I have two dynamic routes that use getStaticPaths, which the CLI uses to determine all the pages that should be built.

When I run build, I get this output:

I've seen several tutorials/plugins on how to generate sitemaps for Next.js sites, like this, this, and this, but the solutions either:

• Require a server (dealbreaker since I'm trying to export a static site)
• Do a simple traversal of the /pages directory to get all routes (doesn't work for me since I'm using dynamic routes)
• Require an explicit exportPathMap that includes every page.

Is there any way to hook into some API (private or public) that will let me just get all the pages in my site. I basically just want to programmatically generate the data seen in the screenshot above. Then, I can use that to generate my sitemap with a script post build and export using that data.

I would really appreciate any help or pointers here. Thank you!

[leerob]

Edit: Moved to docs #28346

[leerob]

Is there any way to hook into some API (private or public) that will let me just get all the pages in my site.

This would be really nice! 💯

[landsman]

Exactly this is what I am looking for too. Calling some Next.js API which would give me abstraction for pages (without using poor .map() on listing in file system).

[Sowed]

const modules = await globby(['modules/**/*{.js,.mdx}']);
const docs = await globby(['docs/**/*{.js,.mdx}']);

@leerob, am assuming that both modules and docs will output only the files that where already present in those directories before build. I am right? @helloworld, will this also include the dynamic files created at build time from getStaticPaths? 🤔

[jack828]

I just want to bring to the surface a potential issue with this solution - if you have a leading newline most XML parsers will treat it as invalid. I am unsure how search engines will ingest it with that.

Otherwise - thank you so much for this. It works great for me with some minor changes for my use case.

Easy fix for the issue I mentioned is to use your prettier version in this #12209 (comment)

[TommySorensen]

@helloworld
You can use the getServerSideProps to set resto make the sitemap live by creating a sitemap.xml.jsfile under /pages. It's kind of a hack. But it works fine.

// pages/sitemap.xml.js

const generateSitemap =(data, origin) => {
  let xml = ''

  data.pages.map(page => {
    xml += `<url>
      <loc>${origin + page.location}</loc>
      <lastmod>${page.lastMod}</lastmod>
    </url>`
  })

  return `<?xml version="1.0" encoding="UTF-8"?>
    <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
      ${xml}
    </urlset>`
}

export async function getServerSideProps({ res }) {
  const data = /* do some async fetching here */ 

  res.setHeader('Content-Type', 'text/xml')
  res.write(generateSitemap(data, 'http://yoursite.com'))
  res.end()

  return {
    props: {},
  }
}

const SitemapIndex = () => null
export default SitemapIndex

This will work perfectly for dynamic sitemaps 👍
You can also to the same with a robots.txt.js file. Just set the Content-Type to text/plain

[pejantantangguh]

GENIUS!
Thanks for the help.

[leerob]

We are adding official docs on sitemaps, please leave any feedback there! 🙏