Implementing Tag Revalidation Caching using NextJS and Sanity

At the time I joined RudderStack, the marketing site was built on Gatsby. One of our main internal customers is the marketing team, with a considerable number of users creating, editing, and publishing varying types of content to the site. Now, a year and a half later, our current major priority is migrating rudderstack.com to Next.js. At this point, a large portion of the site has been incrementally migrated, unlocking potential cutting-edge features. One such feature we recently implemented is the new tag revalidation (or on-demand) caching strategy from Next.js. This new feature decreased our site build times, decreased requests to Sanity, our CMS, simplified our content editors' workflow, and significantly sped up release times. Cost ⬇️ + Efficiency ⬆️ = Business 📈

In this guide, I’ll provide the details of our implementation with a nitty gritty how-to. First, I’ll give a short overview of the project and the impact it’s had on our build times and workflow. Here’s a look at our web stack for context before we get started:

• Content Management System: Sanity
• Deployment and Hosting: Vercel
• Frameworks: Gatsby & Next.js
• Notable Dependencies: TypeScript, Material UI, Turborepo

Overview

Many websites use static site generators to create a fully-static HTML website from raw data and a set of templates. This method automates the coding of individual HTML pages and prepares them to be served to users before they are requested. There are several benefits to server-side rendering, namely speed and caching.

Other applications require more dynamic data (like the information only available to a logged-in user). In these cases, applications may generate pages at request time or push code to users' browsers that offloads the fetching of data and building of HTML. There are a few benefits to client-side rendering.

Next.js is a React framework for building full-stack web applications, offering both server-side and client-side rendering. For a marketing website like ours, rudderstack.com, we want as render as many pages on the server as possible. Next.js allows us to generate those pages during the build process.

The Next.js build process occurs frequently at RudderStack – whenever the development team pushes code to a branch in our GitHub repository, when website content editors from business teams manually trigger a deployment via the Sanity Studio, or when we deploy from the Vercel CLI. This process takes about 4-8 minutes to complete, delaying our content editors from immediately releasing changes to production. To enable a more efficient workflow, we wanted to drastically reduce these build times.

Production Releases: Before & After

Before we changed our caching strategy, our implementation of Next and Sanity required a deploy every time we wanted to release a content change to production because we wanted the speed of statically generated pages on the server. The flow was as follows:

1. Make content edits – Sanity utilizes a "published" state to determine which content is production-ready
2. Trigger Deploy – custom Sanity dashboard buttons allow our content team to trigger Vercel deploys
3. Build Process – a 5-10 minute build process begins where data is fetched from Sanity and used to build out statically generated pages
4. Production Release – at the end of a successful build, the deployment process automatically releases the build to production at rudderstack.com

This flow worked well, but releases could take longer than desired and would be delayed if there happened to be an error in the build process. With the tag revalidation caching strategy, our new flow follows this pattern:

1. Make content edits – Sanity utilizes a "published" state to determine which content is production-ready. This now triggers a webhook that hits an endpoint on the Next app with the type of data that was changed (

   tag

   ).
2. Tag Revalidation – the endpoint on the frontend fires off the new

   revalidateTag()

   function, causing the server to refetch any queries that are dependent on the

   tag

   that was received from the Sanity webhook.
3. Server Re-render – any pages that were dependent on that newly fetched (revalidated) data are re-rendered on the server and served up on the next request.

This new flow significantly decreased both our total build times and total fetches to Sanity, saving us money. Perhaps more importantly, it simplified our content editor's workflow and sped up our release times, increasing our efficiency.

Implementation: On-demand revalidation with Next.js

The Next.js cache tagging system for invalidating

fetch

revalidateTag()

Utilizing this functionality gave us nearly-instantaneous deploy-less content updates.

Critical Dependencies

Please note that that a critical change for this functionality landed in

next

• next

  @ 14.0.4 (App Router)

• next-sanity

  @ 7.0.3

• sanity

  @ 3.20.2

Server-side Rendering

If you're coming from the world of single-page applications (SPAs), you may be familiar with the concept of loading client-side Javascript that fetches data and utilizes libraries like React or Vue to render and display that content. And if you've been a developer for any amount of time, you know that nothing stays new for very long. In fact, you may have realized that the web works in cycles and we’re all in the process of getting back to our roots. In the past, everything was rendered on the server. Then, up until recently, we were rendering entirely too much on the client. Now, we’re attempting to tackle the best of both worlds: the speed of the server and the dynamic nature of the client. Enter React Server Components, or RSCs, which individually fetch data and render HTML entirely on the server, eliminating the need for client-side re-rendering, thereby improving performance.

React Server Components

One of the reasons we love Next is due to their opinionated, prolific use of RSCs. If you spin up a brand new Next app and include TypeScript, you'll get some wonderful tooling that helps alleviate some of the confusion around when, where, and how to implement RSCs. If you're unfamiliar with the flow and benefits of RSCs, Next has some great documentation on rendering environments and Vercel's blog post, "Understanding React Server Components" is a must-read. Josh W. Comeau has a great primer on RSCs, as well.

Front-end Implementation

Instrumenting

Before the introduction of the tag revalidation caching strategy,

next-sanity

@sanity/client

React.cache()

TSX
Copy
// @/lib/client.tsx

import { createClient, type ClientConfig } from 'next-sanity'
import { cache } from 'react'

const projectId = process.env.NEXT_PUBLIC_SANITY_PROJECT_ID
const dataset = process.env.NEXT_PUBLIC_SANITY_PROJECT_DATASET
const apiVersion = process.env.NEXT_PUBLIC_SANITY_API_VERSION
const token = process.env.SANITY_READ_TOKEN

const baseConfig = {
  projectId,
  dataset,
  apiVersion,
  useCdn: false,
  token,
  studioUrl: 'https://rudderstack.sanity.studio/',
  encodeSourceMap: false,
  perspective: 'published'
} as ClientConfig
export const client = createClient(baseConfig)

// Wrap the cache function in a way that reuses the TypeScript definitions
const clientFetch = cache(client.fetch.bind(client))

This gave us optimized and cached fetches but only within the React context, meaning that if an exact duplicate request was made, React would prevent the fetch to Sanity and utilize locally cached data. A welcome optimization, for sure, but unhelpful when the data has changed since the initial request response was cached.

With the advent of tags, we were able to remove

React.cache()

client.tsx

clientFetch()

tags

TSX
Copy
// @/lib/fetch.tsx

import 'server-only'
import type { QueryParams } from 'next-sanity'
import { draftMode } from 'next/headers'
import { client } from './client'
import { token } from '@/lib/constants'

const DEFAULT_PARAMS = {} as QueryParams
const DEFAULT_TAGS = [] as string[]

export async function clientFetch<QueryResponse>({
  query,
  params = DEFAULT_PARAMS,
  tags = DEFAULT_TAGS,
}: {
  query: string
  params?: QueryParams
  tags: string[]
}): Promise<QueryResponse> {
  const isDraftMode = draftMode().isEnabled
  if (isDraftMode && !token) {
    throw new Error('The `SANITY_READ_TOKEN` environment variable is required.')
  }

  return client.fetch<QueryResponse>(query, params, {
    token,
    ...(isDraftMode && {
      perspective: 'previewDrafts',
    }),
    next: {
      ...(isDraftMode && { revalidate: 30 }),
      tags, // for tag-based revalidation
    },
  })
}

This new function also receives a return type (

QueryResponse

isDraftMode

next/headers:draftMode()

Fetching

Now that our new tag-revalidated fetching functionality is available, we can pass an array of

tags

tags

tags

tags

tags

types

_type

Sanity utilizes a custom query language called GROQ which is very powerful but has a bit of a learning curve. To fetch the necessary data to render rudderstack.com/learn, our

groq

TSX
Copy
// app/learn/queries.tsx

import { groq } from 'next-sanity'

export const learnQuery = groq`{
  'articles': *[_type == "learning_center"] {
    _updatedAt,
    title,
    slug {
      current
    },
    'category': learning_category->{
      title,
      slug { 
        current
      }
    },
    'description': short_description
  },
  'categories': *[_type == 'learningCenterCategories'] {
    title,
    slug {
      current
    }
  }
}`

Without worrying too much about the syntax or the mixing of snake- and camel-case, you can see that two types of documents are being fetched:

learning_center

learningCenterCategories

When we use this query, we ensure that those document types are passed as

tags

TSX
Copy
// app/learn/page.tsx

import { clientFetch } from '@/lib/fetch'
import { learnQuery } from './queries'
import { LearnLayoutType } from './types'
import { Learn } from './Renderer'

const getData: () => Promise<LearnLayoutType | void> = async () => {
  const data = await clientFetch<LearnLayoutType>({
    query: learnQuery,
    tags: ['learning_center', 'learningCenterCategories']
  }).catch((err) => console.error(err))

  if (!data) throw new Error('No data returned for LearnIndex')

  return data
}

const LearnIndex = async () => {
  const data = await getData()
  if (!data) return <FourOhFour />
  return <Learn data={data} />
}

export default LearnIndex

Now, whenever the

learning_center

learningCenterCategories

API Route Handler

So, how do we revalidate those

tags

clientFetch()

/learn

revalidateTag

TSX
Copy
// api/revalidate/route.tsx

import { revalidateTag } from 'next/cache'
import type { NextApiRequest } from 'next'
import type { NextRequest } from 'next/server'
import { NextResponse } from 'next/server'
import { parseBody } from 'next-sanity/webhook'

export const runtime = 'edge'

export async function POST(req: NextRequest) {
  try {
    const { isValidSignature, body } = await parseBody<BodyInit & { _type: string; }>(
      req as unknown as NextApiRequest,
      process.env.SANITY_REVALIDATE_SECRET,
    )

    if (!isValidSignature) {
      const message = 'Invalid signature'
      return new Response(JSON.stringify({message, isValidSignature, body}), {status: 401})
    }

    if (!body?._type) {
      return new Response(body, {status: 400})
    }

    // All `client.fetch` calls with `{next: {tags: [_type]}}` will be revalidated
    revalidateTag(body._type)
    console.log(`Revalidated ${body._type}`)

    return NextResponse.json({
      status: 200,
      revalidated: true,
      now: Date.now(),
      body
    })
  } catch (err: unknown) {
    console.error(err)
    if (err instanceof Error) {
      return new Response(err.message, { status: 500 })
    }
    return new Response('Error', { status: 500 })
  }
}

First, we use

next-sanity/webhook:parseBody()

process.env.SANITY_REVALIDATE_SECRET

If the request includes a

_type

revalidateTag()

types

tags

Upon successful revalidation, we return a

200

revalidated

body

At this point, you'll need to deploy your next app to Vercel in order to expose your API endpoint publicly. It won't work until you create your Sanity webhook, generate a Secret, and add it to the Environmental Variables in your Vercel project and copy the deployment URL.

Sanity Webhook

Head over to Sanity's project management tool (not the Studio) to create the webhook that will ping the revalidate endpoint when any document is published. The URL for your Sanity project management tool should follow this format, where the

organizationId

projectId

https://www.sanity.io/organizations/[organizationId]/project/[projectId]

Once you're in, click the API tab and the button to create a new webhook. Paste the deployment URL you copied from the API Route Handler step above, appending

/api/revalidate

• Dataset – 

  development

  , assuming you have a dataset that can be tested against
• Trigger On – check all 3: create, update, and delete
• Filter – leave this blank unless there are only particular types of documents that you want to revalidate
• Projection – add "

  { _type }

  "
• Status – check this and enable the webhook
• HTTP Method –

  POST

• HTTP Headers – leave blank
• API Version – the latest here should be fine but ensure that it aligns with what you have set in your Sanity project
• Drafts – leave unchecked

At the bottom of the list, generate a Secret and copy the value. Save your new webhook and return to the webhook listing page. To the right of your newly-created webhook, you'll see 2 buttons, one to expose more options. Click it and select "Show attempt log", opening a new tab with the attempt log.

Sanity's Webhook Options

Head back to Vercel and open your Next project. Click the "Settings" tab, then "Environmental Variables" on the left sidebar. Create a new ENV VAR called

SANITY_REVALIDATE_SECRET

Testing

At this point, you should be able to publish a document change in Sanity, and, after a few moments, see that the deployment URL that corresponds to that change (in our example above, we would append

/learn

If you're having issues, check the attempts log you opened earlier. You can also open up the logs for your Vercel project and expand the "Functions" filter in the sidebar, selecting your new

api/revalidate

Once you're ready to go to production, be sure to change your webhook Dataset to production.