Fallback after getServerSideProps() returns notFound

[davidbarratt]

Describe the feature you'd like to request

In order to transition from a legacy website to Next.js we need a series of fallbacks.

1. Predefined routes in Next.js
2. Catch-all route for CMS content where editors can specify whatever route they want, but should not be able to override the predefined routes
3. Fallback to legacy system if neither of the above conditions are met.

The problem is that Next.js does not allow one to have multiple fallback routes. In other words, if I have a catch-all route like pages/[[...path]].ts and that routes getServerSideProps() returns { notFound: true } then the fallback rewrites are not run.

This is a problem during a transition from a legacy system because there is no way to say "we can handle some routes, some CMS objects, and everything else should fallback."

Describe the solution you'd like

I would like a configuration option that would allow Next.js to continue the routing process if the first matched found route intends to return a not found status.

For instance, perhaps instead of this:

{
  notFound: true,
}

A route could return something like:

{
  notFound: true,
  fallback: true, // alternative names could be `continue`
}

This would allow each route to opt-into continuing the route matching.

This could also be useful in scenarios where you have something like this:

/pages/article/[slug].js
/[[...path]].js

If you would like, the article, if not found, could continue to the next matched path (and so on and so forth) until something actually handles the response.

Alternatively, a more powerful API could be to allow a callback funciton to be invoked to make routing decisions. This could be part of the rewrites config. Perhaps the source could accept an async callback function that recives a the current request object. This function wouldn't be called until the router failed to match anything before it.

Yet another alternative could be to allow for middleware to opt-into retrieving the actual response and rewritting the request multiple times (until you get something other than a 404, etc.). I realize that would break streaming, but it feels less worse than the solution I came up with.

Describe alternatives you've considered

My first attempt was to simple try doing it what I expected. As the docs say:

fallback rewrites are checked/applied, these are applied before rendering the 404 page and after dynamic routes/all static assets have been checked.

That isn't completely accurate. If a route is matched, that route is used and the fallbacks are never called even if { notFound: true } is returned. I kept getting the 404 page and never got to the fallback.

My second attempt was to instantiate a Custom Server from within an API Route. This worked for a single request and I think there is a memory leak in doing this because it quickly crashed Node.

My third attempt was to implement a Custom Server from the root and prevent Next.js from sending a response before I inspect the response, determine it to be a 404, and re-render. I could not find an easy way to do this. Despite my efforts, the headers and body are written directly to the stream and unless I was to override the methods in the response object (eww) there wasn't a great way to do this.

My final attempt, and the only thing that seemed to reasonably work, was to implement a Custom Server from the root and attach the app.render() method to the response object and from there I could use it from an API route:

const { createServer } = require('http');
const next = require('next');

const dev = process.env.NODE_ENV !== 'production';
const hostname = 'localhost';
const port = process.env.PORT || 3000;
// when using middleware `hostname` and `port` must be provided below
const app = next({ dev, hostname, port });
const handle = app.getRequestHandler();

app.prepare().then(() => {
  createServer(async (req, res) => {
    // Attach the renderer so the app can be rendered from API routes.
    res.render = app.render.bind(app);

    try {
      await handle(req, res);
    } catch (err) {
      console.error('Error occurred handling', req.url, err);
      res.statusCode = 500;
      res.end('internal server error');
    }
  }).listen(port, () => {
    console.log(`> Ready on http://${hostname}:${port}`);
  });
});

and in the API route handler:

export default async function handler(
  req: NextApiRequest,
  res: NextApiResponse & RenderableResponse
): Promise<void> {
  const target = new URL(req.url, `http://${req.headers.host}`);
  const id = await getCMSPageIdByPath(target.pathname);

  // Fallback to a CMS page if Next.js knows how to handle it.
  if (id) {
    const parsedUrl = parse(req.url, true);
    const { query } = parsedUrl;
    return res.render(req, res, `/_cms/${id}`, query);
  }

  // Fallback to Legacy site
  return legacy(req, res);
}

This isn't great, but it does work. I do get this warning in the console:

warn  - You should not access 'res' after getServerSideProps resolves.
Read more: https://nextjs.org/docs/messages/gssp-no-mutating-res

It feels like Next.js is missing a feature that would make a huge improvment on migrating from one system to another. Of course we wont need this features as soon as we are done with the migration.

[bertho-zero]

Is there a less hacky solution?

[davidbarratt]

If there is, I haven't found it.

[KELiON]

Main problem with custom server solution is that it doesn't support Vercel deployments. And looks like now there is no way to make it alive on Vercel.

We use rewrites in next.config.js like this:

async rewrites() {
  return {
    fallback: [
      {
        source: "/:match*(/?)",
        destination: `${process.env.BACKEND_HOST}/:match*`,
      },
    ],
  };
}

and expect that returning { notFound: true } should be a sign to follow this fallback rule

[KELiON]

Ok, there is a working solution that also supports Vercel deployments.

Instead of returning { notFound: true } http-proxy can be used to proxy request to backend, like this:

import httpProxy from "http-proxy";
import { GetServerSidePropsContext } from "next";

// In TS we say that it returns `NotFound` type but in reality 
// `Promise` is never resolved and `http-proxy` generates response using `context.res` directly
type NotFound = { notFound: true };

const proxyToBackend = async (context: GetServerSidePropsContext): Promise<NotFound> =>
  new Promise(() => {
    context.res.setHeader("Cache-Control", "no-cache");
 
    // In some cases `changeOrigin: true` should be used 
    const proxy = httpProxy.createProxyServer();
    proxy.web(context.req, context.res, { target: process.env.RAYCAST_STORE_HOST });
  });

export default proxyToBackend;

[davidbarratt]

so maybe something like this?

export function getServerSideProps(context: GetServerSidePropsContext): Promise<GetServerSidePropsResult<{}>> {
  return new Promise((resolve, reject) => {
    const proxy = createProxyServer({
      target: 'http://backend.example',
      changeOrigin: true,
      xfwd: true,
    });

    proxy.on('proxyReq', proxyReq => {
      proxyReq.on('close', () => resolve({ props: {}));
    });

    proxy.on('error', reject);

    proxy.web(context.req, context.res);
  });
}

[davidbarratt]

And if this page is one of the fallbacks I assume Next.js wont attempt to make a request to /_next/data ? or is there a better way to prevent that from happening?

EDIT: I suppose I could check for /_next/data and if it is there, redirect to the resolvedUrl ?

[ijjk]

It would request _next/data if next/link is used on the client for those paths although for those you can return an empty 404 (res.statusCode = 404 with an empty body) which will trigger a hard navigation on the client.

[davidbarratt]

ok, so a more complete example could be something like:

export async function getServerSideProps(context: GetServerSidePropsContext): Promise<GetServerSidePropsResult> {
    const page = await getCMSPageByPath(context.resolvedUrl);

    if (page) {
        return {
            props: {
                page,
            }
        };
    }

    // Signal to the client to perform a hard navigation.
    if (context.req.url.startsWith('/_next/data')) {
        context.res.writeHead(404).end();
        return {
            props: {},
        };
    }

    // Fallback to legacy system
    await new Promise((resolve, reject) => {
      const proxy = createProxyServer({
        target: 'http://legacy.example',
        changeOrigin: true,
        xfwd: true,
      });
  
      proxy.on('proxyReq', proxyReq => {
        proxyReq.on('close', resolve);
      });
  
      proxy.on('error', reject);
  
      proxy.web(context.req, context.res);
    });

    return {
        props: {},
    };
  }

[davidbarratt]

This solution results in this warning:

warn  - You should not access 'res' after getServerSideProps resolves.
Read more: https://nextjs.org/docs/messages/gssp-no-mutating-res

I don't know a way around that though

[ericsalerno]

Hi folks, I just wanted to add some support for this idea. My team is working on supporting some legacy rewrites/redirects in a NextJS web app where the functionality may be different depending on the specific endpoint/page the user lands on. While we are able to return a redirect or a 404, if we were also able to proxy (or rewrite) an endpoint as an option we thought it would be a much cleaner developer experience. While we are currently considering a handful of other options for the feature, we believe this would be a powerful and useful feature to include in NextJS.

To go into a bit more detail on our use case, in essence, we have a legacy database with over 2 million rows of URL slugs to content that may have had its slug updated or manually input. These slugs live alongside a rewrite/redirect destination. A large subset of the contents of this table were deemed important to reimplement at a business level.

Since these routes may surface either a redirect or a rewrite/proxy, one of the solutions we were looking into was using NextJS middleware to test the current route against the database. We were running into limitations with the Edge Runtime communicating with Redis, where we cache route mappings. Since these are relatively high traffic routes, we were hesitant to put an extra HTTP request in front of every URL that matched.

Ideally we would be able to modify the page route configurations we intend to rewrite/redirect to and within their getServerSideProps return the proxy/rewrite to the appropriate route. We understand that supporting this when an external (node-http-proxy) library based solution may exist may not make sense, but we’d be really interested in pursuing this as a solution.

[davidbarratt]

A better work around, and the one I ended up implementing, was using nginx to catch a 404 from Next.js and retry the request on the legacy system. This worked surprisingly well, but was obviously more complex.

[dimitriBouteille]

Hi @davidbarratt

Any news ? This feature can be added in next ❤️

[davidbarratt]

As previously mentioned, I would use nginx.

[dimitriBouteille]

For technical reason, I can only use node server

[davidbarratt]

I guess your best option would be to create a (reverse-)proxy in Node.js, or you could use another service like a Cloudflare Worker, etc.