SEO mistakes I've made and how I fixed them

Why SEO is so important?

You might know very little about what SEO does behind the scenes. To me, at least, it looked like an obscure, inconsistent, pseudo-science that only marketing people could understand (spoiler alert, it still kind of is). However, after getting @monicalent's awesome course bloggingfordevs, it made the inner workings and good practices related to SEO a bit clearer to me. To quote her from her first newsletter

SEO is a way of making sure that search engines can understand what your page is about, that it contains quality up-to-date information from an authoritative source, and will answer the question that the searcher had in mind.

With good SEO, search engines can know what your content is about, discover all the blog posts you've written and, if you're lucky, catapult you to the top search result for a given set of keywords. Moreover, where sharing my newest articles on Twitter and Reddit would just cause a spike in traffic for a few days, SEO helps you get a more consistent traffic on your website, and for a longer time. The latter is what I was lacking for the longest time, despite having set up my Gatsby website and SEO component properly (or at least I thought so).

Gatsby's documentation has an incredibly well-written section on how to build an SEO component to help you get started. However, that alone wasn't enough to make my blog discoverable early on, as you can see in the chart below representing the number of daily impressions I got since starting this blog:

For most of its first year, my blog was getting less than 50 daily impressions. Today, after fixing the issues I'm about to talk about, I get over 1000 daily impressions and it's still growing! Of course, SEO is not the only component here, I also created more content this year and choose a better way to promote them, but it is still a significant driver to the growth you can see above.

Trailing slashes chaos

The blog you're reading this article on is built with Gatsby and hosted on Netlify. Sadly, using these two tools together without taking care of inconsistent trailing slash / at the end of your URLs can result in some undesirable outcomes.

One of these outcomes was that I was seeing a lot of 301 redirects logged in my analytics as readers were navigating to my articles. On my blog, a link to one of my blog posts would typically look like this: /posts/learning-in-public but when a reader clicked on it Netlify would append a trailing slash at the end of it thus redirecting the user.

That, my friends, is extremely bad for SEO. It impacted several unrelated areas of my website, such as:

• Opengraph images or Twitter cards not being rendered consistently: readers would share a link sometimes with or without the trailing slash which would make it hard for some services to get the proper metadata and thus render a simple link instead of a proper preview card.
• Invalid URLs in sitemap: my sitemap is generated automatically at build time with a Gatsby plugin based on the URLs and pages of my website. Since I did not have trailing slashes at the end of my URLs it would generate my sitemap without them which once uploaded to Google Search Console would result in tons of warnings about invalid URLs since Google referenced the ones with the trailing slashes.

How I fixed this

I could have fixed this in two different ways:

1. Disable the "Pretty URLs" option in Netlify's asset optimization settings. (see screenshot below)
2. Add a trailing slash to all my URLs on my blog.

As Google already referenced my blog posts with a trailing slash, I decided to go with option number 2.

That change might look insignificant, but it resulted in a lot of weird issues suddenly disappearing. Additionally, it was essential for me to fix this before addressing the issue I'm just about to start talking about 😄!

Canonical links

If you've been following me for a while, you might have started reading my content on Medium. I started blog.maximeheckel.com just about a year ago as of the time I'm writing these words. All the content on this site that dates back to before August 2019, was originally published on Medium.

On top of that, I did not edit the articles when migrating them to this website, nor did I delete the Medium ones. This resulted in duplicated content, which meant that my newly deployed blog was in competition with Medium on the same keyword, the same content, from the same author when it comes to ranking on Google.

Thankfully there's a way to avoid this: setting canonical URLs. These URLs are placed in the <head> of your blog posts source code and designate that this post is the "original" post with that given content.

There are 2 steps to add valid canonical URLs to your website:

1. You need to add them to the <head> of your post. See example below
2. Head over to any third party platform you used in the past that has the content and add this canonical URL into the setting page of your post. I followed this guide on Medium to update my old blog post.

1
<link
2
  rel="canonical"
3
  href="https://blog.maximeheckel.com/posts/learning-in-public/"
4
/>

Of course, you cannot do the first step until you've fixed any potential trailing slashes issues you may have like the ones I shared just above.

1
import { graphql, StaticQuery } from 'gatsby';
2
import React from 'react';
3
import Helmet from 'react-helmet';
4

5
const query = graphql`
6
  query SEO {
7
    site {
8
      siteMetadata {
9
        defaultTitle: title
10
        author
11
        keywords
12
        siteUrl: url
13
        defaultDescription: description
14
        twitter
15
      }
16
    }
17
  }
18
`;
19

20
const SEO = ({ title, desc, image, pathname, date }) => (
21
  <StaticQuery
22
    query={query}
23
    render={({
24
      site: {
25
        siteMetadata: {
26
          author,
27
          defaultTitle,
28
          siteUrl,
29
          keywords,
30
          defaultDescription,
31
          twitter,
32
        },
33
      },
34
    }) => {
35
      const seo = {
36
        description: desc || defaultDescription,
37
        image: `${siteUrl}${image}`,
38
        date: date ? date : '',
39
        title: title || defaultTitle,
40
        url: `${siteUrl}/${pathname ? `posts/${pathname}` : ''}`,
41
      };
42

43
      return (
44
        <Helmet title={seo.title} defer={false}>
45
          <html lang="en" />
46
          <meta name="description" content={seo.description} />
47
          <meta name="image" content={seo.image} />
48
          <link rel="canonical" href={seo.url} />
49
          <meta property="og:url" content={seo.url} />
50
          <meta property="og:type" content="article" />
51
          <meta
52
            name="keywords"
53
            content={keywords && keywords.length > 0 ? keywords.join(`, `) : ''}
54
          />
55
          <meta property="og:title" content={seo.title} />
56
          <meta property="og:description" content={seo.description} />
57
          <meta property="og:image" content={seo.image} />
58
          <meta name="twitter:card" content="summary_large_image" />
59
          <meta name="twitter:creator" content={twitter} />
60
          <meta name="twitter:site" content={twitter} />
61
          <meta name="twitter:title" content={seo.title} />
62
          <meta name="twitter:description" content={seo.description} />
63
          <meta name="twitter:image" content={seo.image} />
64
        </Helmet>
65
      );
66
    }}
67
  />
68
);

Server-side rendering and missing meta tags

In this part, we'll look at the one instance where Gatsby's server-side rendering mixed with my carelessness completely broke my SEO. By completely I mean all my custom SEO meta tags that I carefully put in my SEO component were gone from the server-side rendered version of the website making it almost invisible to any search engine.

How it happened

This issue stemmed from what I would qualify as an interrupted static HTML build.

When building your Gatsby site the last steps of the build process involve building your production JS files and also generating the HTML for each page. If you're looking for more details you can check out this section of the Gatsby documentation about the build process.

However, I wrote a ThemeProvider that wrapped the whole application. Thus any component or page can know which theme (dark or light) is currently enabled and the colors to use. This component was added to the gatsby-ssr and gatsby-browser files.

Under the hood, this ThemeProvider worked as follow:

• the state of the theme (dark or light) was injected via a React Provider to the whole app, that's how I can allow users to toggle between each theme.
• that same state was also saved in the local storage to make sure revisiting the website would keep the previous theme enabled. When a reader loads this blog, the ThemeProvider will check for the presence of a specific variable in localStorage before setting the theme accordingly.

I dedicated a blog post for this: Switching off the lights - Adding dark mode to your React app and it actually contains the mistake that triggered the missing meta tags:

• Getting the variable set to the current theme from local storage was done in a React useEffect. Thus, for a brief instant when loading or refreshing the website, the website would fallback to the default theme as the effect to set the proper theme was only ran after the server-rendered page was already served.
• To avoid this issue, I added a small tweak to track whether the theme was fetched from local storage or not and render an empty div while the theme was being retrieved.

The code snippet below is an excerpt of my original implementation for the ThemeProvider of this blog.

1
const ThemeProvider = ({ children }: { children: ReactNode }) => {
2
  const [themeState, setThemeState] = useDarkMode();
3
  if (!themeState.hasThemeLoaded) {
4
    /*
5
      If the theme is not yet loaded we don't want to render
6
      this is just a workaround to avoid having the app rendering
7
      in light mode by default and then switch to dark mode while
8
      getting the theme state from localStorage
9
    */
10
    return <div />;
11
  }
12
  const theme = themeState.dark ? theme('dark') : theme('light');
13
  const toggle = () => {
14
    // toogle function goes here
15
  };
16

17
  // Once the theme is loaded, render the rest of the DOM
18
  return (
19
    <EmotionThemeProvider theme={theme}>
20
      <ThemeContext.Provider
21
        value={{
22
          dark: themeState.dark,
23
          toggle,
24
        }}
25
      >
26
        {children}
27
      </ThemeContext.Provider>
28
    </EmotionThemeProvider>
29
  );
30
};

Rendering that empty div is what made my SEO meta tags disappear. The static HTML build would only generate the tree up to that div since the theme had no way to be set at build time, and thus would skip all the rest of the DOM which included my pages and components, as well as the SEO component 😱. Since, the code of the SEO component was not being reached during that step of the build, the meta tags couldn't be injected in the static HTML.

Additionally, that kind of issue was hard to track because the tags were showing up when inspecting the page with the dev tools client-side, they were however completely absent from the source of the page (the one you can get by right-clicking on a web page and clicking on "View Page Source").

Having SEO tags completely missing from the page source made SEO third-party services like Twitter Card Validator simply unusable with my blog. My articles would only show up as basic links on social media. No cards, no preview, not even a title which is very bad when you try to grab the attention of your audience!

My SEO meta tags disappeared second time earlier this year in July 2020, after adding gatsby-plugin-feed and trying to make it work on my blog. Once again, be extremely careful when adding Gatsby plugins that can write on the <head/> of your pages. There might create some undesirable outcomes without even you knowing!

The long term solution

As you can imagine, I was tired of these issues coming out of nowhere and didn't want to manually check every single change I'd make in the future to ensure meta tags would not be removed. To this problem, I brought a solution that I'd usually bring up at work: I wrote an automated test.

1
const META_RE = /<meta\s[A-Za-z0-9="-:;!@\/\s]*/g;
2
const CANONICAL_RE = /rel="canonical"\s[A-Za-z0-9="-:;!@\/\s]*/g;
3

4
describe('SEO: Verify meta tag integrity', () => {
5
  it('has all the meta tags and the expected canonical url set in the landing page head', async () => {
6
    const res = await fetch('/');
7
    const text = await res.text();
8

9
    const metaTags = text.match(META_RE) || [];
10
    const canonicalTag = text.match(CANONICAL_RE) || [];
11

12
    expect(metaTags).to.have.length(16);
13
    expect(canonicalTag).to.have.length(1);
14
    cy.wrap(metaTags).snapshot();
15
    cy.wrap(canonicalTag).snapshot();
16
  });
17

18
  it('has all the meta tags and the expected canonical url set in the blog post head', async () => {
19
    const res = await fetch('/posts/how-to-build-first-eslint-rule');
20
    const text = await res.text();
21

22
    const metaTags = text.match(META_RE) || [];
23
    const canonicalTag = text.match(CANONICAL_RE) || [];
24

25
    expect(metaTags).to.have.length(19);
26
    expect(canonicalTag).to.have.length(1);
27
    cy.wrap(metaTags).snapshot();
28
    cy.wrap(canonicalTag).snapshot();
29
  });
30
});

(Don't judge my regex skills 😅)

The code snippet above is the test that I run with Cypress on every PR without exception. This test:

• fetches the source code of the landing page and a blog post against the built version of the blog
• looks at the text-based body of the request. That text-based result contains the HTML code for the entire page and thus should contain all the meta tags I set up in my SEO component.
• compares the obtained string of meta tags to a snapshot. That snapshot contains the source of truth when it comes to the expected state of my meta tags

Conclusion

In a nutshell:

• Be mindful of the consistency of your trailing slashes! Inconsistencies can lead to poor ranking.
• If you syndicate your content, do not forget to add canonical URLs. I was basically in competition with my own Medium posts up until late this year and missed on a lot more traffic and potential readers.
• Do not blindly trust gatsby plugins! Especially the ones that inject things in the <head> of your pages. If misused, they can be pretty harmful without even you knowing.
• Check the page source of your website! Inspecting via the dev tools is sometimes not enough to ensure the meta tags are properly injected in your site.
• Make sure your meta tags can't be blocked from rendering because of a client side side effect when relying on SSR.
• When in doubt: Write tests! I dedicated a whole blog post about CI/CD where I showcase how great tests and a great CI/CD pipeline help me keep my peace of mind.

If you want to go further into how to build an audience, and learn more about creating content and SEO, I would highly encourage you to follow @monicalent on Twitter as well as her Blogging For Devs course. She's an SEO expert and I learned more about efficient SEO techniques in a single newsletter than I would have otherwise!