jessie.codes

GitHub Actions for Node Modules in a Monorepo

One of the projects I have been working on uses a monorepo, where we have multiple frontend applications with shared private dependencies. We decided all of our shared node modules would live under one top-level folder to keep things organized. When it came time to set up our CI for the project, I found that writing some custom bash scripts was the easiest way to avoid setting up a workflow per module. This strategy might not work well for you if you have quite a few private packages; however, since we only have a handful, I determined our best bet would be to have one workflow that ran anytime any of our packages were updated. As we keep all of our these packages under a folder named `private_modules`, I set the workflow to target anything with a path of `private_modules/**` and then had it run a custom shell script that could take in a yarn script to run. We're using [yarn](https://yarnpkg.com/) to manage our dependencies, but this should work with npm as well. ```yml name: Private Packages on: pull_request: paths: - "private_modules/**" jobs: test: runs-on: ubuntu-latest strategy: matrix: node-version: [12.x] steps: - uses: actions/checkout@v1 - name: Use Node.js ${{ matrix.node-version }} uses: actions/setup-node@v1 with: node-version: ${{ matrix.node-version }} - name: yarn install and test run: | cd private_modules ./yarn_run.sh test ``` As you've seen, all we have to do without our workflow file is `cd` into the top-level directory and run our bash script. If you need to run more than one yarn command, you can either add a new step for each command or update the bash script to take multiple commands. The bash script itself is straight forward. It will loop through all of the subdirectories within the folder, install any dependencies, and then run the yarn script specified in the arguments. This way, we could use it for all of our CI steps, whether it be linting, testing, or publishing. ```bash #!/bin/bash CMD=$1 dir_resolve() { cd "$1" 2>/dev/null || return $? echo "`pwd -P`" } for dir in ./*/ do ( abs_path="`dir_resolve \"$dir\"`" echo "Installing $abs_path" cd $abs_path yarn install yarn run $CMD --max-warnings=0 2>/dev/null if [ $? -ne "0" ] then exit 1 fi ) || exit 1 done ``` Using this bash script has kept our workflows both easy to manage and maintain. The main thing to keep in mind is that every package will need to have the yarn command configured that you intend to pass into the bash script, or the script will fail.

Running Cypress in Gitlab CI

I ran into some difficulties a few weeks ago when attempting to run [Cypress](https://www.cypress.io/) tests as part of my CI script in Gitlab. The application I was working with has a Golang backend and the front end is generated with [Nuxt](https://nuxtjs.org/). As this project had private dependencies, my CI script built the application in separate stages. One for the server and one to generate static files. This pattern worked well for creating a final Docker image. However; when trying to add E2E tests as part of the flow, I rapidly ran into issues. The articles I came across in my search were either outdated or provided incomplete solutions. In case you run into similar issues, here's what I did to get this running. ## 1 - Create a New Dockerfile for Cypress Cypress does have an official Docker image that comes with all of the dependencies to run Cypress, except Cypress itself. As I didn't want to create a `package.json`, nor did I want to install a bunch of dependencies that I wouldn't need for this step, I opted to take care of it inside of the Dockerfile. ```docker FROM cypress/base:10 WORKDIR /app # This allows for creating a package.json within the Dockerfile itself RUN npm init --yes RUN npm i cypress # Copying both the test files and the config for cypress COPY cypress cypress COPY cypress.json . # I had a local copy of this script, but it is easily findable online COPY wait-for-it.sh RUN chmod +x wait-for-it.sh ``` ## 2 - Create a Docker Compose File Now that I had a Dockerfile to run the Cypress tests, the next step was to get it to run against my application. I already had an existing Dockerfile to assemble my application, so all I needed to get it running was Docker Compose. ```yaml version: "3" services: web: build: context: . dockerfile: "docker/Dockerfile.web" ports: - "8080:8080" cypress: build: context: . dockerfile: "docker/Dockerfile.cypress" depends_on: - web environment: - CYPRESS_BASEURL=http://web:8080 command: [ "./wait-for-it.sh", "web:8080", "-s", "-t", "0", "--", "npx", "cypress", "run" ] ``` Since I had added the `wait-for-it.sh` script to my Dockerfile, I was able to ensure that my application was running and accessible before attempting to run any tests against it. ## 3 - Update .gitlab-ci.yml to Run E2E Tests The final step was to get Gitlab to run the tests. I've added a simplified version of my `.gitlab-ci.yml` file below. The key things to get this working were: + Setting the entrypoint as an empty string. + Most of the examples I came across for running Cypress involved setting a custom entrypoint. However; Gitlab will continue to run `sh` as the entrypoint. + After a lot of digging, I found the solution here: [Gitlab Runner Issue #2692](https://gitlab.com/gitlab-org/gitlab-runner/issues/2692) + Having docker-compose exit when the Cypress container shut down. + This is needed in order to prevent the build from hanging as well as fail the build if the tests fail. ```yaml image: docker:stable-git services: - docker:dind variables: ARTIFACTS_DIR: $CI_PROJECT_DIR/artifacts GO_LINK_OPTIONS: -s cache: paths: - node_modules/ stages: - build-test - e2e-test before_script: - set -e - if [ "$CI_JOB_NAME" = "build-test:go" ]; then apk update && apk add --no-cache build-base git curl bash docker; fi - mkdir -p $ARTIFACTS_DIR build-test:node: image: node:12 stage: build-test artifacts: expires_in: 1 day paths: - $ARTIFACTS_DIR/ script: - npm ci - npm test - npm run generate - cp -r .nuxt $ARTIFACTS_DIR/.nuxt - cp -r dist $ARTIFACTS_DIR/dist build-test:go image: golang:1.12-alpine stage: build-test artifacts: expires_in: 1 day paths: - $ARTIFACTS_DIR/ script: - cd cmd - go test ./... - go build -ldflags=all=$GO_LINK_OPTIONS -o $ARTIFACTS_DIR/$CI_PROJECT_NAME . test:integration: image: name: docker/compose:1.24.0 # needed in order to be able to run custom scripts entrypoint: [""] stage: e2e-test dependencies: - build-test:node - build-test:go artifacts: expires_in: 1 day paths: - $ARTIFACTS_DIR/ script: # Copy buld artifacts - cp -r $ARTIFACTS_DIR/dist ./dist - cp -r $ARTIFACTS_DIR/.nuxt ./.nuxt - cp -r $ARTIFACTS_DIR/$CI_PROJECT_NAME ./cmd/$CI_PROJECT_NAME - docker-compose build # Without this the build will hang and will not fail on failing tests - docker-compose up --abort-on-container-exit --exit-code-from cypress ``` I hope this helps you resolve similar issues you may be facing getting Cypress to work in your CI scripts.

Building a SEO Friendly Blog: Using Markdown and Front Matter

In this series' previous installment, I covered the basics of creating a static generated site with Nuxt. In today's installment, I'll show you how to use Markdown and Front Matter to develop rich content for your blog. Before we write any code, the first thing we'll need to do is install a [webpack](https://webpack.js.org/) loader for markdown. In your terminal, while in the project directory, run `npm i frontmatter-markdown-loader` to add this dependency to your `package.json` file. Next, we need to configure Nuxt to know how to load markdown files. In the root directory, create a file named `nuxt.config.js` and add the following code to extend webpack. ```javascript export default { target: 'static', build: { extend (config, ctx) { config.module.rules.push({ test: /\.md$/, use: [{ loader: 'frontmatter-markdown-loader' }] }) } } } ``` Now that we've configured our project to work with markdown, we can create our first article. Directly under the root directory, create a directory named `articles`. Inside of this directory, create a markdown file named `dogs.md`. ```markdown --- slug: dogs title: 'Dogs are the Best Animal - Fight Me' date: '2020-09-25' tags: dogs,doggo,pupper,floofer,woofters description: "Whether you call them dogs, doggos, puppers, floofers, or woofters, they are the best animal. I am willing to fight you over this if you say I'm wrong." --- # Dogs are the Best Animal - Fight Me Whether you call them dogs, doggos, puppers, floofers, or woofters, they are the best animal. I am willing to fight you over this if you say I'm wrong. All doggos are a heckin' good fren. Anyone who disagrees is a monster. ``` You'll notice there's a bit of YAML code at the beginning of our markdown file. This is called [front matter](https://assemble.io/docs/YAML-front-matter.html), and it's a way to include metadata with your markdown content. The next step is to make our slug pull in the correct markdown file based on the URL parameter. While I would generally recommend not dynamically requiring any dependencies in your JavaScript code, we can safely do that here with our markdown file. We know this code will run at the time of generation, so if there are any missing files, we will catch them during the build process. Inside of `slug.vue` update the `asyncData` function to be the following. ```javascript asyncData ({route}) { const { params: { slug } } = route const markdown = require(`~/articles/${slug}.md`) return { markdown } } ``` This function grabs the URL's parameter, requires the matching markdown file, and makes the markdown file's results available to the template. The result is an object with two keys, `html` and `attributes`. The `html` key is what it sounds like; it is the HTML representation of your markdown file. The `attributes` key is an object of key/value pairs of all of the metadata you included with front matter. The last part of wiring this up is to add it to the template. We'll be using `v-html` to do this, but this is a feature of Nuxt that you want to use with caution. This function will insert any HTML it's given into the page, which can leave you vulnerable to an XSS attack. Never use `v-html` to render user-generated content. This feature should only ever be used with trusted content. To render the contents of the article, update the template to the following code: ```html ``` If you issue the command `npm run dev` and navigate to `http://localhost/blog/dogs`, you should see your article's content. Now let's update our landing page to dynamically generate a list of articles based on our markdown files. Webpack allows for requiring files via context, meaning you can require all files that match a certain pattern. This is extremely useful for our case as we can get data about every article we've written without manually requiring each file individually. In your `index.vue` file, add the following to the bottom of your file. ```html ``` This script is loading all of the articles, putting them into an array, and making them available to the page's template. As we included a `slug` parameter in the article's metadata, we can use that to output an article list on the landing page. Here's the new code for our template. ```html ``` If you navigate to `http://localhost:3000` you should now see a single link to the one article we've written. If you were to add another article to the articles directory, it would show up here too. That wrap's it up for this part! In the next and final part of this series, we'll go into how to use the attributes we set in front matter to give search engines more context about our article.

Building a SEO Friendly Blog: Getting Started with Nuxt

Throughout my career, I have worked with several firms that specialize in Search Engine Optimization. After you've worked with a few, you realize that much of SEO is a moving target, with no one being quite sure what is the best practice on all fronts. Company A will focus on each product page or post's paths, and when you switch to Company B, they'll have you change the routes that you just set due to the previous company's suggestion. How search engines index websites will always be in flux, but there are specific strategies that you can employ to get higher rankings and ensure those that would be interested in your content are more likely to find it. While we know that web crawlers will now execute a page's JavaScript, what is unclear how long the robot will wait for your content to finish loading. What we do know is that sites that load faster tend to perform better, meaning not having to wait for an expensive API call to render the main content will boost your site's performance. While you may decide to use AJAX to retrieve comments on your post to ensure you display the latest content, there is typically no reason the blog post itself cannot be rendered server-side or delivered as static content. Using a framework like [Nuxt](https://nuxtjs.org/) makes creating a statically generated site straight forward. For this tutorial, you will need to have Node version `8.9.0` or later installed to follow along. This tutorial does assume you understand the basics of working with `npm`. If you are unfamiliar with `npm`, I recommend reading this article from Nodesource: [An Absolute Beginner's Guide to Using npm](https://nodesource.com/blog/an-absolute-beginners-guide-to-using-npm). In the directory where you're going to build your project, first, create a `package.json` file using the command `npm init`. This command will start a wizard in your terminal that will let you specify your project's base configuration. After going through the wizard, you will have a file that looks similar to this: ```json { "name": "sample-blog", "version": "1.0.0", "description": "A sample blog using Nuxt", "main": "index.js", "scripts": { "test": "echo \"Error: no test specified\" && exit 1" }, "author": "Jessie Barnett", "license": "MIT", "dependencies": { "nuxt": "^2.14.5" } } ``` The next step is to install Nuxt by running the command `npm install nuxt`. This command will add it to your `package.json` and create a `package-lock.json` file. With Nuxt installed, we need to add two scripts to the `package.json` file to support running the application in development mode and generating a static site. ```json { "name": "sample-blog", "version": "1.0.0", "description": "A sample blog using Nuxt", "main": "index.js", "scripts": { "test": "echo \"Error: no test specified\" && exit 1", "dev": "nuxt", "generate": "nuxt generate" }, "author": "Jessie Barnett", "license": "MIT", "dependencies": { "nuxt": "^2.14.5" } } ``` Nuxt is an opinionated framework that utilizes pre-determined directory structures to figure out how to stitch together your website. To create a page for your website, you will need to create a `pages` directory using `mkdir pages`. Inside of this directory, you will need to create an `index.vue` file, which will be the page served up by going to your website's `/` route. We'll create a basic template for our landing page for now. Update your `index.vue` file to have the following code: ```html ``` If you run the command `npm run dev` and navigate to `http://localhost:3000` you can see your rendered template. Now let's add a dynamic route for our blog posts. We will be creating a slug, which is noted by the filename starting with an underscore. Let's make a `blog` directory in the `pages` directory and add a `_slug.vue` file to it. To see the dynamic page in action, we will use the `asyncData` hook provided by Nuxt. This hook takes in a context object, which has quite a few useful attributes, but we will focus on the `route`. We can get the path parameter from `context.route.params.slug`, where `slug` represents what we called the file. If we had called the file `_post.vue` instead, we'd need to address it via `context.route.params.post`. Our hook will need to return an object with this parameter to render it in the template. Update your `_slug.vue` file to have the following code: ```html ``` Assuming your dev server is still running, if you navigate to `http://localhost:3000/blog/my-post` you should see that the page prints out the URI's dynamic parameter. The last thing we need to address is static site generation. Nuxt uses a crawler to determine what dynamic routes to generate, meaning we either need to manually config Nuxt to know about all of our routes or add links to them. For today, we will go with adding links. Open up your `index.vue` file and update it to have the following code: ```html ``` You're now ready to generate a static site using the command `npm run generate`. The output of this command will be a `dist` folder, in which you should see an `index.html` for your landing page and all three "blog posts" we linked to from the landing page. That's it for this part of the series! In the next part we'll discuss how you can use a combination of Markdown, Frontmatter, and a context loader to import articles and create an article list.

Building a SEO Friendly Blog: Adding Schemas and Open Graph Protocol

In this series' previous installment, I covered how to use markdown and front matter to create content-rich articles. In the final part of this series, I will show you how to help search engines understand your content by using schemas and Open Graph protocol. As there is an infinite amount of types of content and many types of content may look similar, it can be challenging for search engines to understand what information your website is trying to convey. One way to help with this is to document the schema of your page's content. [Schema.org](https://schema.org/) demonstrates how to define structured data on your website, whether it be an [article](https://schema.org/Article), [recipe](https://schema.org/Recipe) or another type of content. Standard schema formats include RDFa, Microdata, and JSON-LD. We'll focus on JSON-LD as I find it's one of the quickest and easiest to understand ways to define a schema. If you remember from the last installment, we defined quite a bit of metadata in our article's front matter. ```markdown --- slug: dogs title: 'Dogs are the Best Animal - Fight Me' date: '2020-09-25' tags: dogs,doggo,pupper,floofer,woofters description: "Whether you call them dogs, doggos, puppers, floofers, or woofters, they are the best animal. I am willing to fight you over this if you say I'm wrong." --- # Dogs are the Best Animal - Fight Me Whether you call them dogs, doggos, puppers, floofers, or woofters, they are the best animal. I am willing to fight you over this if you say I'm wrong. All doggos are a heckin' good fren. Anyone who disagrees is a monster. ``` We can use this metadata to populate our JSON-LD for an article. To start, we'll first need to install a plugin for Nuxt, [nuxt-jsonld](https://github.com/ymmooot/nuxt-jsonld). Inside of your project's directory, run the command `npm i nuxt-jsonld`. We need to add a `jsonld` function to our `_slug.vue` page to use this plugin. ```javascript jsonld () { return { '@context': 'http://schema.org', '@type': 'Article', author: 'Jessie Barnett', headline: this.markdown.attributes.title, tags: this.markdown.attributes.tags, wordcount: this.markdown.html.split(' ').length, datePublished: this.markdown.attributes.date, description: this.markdown.attributes.description, proficiencyLevel: this.markdown.attributes.proficiencyLevel, dependencies: this.markdown.attributes.dependencies } } ``` Since we defined the `markdown` object in `asyncData`, it is available in the template and other functions with the `this` scope. This means we can use our front matter metadata to describe our article with JSON-LD. When you run the `generate` command, it will add a script of type `application/ld+json` to the HTML document's head tag with your JSON-LD that you created in the function. Now that we've set up JSON-LD, let's move on to the Open Graph protocol. [Open Graph protocol](https://ogp.me/) is primarily used by social media networks. Using Open Graph protocol can allow links to your article to be more easily found on social media and help those sites create more descriptive previews of your article by specifying a description, image, and more. To add Open Graph protocol metatags to your article's page, we can use the `head` function provided by Nuxt to specify the page-specific configuration. ```javascript head () { return { title: `${this.markdown.attributes.title}`, meta: [ { hid: 'description', name: 'description', content: this.markdown.attributes.description }, { property: 'og:title', content: `${this.markdown.attributes.title}` }, { property: 'og:url', content: `https://your-domain.com${this.$route.fullPath}` }, { property: 'og:description', content: this.markdown.attributes.description }, { property: 'og:type', content: 'article' }, { property: 'article:author', content: 'https://your-domain.com' }, { property: 'article:publisher', content: 'https://your-domain.com' }, { property: 'article:published_time', content: this.markdown.attributes.date }, { property: 'article:tag', content: this.markdown.attributes.tags } ] } } ``` Now that we've added JSON-LD and Open Graph protocol to our static site, our articles will be easy to index by search engines and find on social media sites. Now that you know how to make an SEO friendly blog, all that's left is to use your design skills to make a great looking site! You can see the final version of the tutorial code on [GitHub](https://github.com/jessie-codes/nuxt-seo-example).

Generating a Sitemap for a JAMstack Site

Sitemaps are a great way of helping search engines understand your website. While modern search engines primarily rely on links to create their search indexes, this can leave lesser-known sites at a disadvantage due to a lack of external backlinks. Search engines will look for a `sitemap.xml` file at the root directory of your website, and if it exists, it will parse that file to find any links it did not previously index. When generating a sitemap, you should be following the schema defined on [http://sitemaps.org](http://sitemaps.org). ```xml https://www.example.org/path ``` Many of the JAMstack frameworks have plugins that can generate a sitemap for you. Still, if the framework you chose doesn't have one, or if you'd prefer to minimize dependencies, it's easy to create a sitemap after running your workflow's site generation step. I'll use Nuxt in this example, but the code can be easily modified to fit your stack. Nuxt creates a `dist` folder on static site generation, with `.html` files for each route within the website. Knowing this, I can write a script that will search that directory for HTML files and generate XML nodes for each unique page. ```javascript const fs = require('fs') const { resolve } = require('path') const glob = require('glob') const sitemap = resolve(__dirname, 'dist', 'sitemap.xml') const nodes = [ '' ] glob('dist/**/index.html', (err, files) => { if (err) { throw err } files.forEach((f) => { nodes.push(`https://jessie.codes${f.replace('dist', '').replace('index.html', '')}`) }) nodes.push('') fs.writeFile(sitemap, nodes.join(''), (err) => { if (err) { throw err } }) }) ``` Since I built my site using JavaScript, I can use `npm`'s ability to create a `post` hook that will run after the `generate` script. Every time a new page is added, the sitemap will automatically be updated, allowing search engines to update their index for my website even if no external backlinks exist. ```json { "scripts": { "generate": "nuxt generate", "postgenerate": "node ./sitemap.js" } } ```