A Pattern for Gulp

Tooling often gets pretty bloated as the application evolves and can easily grow into a tangled mess. As requirements change new tasks are patched in; making it harder than it should for newcomers to decipher the application's tooling workbelt. Today, a lot of teams are moving to a service oriented architecture, which means teams end up owning a number of different applications, APIs, libraries and the like.

This is when things can get pretty confusing; each application probably has different workflows. Each one usually flavoured with the particular developer's style. gulp help just spits out a long list of gulp tasks - quite useless. Context-switching has a new speed bump.

As a developer who likes consistency, or possibly just has mild OCD and a bad memory, I wanted to add some structure to my tooling across all my projects. This way, I know what tools I've got in my belt when I go back to a project I haven't touched in a while or consistently switch between a few.

To accomplish this I wrote a simple, little gulp plugin; gulp-pattern.

The idea is to:

  1. Add structure to the task files, so you can expect to find the same structure across your repos, or those of your team.
  2. Keep gulpfile as thin as possible, to keep a tight rein on the bloating.
  3. Define a few conventions for writing workflows for front-end applications and libraries.

The requirements are almost always the same, regardless of which technologies are used, or the type of application or library. Develop, test, vet, release.

The Pattern

Favouring Ruby's convention over configuration style, most of this pattern's rules are simply conventions to follow. This tool just facilitates these conventions.

Directory Structure

All the files which will be used by gulp are organised in a gulp/ folder, in the root of the application's directory. gulpfile.js will then be told to look through this folder and look for any tasks and config (what little there is) required for the application.

The directory structure should look like this:

gulp/
    tasks/
    workflows/
    config/

The tasks/ directory is for individual gulp tasks. Tasks should be as coherent as possible, with each one attempting to do one thing only. If you want to do more than one thing, then you should think about using a workflow.

Naming convention: [npm or task name].tsk.js

The workflows/ directory is for specific workflows, which calls a group of the individual tasks, related to that workflow. A workflow should be thought of as a group of tasks, and should be representative of a particular action run against your application.

Naming convention: [workflow name].wflow.js

The config/, is for general config files for the app in general: app.conf.js, and other custom config files for specific tasks, like; jshint, stylus.

Naming convention: app.conf.js

This config file should ideally be very thin, to limit the amount of config you'd need to remember when writing out your tasks and flows.

Using the Pattern

Now that you've got the directory structure, you can use gulp-pattern in your tooling process.

Check the README for the details on how.

This plugin also adds a custom task: gulp list which outputs your application's tasks and workflows in the respective categories, also listing all the dependencies for a given workflow.

You can also list single categories by passing in arguments:

gulp list
// lists tasks and workflows

gulp list -t
// lists only the tasks

gulp list -w
// lists only the workflows

Running gulp list -w will now show something like this (given you write the comments in a similar way):

Workflow Conventions

As mentioned prior, more often than not the tooling needs of all applications consists of four main categories:

  1. building the code for development
  2. running tests
  3. vetting the code for quality standards
  4. building the code for release / production

So why not just define these as the main workflows of every application? To make things as easy as possible to remember and be application agnostic, I've defined these four areas as the main workflows in my tooling: build, test, vet, release.

To allow for more flexibility without adding numerous other workflows, making naming harder, I use arguments to define different variations of the workflow, similar to gulp list (-t / -w).

1. gulp build

This workflow refers to building the src code for development only. I think there needs to be a distinction between building for dev and building for production, so I've reflected this with two workflows. The flow variation should be applied at this level.

At the root, this workflow should build all the src code into the development application, whatever structure that may be. Adding args then granulates the target of the flow. Any additional build flows needed, unique to that particular application, like for example wiredep can be added here.

-css build only the application's styles

-js build only the application's javascript

-a / --auto build and add a watch on the src files

2. gulp test

Pretty self-explanatory, this workflow runs all the tests. At the root, it should target all tests.

-u / --unit run only the unit tests

-i / --integration run only the integration tests

-a / --auto run tests and add a watch on the src files

3. gulp vet

This workflow is reponsible for making sure the application's code adheres to the quality standards and conventions defined for it. Anything related to this should be added here.

-s / --src vet only the src code

-d / --dist vet only the distributable code

-jshint lint the javascript code

-a / --auto vet and add a watch on the src files

4. gulp release

This flow deals with building your application for release.

Task Conventions

Tasks should be written as small units of work, which plug into specific workflows. They manage their own dependencies, allowing different workflows to use the same tasks. If you have variations of a particular task - then you can do this in the following way:

gulp jshint:src
gulp jshint:dist

What Do I Want to Do?

Now, adding your gulp steps in your CI tool, or git hooks, is just asking yourself what do I want to do?

I want to run all tests, vet the src, build the code for release, vet the dist code. You can write your steps without even reading any of the tooling code.

1. gulp test
2. gulp vet --src
3. gulp release
4. gulp vet --dist

Any changes to your tooling - like adding new tasks or new workflows, wouldn't affect these steps at all. For example, if down the line you start vetting with plato, and building your release code with webpack so you don't have to wait more to use ES6, then you're still vetting and releasing. The flows are still relevant. Any automation / githooks using these steps don't need to be changed.

I prefer not creating workflows with direct references to this layer. For example gulp build:ci. There isn't a need for this 'coupling', as it were.

So..

That's pretty much it. To automate as much as possible, I've added these conventions as shells in a public repo scaffolds. Follow the steps in the README, and then you can use them as boilerplate for you own tooling.

Johnny Copperstone
Johnny Copperstone

Senior Developer at Marketing Cloud, Salesforce.