~petersanchez/migrate

Simple database migrations for your Go project.

#2 When migrating DOWN, we should stop BEFORE running STOP ID Migration

~petersanchez filed ticket on migrate todo

3 months ago

#1 Add basic command support

~petersanchez commented on migrate todo

3 months ago

#migrate godoc

Database migrations for your Go project. Supports files, functions, and query migrations.

Version: 0.2.0

Project Links: Docs - Issues - Mailing List - Contributing

Author: Peter Sanchez (https://petersanchez.com)

#Requirements

  • Go 1.16+ (probably works with older, I just didn't test it)

#What?

This is a simple tool to manage database migrations from within your Go project. It's not a CLI tool, but a module that you plugin to your own project as needed.

It's based on github.com/joncalhoun/migrate but I've done many changes to make it fit my needs. That is why I am just maintaining my own fork versus submitting a PR. I feel mine has diverged too far from what Jon originally wanted.

Here's some of the changes:

  • Removed sqlx as dependency.
  • Added support for a global context.Context within migrations.
  • Added support for specific timeouts (via context.Context) per migration.
  • Added ability to specify a up (migrate) or down (rollback) specific version.
  • Added ability to "fake" a migration (up only). Essentially just adding the entry to migration history without actually running it. Useful for edge cases.
  • Added checks for migration ID duplication.
  • Added check to ensure migration ID exists when a specific ID is given. This is to avoid running migrations when an entry error has occurred.

My fork is a copy of the original repo as of this commit. I probably should have done a repo migration to Mercurial but here we are. All code up to the commit above is licensed under the MIT license, see the PREVIOUS_LICENSES document.

#Example usage

This example will assume you're using PostgreSQL as well (note the changing of binding type to migrate.DOLLAR).

Let's say you have a simple directory structure like so:

.
├── accounts
│   ├── input.go
│   ├── models.go
│   ├── routes.go
├── cmd
│   └── webapp
│       └── main.go
├── go.mod
├── go.sum
├── webapp_binary
├── Makefile
├── migrations
│   ├── 0001_initial_rollback.sql
│   └── 0001_initial.sql
├── templates
    ├── login.html
    └── register.html

You can see there is a migrations directory. You can store your migration files anywhere you want, but in this example we'll store them there.

Now let's add migrations to the main binary. We'll edit cmd/webapp/main.go to always call migrations when starting. I'm not recommending this, just a simple example.

import (
	...
	"petersanchez.com/migrate",
	...
)

func main() {
	// `db` is your database connection (*sql.DB)

	// Custom functions should be watching for the context's Done() channel
	migrations := []migrate.Migration{
		migrate.FileMigration("0001_initial", "migrations/0001_initial.sql", "migrations/0001_initial_rollback.sql", 0),
		migrate.Migration{
			ID: "0002_function",
			Migrate: func(ctx context.Context, tx *sql.Tx) error {
				// Run your migrate code here.
				return nil
			},
			Rollback: func(ctx context.Context, tx *sql.Tx) error {
				// Run your rollback code here.
				return nil
			},
			Timeout: 40, // Set a 40 second timeout on the functions
		},
	}

	if len(os.Args) > 0 {
		if os.Args[1] != "migrate" {
			err := migrate.Command(db, migrations, migrate.DOLLAR)
			// handle err
			os.Exit(0)
		}
	}

	// rest of your main() code
}

#Why?

I found the larger database migration projects for Go to be overkill for my needs. Especially if you wanted to run custom function migrations.

During my research I stumbled on an article titled Database migrations in Go by Jon Calhoun. I had already been thinking of how to create my own migrations package as minimally as possible and this article was close enough to what I was thinking. John even created the original migrate package, but it lacked several things I wanted in my migration solution.

So I stripped out external dependencies, added a bunch of stuff I wanted, and a few development hours later, here we are.

#Status

My needs are fairly simple and the current state of the module basically covers them. Please submit bugs and/or patches if you find any issues.

#Contributing

We accept patches submitted via hg email which is the patchbomb extension included with Mercurial.

The mailing list where you submit your patches is ~petersanchez/public-inbox@lists.code.netlandish.com. You can also view the archives on the web here:

https://lists.code.netlandish.com/~petersanchez/public-inbox

To quickly setup your clone of migrate to submit to the mailing list just edit your .hg/hgrc file and add the following:

[email]
to = ~petersanchez/public-inbox@lists.code.netlandish.com

[patchbomb]
flagtemplate = {separate(' ', 'migrate', flags)}

[diff]
git = 1

We have more information on the topic here:

All documentation, libraries, and sample code are Copyright 2022 Peter Sanchez <pjs@petersanchez.com>. The library and sample code are made available to you under the terms of the BSD license which is contained in the included file, LICENSE.