snapshot of https://github.com/ent/ent.git (master, 2026-09-15; blobs > 131072B omitted)
This commit is contained in:
58
doc/md/versioned/01-intro.md
Normal file
58
doc/md/versioned/01-intro.md
Normal file
@@ -0,0 +1,58 @@
|
||||
---
|
||||
id: intro
|
||||
title: Introduction
|
||||
---
|
||||
## Schema Migration Flows
|
||||
|
||||
Ent supports two different workflows for managing schema changes:
|
||||
* Automatic Migrations - a declarative style of schema migrations which happen entirely at runtime.
|
||||
With this flow, Ent calculates the difference between the connected database and the database
|
||||
schema needed to satisfy the `ent.Schema` definitions, and then applies the changes to the database.
|
||||
* Versioned Migrations - a workflow where schema migrations are written as SQL files ahead of time
|
||||
and then are applied to the database by a specialized tool such as [Atlas](https://atlasgo.io) or
|
||||
[golang-migrate](https://github.com/golang-migrate/migrate).
|
||||
|
||||
Many users start with the automatic migration flow as it is the easiest to get started with, but
|
||||
as their project grows, they may find that they need more control over the migration process, and
|
||||
they switch to the versioned migration flow.
|
||||
|
||||
This tutorial will walk you through the process of upgrading an existing project from automatic migrations
|
||||
to versioned migrations.
|
||||
|
||||
## Supporting repository
|
||||
|
||||
All of the steps demonstrated in this tutorial can be found in the
|
||||
[rotemtam/ent-versioned-migrations-demo](https://github.com/rotemtam/ent-versioned-migrations-demo)
|
||||
repository on GitHub. In each section we will link to the relevant commit in the repository.
|
||||
|
||||
The initial Ent project which we will be upgrading can be found
|
||||
[here](https://github.com/rotemtam/ent-versioned-migrations-demo/tree/start).
|
||||
|
||||
## Automatic Migration
|
||||
|
||||
In this tutorial, we assume you have an existing Ent project and that you are using automatic migrations.
|
||||
Many simpler projects have a block of code similar to this in their `main.go` file:
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
func main() {
|
||||
// Connect to the database (MySQL for example).
|
||||
client, err := ent.Open("mysql", "root:pass@tcp(localhost:3306)/test")
|
||||
if err != nil {
|
||||
log.Fatalf("failed connecting to mysql: %v", err)
|
||||
}
|
||||
defer client.Close()
|
||||
ctx := context.Background()
|
||||
// Run migration.
|
||||
// highlight-next-line
|
||||
if err := client.Schema.Create(ctx); err != nil {
|
||||
log.Fatalf("failed creating schema resources: %v", err)
|
||||
}
|
||||
// ... Continue with server start.
|
||||
}
|
||||
```
|
||||
|
||||
This code connects to the database, and then runs the automatic migration tool to create all schema resources.
|
||||
|
||||
Next, let's see how to set up our project for versioned migrations.
|
||||
43
doc/md/versioned/02-auto-plan.mdx
Normal file
43
doc/md/versioned/02-auto-plan.mdx
Normal file
@@ -0,0 +1,43 @@
|
||||
---
|
||||
title: Automatic migration planning
|
||||
id: auto-plan
|
||||
---
|
||||
|
||||
import InstallationInstructions from '../components/_installation_instructions.mdx';
|
||||
import AtlasMigrateDiff from '../components/_atlas_migrate_diff.mdx';
|
||||
|
||||
## Automatic migration planning
|
||||
|
||||
One of the convenient features of Automatic Migrations is that developers do not
|
||||
need to write the SQL statements to create or modify the database schema. To
|
||||
achieve similar benefits, we will now add a script to our project that will
|
||||
automatically plan migration files for us based on the changes to our schema.
|
||||
|
||||
To do this, Ent uses [Atlas](https://atlasgo.io), an open-source tool for managing database
|
||||
schemas, created by the same people behind Ent.
|
||||
|
||||
If you have been following our example repo, we have been using SQLite as our database
|
||||
until this point. To demonstrate a more realistic use case, we will now switch to MySQL.
|
||||
See this change in [PR #3](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/3/files).
|
||||
|
||||
## Using the Atlas CLI to plan migrations
|
||||
|
||||
In this section, we will demonstrate how to use the Atlas CLI to automatically plan
|
||||
schema migrations for us. In the past, users had to create a custom Go program to
|
||||
do this (as described [here](07-programmatically.mdx)). With recent versions of Atlas,
|
||||
this is no longer necessary: Atlas can natively load the desired database schema from an Ent schema.
|
||||
|
||||
<InstallationInstructions />
|
||||
|
||||
Then, run the following command to automatically generate migration files for your Ent schema:
|
||||
|
||||
<AtlasMigrateDiff/>
|
||||
|
||||
:::info The role of the [dev database](https://atlasgo.io/concepts/dev-database)
|
||||
Atlas loads the **current state** by executing the SQL files stored in the migration directory onto the provided
|
||||
[dev database](https://atlasgo.io/concepts/dev-database). It then compares this state against the **desired state**
|
||||
defined by the `ent/schema` package and writes a migration plan for moving from the current state to the desired state.
|
||||
:::
|
||||
|
||||
|
||||
Next, let's see how to upgrade an existing production database to be managed with versioned migrations.
|
||||
78
doc/md/versioned/03-upgrade-prod.mdx
Normal file
78
doc/md/versioned/03-upgrade-prod.mdx
Normal file
@@ -0,0 +1,78 @@
|
||||
---
|
||||
id: upgrade-prod
|
||||
title: Upgrading Production
|
||||
---
|
||||
|
||||
import Tabs from '@theme/Tabs';
|
||||
import TabItem from '@theme/TabItem';
|
||||
|
||||
:::info Supporting repository
|
||||
|
||||
The change described in this section can be found in
|
||||
[PR #5](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/5/files)
|
||||
in the supporting repository.
|
||||
|
||||
:::
|
||||
|
||||
## Upgrading our production database to use versioned migrations
|
||||
|
||||
If you have been following our tutorial to this point, you may be asking yourself how do we
|
||||
upgrade the production instance of our database to be managed by the versioned migraions workflow?
|
||||
With local development, we can just delete the database and start over, but that is not an option
|
||||
for production for obvious reasons.
|
||||
|
||||
Like many other database schema management tools, [Atlas](https://atlasgo.io) uses a metadata table
|
||||
on the target database to keep track of which migrations were already applied.
|
||||
In the case where we start using Atlas on an existing database, we must somehow
|
||||
inform Atlas that all migrations up to a certain version were already applied.
|
||||
|
||||
To illustrate this, let's try to run Atlas's `migrate apply` command on a database
|
||||
that is currently managed by an auto-migration workflow using the migration directory that we just
|
||||
created. Notice that we use a connection string to a database that _already has_ the application schema
|
||||
instantiated (we use the `/db` suffix to indicate that we want to connect to the `db` database).
|
||||
|
||||
```text
|
||||
atlas migrate apply --dir file://ent/migrate/migrations --url mysql://root:pass@localhost:3306/db
|
||||
```
|
||||
|
||||
Atlas returns an error:
|
||||
|
||||
```text
|
||||
Error: sql/migrate: connected database is not clean: found table "atlas_schema_revisions" in schema "db". baseline version or allow-dirty is required
|
||||
```
|
||||
|
||||
This error is expected, as this is the first time we are running Atlas on this database, but as the error said
|
||||
we need to "baseline" the database. This means that we tell Atlas that the database is already at a certain state
|
||||
that correlates with one of the versions in the migration directory.
|
||||
|
||||
To fix this, we use the `--baseline` flag to tell Atlas that the database is already at
|
||||
a certain version:
|
||||
|
||||
```text
|
||||
atlas migrate apply --dir file://ent/migrate/migrations --url mysql://root:pass@localhost:3306/db --baseline 20221114165732
|
||||
```
|
||||
|
||||
Atlas reports that there's nothing new to run:
|
||||
|
||||
```text
|
||||
No migration files to execute
|
||||
```
|
||||
|
||||
That's better! Next, let's verify that Atlas is aware of what migrations
|
||||
were already applied by using the `migrate status` command:
|
||||
|
||||
```text
|
||||
atlas migrate status --dir file://ent/migrate/migrations --url mysql://root:pass@localhost:3306/db
|
||||
```
|
||||
Atlas reports:
|
||||
```text
|
||||
Migration Status: OK
|
||||
-- Current Version: 20221114165732
|
||||
-- Next Version: Already at latest version
|
||||
-- Executed Files: 1
|
||||
-- Pending Files: 0
|
||||
```
|
||||
Great! We have successfully upgraded our project to use versioned migrations with Atlas.
|
||||
|
||||
Next, let's see how we add a new migration to our project when we make a change to our
|
||||
Ent schema.
|
||||
129
doc/md/versioned/04-new-migration.mdx
Normal file
129
doc/md/versioned/04-new-migration.mdx
Normal file
@@ -0,0 +1,129 @@
|
||||
---
|
||||
title: Planning a Migration
|
||||
id: new-migration
|
||||
---
|
||||
|
||||
import Tabs from '@theme/Tabs';
|
||||
import TabItem from '@theme/TabItem';
|
||||
|
||||
:::info Supporting repository
|
||||
|
||||
The change described in this section can be found in
|
||||
[PR #6](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/6/files)
|
||||
in the supporting repository.
|
||||
|
||||
:::
|
||||
|
||||
|
||||
## Planning a migration
|
||||
|
||||
In this section, we will discuss how to plan a new schema migration when we
|
||||
make a change to our project's Ent schema. Consider we want to add a new field
|
||||
to our `User` entity, adding a new optional field named `title`:
|
||||
|
||||
```go title=ent/schema/user.go
|
||||
// Fields of the User.
|
||||
func (User) Fields() []ent.Field {
|
||||
return []ent.Field{
|
||||
field.String("name"),
|
||||
field.String("email"). // <-- Our new field
|
||||
Unique(),
|
||||
// highlight-start
|
||||
field.String("title").
|
||||
Optional(),
|
||||
// highlight-end
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
After adding the new field, we need to rerun code-gen for our project:
|
||||
|
||||
```shell
|
||||
go generate ./...
|
||||
```
|
||||
|
||||
Next, we need to create a new migration file for our change using the Atlas CLI:
|
||||
|
||||
|
||||
<Tabs
|
||||
defaultValue="mysql"
|
||||
values={[
|
||||
{label: 'MySQL', value: 'mysql'},
|
||||
{label: 'MariaDB', value: 'maria'},
|
||||
{label: 'PostgreSQL', value: 'postgres'},
|
||||
{label: 'SQLite', value: 'sqlite'},
|
||||
]}>
|
||||
<TabItem value="mysql">
|
||||
|
||||
```shell
|
||||
atlas migrate diff add_user_title \
|
||||
--dir "file://ent/migrate/migrations" \
|
||||
--to "ent://ent/schema" \
|
||||
--dev-url "docker://mysql/8/ent"
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="maria">
|
||||
|
||||
```shell
|
||||
atlas migrate diff add_user_title \
|
||||
--dir "file://ent/migrate/migrations" \
|
||||
--to "ent://ent/schema" \
|
||||
--dev-url "docker://mariadb/latest/test"
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="postgres">
|
||||
|
||||
```shell
|
||||
atlas migrate diff add_user_title \
|
||||
--dir "file://ent/migrate/migrations" \
|
||||
--to "ent://ent/schema" \
|
||||
--dev-url "docker://postgres/15/test?search_path=public"
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="sqlite">
|
||||
|
||||
```shell
|
||||
atlas migrate diff add_user_title \
|
||||
--dir "file://ent/migrate/migrations" \
|
||||
--to "ent://ent/schema" \
|
||||
--dev-url "sqlite://file?mode=memory&_fk=1"
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
Observe a new file named `20221115101649_add_user_title.sql` was created under
|
||||
the `ent/migrate/migrations/` directory. This file contains the SQL statements
|
||||
to create the newly added `title` field in the `users` table:
|
||||
|
||||
```sql title=ent/migrate/migrations/20221115101649_add_user_title.sql
|
||||
-- modify "users" table
|
||||
ALTER TABLE `users` ADD COLUMN `title` varchar(255) NULL;
|
||||
```
|
||||
|
||||
Great! We've successfully used the Atlas CLI to automatically
|
||||
generate a new migration file for our change.
|
||||
|
||||
To apply the migration, we can run the following command:
|
||||
|
||||
```shell
|
||||
atlas migrate apply --dir file://ent/migrate/migrations --url mysql://root:pass@localhost:3306/db
|
||||
```
|
||||
Atlas reports:
|
||||
```shell
|
||||
Migrating to version 20221115101649 from 20221114165732 (1 migrations in total):
|
||||
|
||||
-- migrating version 20221115101649
|
||||
-> ALTER TABLE `users` ADD COLUMN `title` varchar(255) NULL;
|
||||
-- ok (36.152277ms)
|
||||
|
||||
-------------------------
|
||||
-- 44.1116ms
|
||||
-- 1 migrations
|
||||
-- 1 sql statements
|
||||
```
|
||||
|
||||
In the next section, we will discuss how to plan custom schema migrations for our project.
|
||||
91
doc/md/versioned/05-custom-migrations.md
Normal file
91
doc/md/versioned/05-custom-migrations.md
Normal file
@@ -0,0 +1,91 @@
|
||||
---
|
||||
title: Custom migrations
|
||||
id: custom-migrations
|
||||
---
|
||||
:::info Supporting repository
|
||||
|
||||
The change described in this section can be found in
|
||||
[PR #7](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/7/files)
|
||||
in the supporting repository.
|
||||
|
||||
:::
|
||||
|
||||
## Custom migrations
|
||||
In some cases, you may want to write custom migrations that are not automatically
|
||||
generated by Atlas. This can be useful in cases where you want to perform changes
|
||||
to your database that aren't currently supported by Ent, or if you want to seed
|
||||
the database with data.
|
||||
|
||||
In this section, we will learn how to add custom migrations to our project. For the
|
||||
purpose of this guide, let's assume we want to seed the users table with some data.
|
||||
|
||||
## Create a custom migration
|
||||
|
||||
Let's start by adding a new migration file to our project:
|
||||
|
||||
```shell
|
||||
atlas migrate new seed_users --dir file://ent/migrate/migrations
|
||||
```
|
||||
|
||||
Observe that a new file named `20221115102552_seed_users.sql` was created in the
|
||||
`ent/migrate/migrations` directory.
|
||||
|
||||
Continue by opening the file and adding the following SQL statements:
|
||||
|
||||
```sql
|
||||
INSERT INTO `users` (`name`, `email`, `title`)
|
||||
VALUES ('Jerry Seinfeld', 'jerry@seinfeld.io', 'Mr.'),
|
||||
('George Costanza', 'george@costanza.io', 'Mr.')
|
||||
```
|
||||
|
||||
## Recalculating the checksum file
|
||||
|
||||
Let's try to run our new custom migration:
|
||||
|
||||
```shell
|
||||
atlas migrate apply --dir file://ent/migrate/migrations --url mysql://root:pass@localhost:3306/db
|
||||
```
|
||||
Atlas fails with an error:
|
||||
```text
|
||||
You have a checksum error in your migration directory.
|
||||
This happens if you manually create or edit a migration file.
|
||||
Please check your migration files and run
|
||||
|
||||
'atlas migrate hash'
|
||||
|
||||
to re-hash the contents and resolve the error
|
||||
|
||||
Error: checksum mismatch
|
||||
```
|
||||
Atlas introduces the concept of [migration directory integrity](https://atlasgo.io/concepts/migration-directory-integrity)
|
||||
as a means to enforce a linear migration history. This way, if multiple developers work on the
|
||||
same project in parallel, they can be sure that their merged migration history is correct.
|
||||
|
||||
Let's re-hash the contents of our migration directory to resolve the error:
|
||||
|
||||
```shell
|
||||
atlas migrate hash --dir file://ent/migrate/migrations
|
||||
```
|
||||
|
||||
If we run `atlas migrate apply` again, we will see that the migration was successfully applied:
|
||||
```text
|
||||
atlas migrate apply --dir file://ent/migrate/migrations --url mysql://root:pass@localhost:3306/db
|
||||
```
|
||||
Atlas reports:
|
||||
```text
|
||||
Migrating to version 20221115102552 from 20221115101649 (1 migrations in total):
|
||||
|
||||
-- migrating version 20221115102552
|
||||
-> INSERT INTO `users` (`name`, `email`, `title`)
|
||||
VALUES ('Jerry Seinfeld', 'jerry@seinfeld.io', 'Mr.'),
|
||||
('George Costanza', 'george@costanza.io', 'Mr.')
|
||||
-- ok (9.077102ms)
|
||||
|
||||
-------------------------
|
||||
-- 19.857555ms
|
||||
-- 1 migrations
|
||||
-- 1 sql statements
|
||||
```
|
||||
|
||||
In the next section, we will learn how to automatically verify the safety of our
|
||||
schema migrations using Atlas's [Linting](https://atlasgo.io/versioned/lint) feature.
|
||||
264
doc/md/versioned/06-verifying-safety.mdx
Normal file
264
doc/md/versioned/06-verifying-safety.mdx
Normal file
@@ -0,0 +1,264 @@
|
||||
---
|
||||
title: Verifying migration safety
|
||||
id: verifying-safety
|
||||
---
|
||||
|
||||
import Tabs from '@theme/Tabs';
|
||||
import TabItem from '@theme/TabItem';
|
||||
|
||||
:::info Supporting repository
|
||||
|
||||
The change described in this section can be found in
|
||||
[PR #8](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/8/files)
|
||||
in the supporting repository.
|
||||
|
||||
:::
|
||||
|
||||
## Verifying migration safety
|
||||
|
||||
As the database is a critical component of our application, we want to make sure that when we
|
||||
make changes to it, we don't break anything. Ill-planned migrations can cause data loss, application
|
||||
downtime and other issues. Atlas provides a mechanism to verify that a migration is safe to run.
|
||||
This mechanism is called [migration linting](https://atlasgo.io/versioned/lint) and in this section
|
||||
we will show how to use it to verify that our migration is safe to run.
|
||||
|
||||
## Linting the migration directory
|
||||
|
||||
To lint our migration directory we can use the `atlas migrate lint` command.
|
||||
To demonstrate this, let's see what happens if we decide to change the `Title` field in the `User`
|
||||
model from optional to required:
|
||||
|
||||
```diff
|
||||
// Fields of the User.
|
||||
func (User) Fields() []ent.Field {
|
||||
return []ent.Field{
|
||||
field.String("name"),
|
||||
field.String("email").
|
||||
Unique(),
|
||||
-- field.String("title").
|
||||
-- Optional(),
|
||||
++ field.String("title"),
|
||||
}
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
Let's re-run codegen:
|
||||
|
||||
```shell
|
||||
go generate ./...
|
||||
```
|
||||
|
||||
Next, let's automatically generate a new migration:
|
||||
|
||||
|
||||
<Tabs
|
||||
defaultValue="mysql"
|
||||
values={[
|
||||
{label: 'MySQL', value: 'mysql'},
|
||||
{label: 'MariaDB', value: 'maria'},
|
||||
{label: 'PostgreSQL', value: 'postgres'},
|
||||
{label: 'SQLite', value: 'sqlite'},
|
||||
]}>
|
||||
<TabItem value="mysql">
|
||||
|
||||
```shell
|
||||
atlas migrate diff user_title_required \
|
||||
--dir "file://ent/migrate/migrations" \
|
||||
--to "ent://ent/schema" \
|
||||
--dev-url "docker://mysql/8/ent"
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="maria">
|
||||
|
||||
```shell
|
||||
atlas migrate diff user_title_required \
|
||||
--dir "file://ent/migrate/migrations" \
|
||||
--to "ent://ent/schema" \
|
||||
--dev-url "docker://mariadb/latest/test"
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="postgres">
|
||||
|
||||
```shell
|
||||
atlas migrate diff user_title_required \
|
||||
--dir "file://ent/migrate/migrations" \
|
||||
--to "ent://ent/schema" \
|
||||
--dev-url "docker://postgres/15/test?search_path=public"
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="sqlite">
|
||||
|
||||
```shell
|
||||
atlas migrate diff user_title_required \
|
||||
--dir "file://ent/migrate/migrations" \
|
||||
--to "ent://ent/schema" \
|
||||
--dev-url "sqlite://file?mode=memory&_fk=1"
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
A new migration file was created in the `ent/migrate/migrations` directory:
|
||||
|
||||
```sql title="ent/migrate/migrations/20221116051710_user_title_required.sql"
|
||||
-- modify "users" table
|
||||
ALTER TABLE `users` MODIFY COLUMN `title` varchar(255) NOT NULL;
|
||||
```
|
||||
|
||||
Now, let's lint the migration directory:
|
||||
|
||||
```shell
|
||||
atlas migrate lint --dev-url mysql://root:pass@localhost:3306/dev --dir file://ent/migrate/migrations --latest 1
|
||||
```
|
||||
|
||||
Atlas reports that the migration may be unsafe to run:
|
||||
|
||||
```text
|
||||
20221116051710_user_title_required.sql: data dependent changes detected:
|
||||
|
||||
L2: Modifying nullable column "title" to non-nullable might fail in case it contains NULL values
|
||||
```
|
||||
|
||||
Atlas detected that the migration is unsafe to run and prevented us from running it.
|
||||
In this case, Atlas classified this change as a data dependent change. This means that the change
|
||||
might fail, depending on the concrete data in the database.
|
||||
|
||||
Atlas can detect many more types of issues, for a full list, see the [Atlas documentation](https://atlasgo.io/lint/analyzers).
|
||||
|
||||
## Linting our migration directory in CI
|
||||
|
||||
In the previous section, we saw how to lint our migration directory locally. In this section,
|
||||
we will see how to lint our migration directory in CI. This way, we can make sure that our migration
|
||||
history is safe to run before we merge it to the main branch.
|
||||
|
||||
[GitHub Actions](https://github.com/features/actions) is a popular CI/CD
|
||||
product from GitHub. With GitHub Actions, users can easily define workflows
|
||||
that are triggered in various lifecycle events related to a Git repository.
|
||||
For example, many teams configure GitHub actions to run all unit tests in
|
||||
a repository on each change that is committed to a repository.
|
||||
|
||||
One of the powerful features of GitHub Actions is its extensibility: it is
|
||||
very easy to package a piece of functionality as a module (called an "action")
|
||||
that can later be reused by many projects.
|
||||
|
||||
Teams using GitHub that wish to ensure all changes to their database schema are safe
|
||||
can use the [`atlas-action`](https://github.com/ariga/atlas-action) GitHub Action.
|
||||
|
||||
This action is used for [linting migration directories](/versioned/lint)
|
||||
using the `atlas migrate lint` command. This command validates and analyzes the contents
|
||||
of migration directories and generates insights and diagnostics on the selected changes:
|
||||
|
||||
* Ensure the migration history can be replayed from any point in time.
|
||||
* Protect from unexpected history changes when concurrent migrations are written to the migration directory by
|
||||
multiple team members.
|
||||
* Detect whether destructive or irreversible changes have been made or whether they are dependent on tables'
|
||||
contents and can cause a migration failure.
|
||||
|
||||
## Usage
|
||||
|
||||
Add `.github/workflows/atlas-ci.yaml` to your repo with the following contents:
|
||||
|
||||
```yaml
|
||||
name: Atlas CI
|
||||
on:
|
||||
# Run whenever code is changed in the master branch,
|
||||
# change this to your root branch.
|
||||
push:
|
||||
branches:
|
||||
- master
|
||||
pull_request:
|
||||
paths:
|
||||
- 'ent/migrate/migrations/*'
|
||||
jobs:
|
||||
lint:
|
||||
services:
|
||||
# Spin up a mysql:8.0.29 container to be used as the dev-database for analysis.
|
||||
mysql:
|
||||
image: mysql:8.0.29
|
||||
env:
|
||||
MYSQL_ROOT_PASSWORD: pass
|
||||
MYSQL_DATABASE: dev
|
||||
ports:
|
||||
- "3306:3306"
|
||||
options: >-
|
||||
--health-cmd "mysqladmin ping -ppass"
|
||||
--health-interval 10s
|
||||
--health-start-period 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 10
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v3.0.1
|
||||
with:
|
||||
fetch-depth: 0 # Mandatory unless "latest" is set below.
|
||||
- uses: ariga/atlas-action@v0
|
||||
with:
|
||||
dir: ent/migrate/migrations
|
||||
dev-url: mysql://root:pass@localhost:3306/dev
|
||||
```
|
||||
Now, whenever we make a pull request with a potentially unsafe migration, the Atlas
|
||||
GitHub action will run and report the linting results. For example, for our data-dependent change:
|
||||

|
||||
|
||||
For more in depth documentation, see the [atlas-action](https://atlasgo.io/integrations/github-actions)
|
||||
docs on the Atlas website.
|
||||
|
||||
Let's fix the issue by back-filling the `title` column. Add the following
|
||||
statement to the migration file:
|
||||
|
||||
```sql title="ent/migrate/migrations/20221116051710_user_title_required.sql"
|
||||
-- modify "users" table
|
||||
UPDATE `users` SET `title` = "" WHERE `title` IS NULL;
|
||||
|
||||
ALTER TABLE `users` MODIFY COLUMN `title` varchar(255) NOT NULL;
|
||||
```
|
||||
|
||||
Re-hash the migration directory:
|
||||
|
||||
```shell
|
||||
atlas migrate hash --dir file://ent/migrate/migrations
|
||||
```
|
||||
|
||||
Re-running `atlas migrate lint`, we can see that the migration directory doesn't
|
||||
contain any unsafe changes:
|
||||
|
||||
```text
|
||||
atlas migrate lint --dev-url mysql://root:pass@localhost:3306/dev --dir file://ent/migrate/migrations --latest 1
|
||||
```
|
||||
|
||||
Because no issues are found, the command will exit with a zero exit code and no output.
|
||||
|
||||
When we commit this change to GitHub, the Atlas GitHub action will run and report that
|
||||
the issue is resolved:
|
||||
|
||||

|
||||
|
||||
## Conclusion
|
||||
|
||||
In this section, we saw how to use Atlas to verify that our migration is safe to run both
|
||||
locally and in CI.
|
||||
|
||||
This wraps up our tutorial on how to upgrade your Ent project from
|
||||
automatic migration to versioned migrations. To recap, we learned how to:
|
||||
|
||||
* Enable the versioned migrations feature-flag
|
||||
* Create a script to automatically plan migrations based on our desired Ent schema
|
||||
* Upgrade our production database to use versioned migrations with Atlas
|
||||
* Plan custom migrations for our project
|
||||
* Verify migrations safely using `atlas migrate lint`
|
||||
|
||||
In the next steps
|
||||
|
||||
:::note For more Ent news and updates:
|
||||
|
||||
- Subscribe to our [Newsletter](https://entgo.substack.com/)
|
||||
- Follow us on [Twitter](https://twitter.com/entgo_io)
|
||||
- Join us on #ent on the [Gophers Slack](https://entgo.io/docs/slack)
|
||||
- Join us on the [Ent Discord Server](https://discord.gg/qZmPgTE6RX)
|
||||
|
||||
:::
|
||||
|
||||
220
doc/md/versioned/07-programmatically.mdx
Normal file
220
doc/md/versioned/07-programmatically.mdx
Normal file
@@ -0,0 +1,220 @@
|
||||
---
|
||||
id: programmatically
|
||||
title: "Appendix: programmatic planning"
|
||||
---
|
||||
|
||||
import Tabs from '@theme/Tabs';
|
||||
import TabItem from '@theme/TabItem';
|
||||
|
||||
In the previous sections, we saw how to use the Atlas CLI to generate migration files. However, we can also
|
||||
generate these files programmatically. In this section we will review how to write Go code that can be used for
|
||||
automatically planning migration files.
|
||||
|
||||
## 1. Enable the versioned migration feature flag
|
||||
|
||||
:::info Supporting repository
|
||||
|
||||
The change described in this section can be found in PR [#2](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/2/files)
|
||||
in the supporting repository.
|
||||
|
||||
:::
|
||||
|
||||
The first step is to enable the versioned migration feature by passing in the `sql/versioned-migration` feature flag.
|
||||
Depending on how you execute the Ent code generator, you have to use one of the two options:
|
||||
|
||||
<Tabs
|
||||
defaultValue="ent"
|
||||
values={[
|
||||
{label: 'Using Ent CLI', value: 'ent'},
|
||||
{label: 'Using the entc package', value: 'entc'},
|
||||
]}>
|
||||
<TabItem value="ent">
|
||||
|
||||
If you are using the default go generate configuration, simply add the `--feature sql/versioned-migration` to
|
||||
the `ent/generate.go` file as follows:
|
||||
|
||||
```go
|
||||
package ent
|
||||
|
||||
//go:generate go run -mod=mod entgo.io/ent/cmd/ent generate --feature sql/versioned-migration ./schema
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="entc">
|
||||
|
||||
If you are using the code generation package (e.g. if you are using an Ent extension like `entgql`),
|
||||
add the feature flag as follows:
|
||||
|
||||
```go
|
||||
//go:build ignore
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"log"
|
||||
|
||||
"entgo.io/ent/entc"
|
||||
"entgo.io/ent/entc/gen"
|
||||
)
|
||||
|
||||
func main() {
|
||||
err := entc.Generate("./schema", &gen.Config{
|
||||
//highlight-next-line
|
||||
Features: []gen.Feature{gen.FeatureVersionedMigration},
|
||||
})
|
||||
if err != nil {
|
||||
log.Fatalf("running ent codegen: %v", err)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
Next, re-run code-generation:
|
||||
|
||||
```shell
|
||||
go generate ./...
|
||||
```
|
||||
|
||||
After running the code-generation, you should see the following
|
||||
[methods added](https://github.com/rotemtam/ent-versioned-migrations-demo/commit/e724fa32330d920fd405b9785fcfece2a46ea56c#diff-370235e5107bbdd35861063f3beff1507723ebdda6e29a39cdde1f1a944594d8)
|
||||
to `ent/migrate/migrate.go`:
|
||||
* `Diff`
|
||||
* `NamedDiff`
|
||||
|
||||
These methods are used to compare the state read from a database connection or a migration directory with the state defined
|
||||
by the Ent schema.
|
||||
|
||||
## 2. Automatic Migration planning script
|
||||
|
||||
:::info Supporting repository
|
||||
|
||||
The change described in this section can be found in PR [#4](https://github.com/rotemtam/ent-versioned-migrations-demo/pull/4/files)
|
||||
in the supporting repository.
|
||||
|
||||
:::
|
||||
|
||||
### Dev database
|
||||
|
||||
To be able to plan accurate and consistent migration files, Atlas introduces the
|
||||
concept of a [Dev database](https://atlasgo.io/concepts/dev-database), a temporary
|
||||
database which is used to simulate the state of the database after different changes.
|
||||
Therefore, to use Atlas to automatically plan migrations, we need to supply a connection
|
||||
string to such a database to our migration planning script. Such a database is most commonly
|
||||
spun up using a local Docker container. Let's do this now by running the following command:
|
||||
|
||||
```shell
|
||||
docker run --rm --name atlas-db-dev -d -p 3306:3306 -e MYSQL_DATABASE=dev -e MYSQL_ROOT_PASSWORD=pass mysql:8
|
||||
```
|
||||
|
||||
Using the Dev database we have just configured, we can write a script that will use Atlas to plan
|
||||
migration files for us. Let's create a new file called `main.go` in the `ent/migrate` directory
|
||||
of our project:
|
||||
|
||||
```go title=ent/migrate/main.go
|
||||
//go:build ignore
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log"
|
||||
"os"
|
||||
|
||||
// highlight-next-line
|
||||
"<project>/ent/migrate"
|
||||
|
||||
atlas "ariga.io/atlas/sql/migrate"
|
||||
"entgo.io/ent/dialect"
|
||||
"entgo.io/ent/dialect/sql/schema"
|
||||
_ "github.com/go-sql-driver/mysql"
|
||||
)
|
||||
|
||||
const (
|
||||
dir = "ent/migrate/migrations"
|
||||
)
|
||||
|
||||
func main() {
|
||||
ctx := context.Background()
|
||||
// Create a local migration directory able to understand Atlas migration file format for replay.
|
||||
if err := os.MkdirAll(dir, 0755); err != nil {
|
||||
log.Fatalf("creating migration directory: %v", err)
|
||||
}
|
||||
dir, err := atlas.NewLocalDir(dir)
|
||||
if err != nil {
|
||||
log.Fatalf("failed creating atlas migration directory: %v", err)
|
||||
}
|
||||
// Migrate diff options.
|
||||
opts := []schema.MigrateOption{
|
||||
schema.WithDir(dir), // provide migration directory
|
||||
schema.WithMigrationMode(schema.ModeReplay), // provide migration mode
|
||||
schema.WithDialect(dialect.MySQL), // Ent dialect to use
|
||||
schema.WithFormatter(atlas.DefaultFormatter),
|
||||
}
|
||||
if len(os.Args) != 2 {
|
||||
log.Fatalln("migration name is required. Use: 'go run -mod=mod ent/migrate/main.go <name>'")
|
||||
}
|
||||
// Generate migrations using Atlas support for MySQL (note the Ent dialect option passed above).
|
||||
//highlight-next-line
|
||||
err = migrate.NamedDiff(ctx, "mysql://root:pass@localhost:3306/dev", os.Args[1], opts...)
|
||||
if err != nil {
|
||||
log.Fatalf("failed generating migration file: %v", err)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
:::info
|
||||
|
||||
Notice that you need to make some modifications to the script above in the highlighted lines.
|
||||
Edit the import path of the `migrate` package to match your project and to supply the connection
|
||||
string to your Dev database.
|
||||
|
||||
:::
|
||||
|
||||
To run the script, first create a `migrations` directory in the `ent/migrate` directory of your
|
||||
project:
|
||||
|
||||
```text
|
||||
mkdir ent/migrate/migrations
|
||||
```
|
||||
|
||||
Then, run the script to create the initial migration file for your project:
|
||||
|
||||
```shell
|
||||
go run -mod=mod ent/migrate/main.go initial
|
||||
```
|
||||
Notice that `initial` here is just a label for the migration file. You can use any name you want.
|
||||
|
||||
Observe that after running the script, two new files were created in the `ent/migrate/migrations`
|
||||
directory. The first file is named `atlas.sum`, which is a checksum file used by Atlas to enforce
|
||||
a linear history of migrations:
|
||||
|
||||
```text title=ent/migrate/migrations/atlas.sum
|
||||
h1:Dt6N5dIebSto365ZEyIqiBKDqp4INvd7xijLIokqWqA=
|
||||
20221114165732_initialize.sql h1:/33+7ubMlxuTkW6Ry55HeGEZQ58JqrzaAl2x1TmUTdE=
|
||||
```
|
||||
|
||||
The second file is the actual migration file, which is named after the label we passed to the
|
||||
script:
|
||||
|
||||
```sql title=ent/migrate/migrations/20221114165732_initial.sql
|
||||
-- create "users" table
|
||||
CREATE TABLE `users` (`id` bigint NOT NULL AUTO_INCREMENT, `name` varchar(255) NOT NULL, `email` varchar(255) NOT NULL, PRIMARY KEY (`id`), UNIQUE INDEX `email` (`email`)) CHARSET utf8mb4 COLLATE utf8mb4_bin;
|
||||
-- create "blogs" table
|
||||
CREATE TABLE `blogs` (`id` bigint NOT NULL AUTO_INCREMENT, `title` varchar(255) NOT NULL, `body` longtext NOT NULL, `created_at` timestamp NOT NULL, `user_blog_posts` bigint NULL, PRIMARY KEY (`id`), CONSTRAINT `blogs_users_blog_posts` FOREIGN KEY (`user_blog_posts`) REFERENCES `users` (`id`) ON DELETE SET NULL) CHARSET utf8mb4 COLLATE utf8mb4_bin;
|
||||
```
|
||||
|
||||
## Other migration tools
|
||||
|
||||
Atlas integrates very well with Ent, but it is not the only migration tool that can be used
|
||||
to manage database schemas in Ent projects. The following is a list of other migration tools
|
||||
that are supported:
|
||||
|
||||
* [Goose](https://github.com/pressly/goose)
|
||||
* [Golang Migrate](https://github.com/golang-migrate/migrate)
|
||||
* [Flyway](https://flywaydb.org)
|
||||
* [Liquibase](https://www.liquibase.org)
|
||||
* [dbmate](https://github.com/amacneil/dbmate)
|
||||
|
||||
To learn more about how to use these tools with Ent, see the [docs](https://entgo.io/docs/versioned-migrations#create-a-migration-files-generator) on this subject.
|
||||
Reference in New Issue
Block a user