Schema versioning

So far, all examples in this crate have focused on schemas that are one-and-done with no eye towards future maintenance requirements. Sadly, this is rarely the case for production databases — schemas change over time, and the usual way to handle this is via migrations, which define versions of the database schema and how to migrate the database contents between said versions. This chapter outlines the intended method of managing database schema iterations and migrations with microrm.¹

The core implementation detail that microrm migrations hinge upon is that the database table name for an entity is determined by the name of the struct that has the Entity derive macro applied to it. Thus, you can define multiple versions of an entity by placing them in separate Rust modules; this leads to a module structure such as:

schema/
    mod.rs
    v1.rs
    v2.rs
    v3.rs
    ...

Here, each vX.rs file contains the database schema for that particular version, and can reference earlier versions of the schema for tables that have not changed, and mod.rs simply re-exports the contents of the most recent schema module. For example, one might have something like the following:

• v1.rs:

use microrm::prelude::*;

/// Initial Foo definition
#[derive(Clone, Entity)]
pub struct Foo {
    #[key]
    pub bar: String,
    pub baz: String,
}

#[derive(Schema)]
pub struct Schema {
    pub foo: microrm::Table<Foo>,
}

• v2.rs:

use microrm::prelude::*;

/// New Foo definition with linked Aleph entity
#[derive(Clone, Entity)]
pub struct Foo {
    #[key]
    pub bar: String,
    pub baz: String,
    pub alephs: microrm::RelationMap<Aleph>
}

/// Aleph record
#[derive(Clone, Entity)]
pub struct Aleph {
    pub bet: u64,
    pub dalet: Vec<u8>,
}

#[derive(Schema)]
pub struct Schema {
    pub foo: microrm::Table<Foo>,
}

• v3.rs:

use microrm::prelude::*;

pub use super::v2::Aleph;

#[derive(Clone, Entity)]
pub struct Foo {
   #[key]
   pub bar: String,
   pub baz: String,
   /// We're adding in an extra (nullable) field
   pub quux: Option<f32>,
   /// We aren't changing Aleph at all, so we can reuse the previous definition here.
   pub alephs: microrm::RelationMap<Aleph>,
}

#[derive(Schema)]
pub struct Schema {
    pub foo: microrm::Table<Foo>,
}

• mod.rs:

mod v1;
mod v2;
mod v3;

pub use v3::*;

type SchemaList = (v1::Schema, v2::Schema, v3::Schema, );

pub fn apply_migrations(pool: &microrm::ConnectionPool) -> microrm::DBResult<Schema> {
    microrm::migration::run_migration::<SchemaList>(pool)
}

As the definition of the entities changes, consumer code only ever sees the most recent types – thus functionally the same as changing the definition in-place – but the previous schemas are available to microrm for the purposes of migrating between different versions. Sharp readers will note that a crucial piece is missing: the description of how to convert the data between the schemas; indeed, the Rust compiler will complain if you tried to compile the above. We will discuss migration logic after a brief segue regarding the handling bidirectional relationships.

Bidirectional relationships

Some care is required when handling bidirectional relationships, due to the Relation tag struct holding a reference to both types. In particular, both types must have updated versions in the new schema, along with a new Relation-implementing tag struct. This can unfortunately require ‘chained’ updates if you have a web of bidirectional relationships among a group of entities.

Migration logic

The primary trait for migrations is MigratableItem; broadly speaking, a schema S that implements MigratableItem<T> can be migrated from the schema T. The implementation of MigratableItem controls the overarching data flow between the two schemas, farming out to other helpers as needed.

One such built-in helper is the MigratableEntity trait. Rather than describing a table-to-table transformation, MigratableEntity allows describing a row-to-row (i.e. entity-to-entity) transformation. As with MigratableItem<T>, an entity E : MigratableEntity<T> allows an instance of E to be constructed from an instance of T.

A helper attribute is built-in to the Entity derive macro to aid with some common implementations of MigratableEntity, specifically migratable_to. This will provide a stub MigratableEntity<T> implementation that performs no data transforms, and is intended for use when the only changes are to relation tags.

Nested schemas

Schemas can be nested to aid in organization. This can help reduce the number of duplicated lines between schema versions if only one sub-schema of a group is untouched.

Complete worked example

TODO

1. microrm is sufficiently flexible that other management strategies may also work. If you find something useful that is materially different, please let us know! ↩