Database migrations are managed using the Alembic library. The consensus for OpenStack and SQLAlchemy is that this library is preferred over sqlalchemy-migrate.
Database migrations can be performed two ways: (1) via the API startup process, and (2) via a separate script.
Database migrations can be optionally enabled during the API startup process. Corollaries for this are that a new deployment should begin with only one node to avoid migration race conditions.
Alternatively, the automatic update startup behavior can be disabled, forcing the use of the migration script. This latter mode is probably safer to use in production environments.
A Barbican deployment goal is to update application and schema versions with zero downtime. The challenge is that at all times the database schema must be able to support two deployed application versions, so that a single migration does not break existing nodes running the previous deployment. For example, when deleting a column we would first deploy a new version that ignores the column. Once all nodes are ignoring the column, a second deployment would be made to remove the column from the database.
To achieve this goal, the following rules will be observed for schema changes:
Prior to invoking any migration steps below, change to your barbican project’s folder and activate your virtual environment per the Developer Guide.
If you are using PostgreSQL, please ensure you are using SQLAlchemy version 0.9.3 or higher, otherwise the generated version files will not be correct.
You cannot use these migration tools and techniques with SQLite databases.
Consider taking a look at the Alembic tutorial. As a brief summary: Alembic keeps track of a linked list of version files, each one applying a set of changes to the database schema that a previous version file in the linked list modified. Each version file has a unique Alembic-generated ID associated with it. Alembic generates a table in the project table space called alembic_version that keeps track of the unique ID of the last version file applied to the schema. During an update, Alembic uses this stored version ID to determine what if any follow on version files to process.
To make schema changes, new version files need to be added to the barbican/model/migration/alembic_migrations/versions/ folder. This section discusses two ways to add these files.
Alembic autogenerates a new script by comparing a clean database (i.e., one without your recent changes) with any modifications you make to the Models.py or other files. This being said, automatic generation may miss changes... it is more of an ‘automatic assist with expert review’. See What does Autogenerate Detect in the Alembic documentation for more details.
First, you must start Barbican using a version of the code that does not include your changes, so that it creates a clean database. This example uses Barbican launched with DevStack (see Barbican DevStack wiki page for instructions).
Note: For anything but trivial or brand new columns/tables, database backups and maintenance-window downtimes might be called for.
Barbican utilizes the Alembic version files as managing delta changes to the database. Therefore the first Alembic version file does not contain all time-zero database tables.
To create the initial Barbican tables in the database, execute the Barbican application per the ‘Via Application’ section.
Thereafter, it is suggested that only the barbican-db-manage.py script above be used to update the database schema per the ‘Manually’ section. Also, automatic database updates from the Barbican application should be disabled by adding/updating db_auto_create = False in the barbican-api.conf configuration file.
The last section of the Alembic tutorial describes the process used by the Barbican application to create and update the database table space automatically.
By default, when the Barbican API boots up it will try to create the Barbican database tables (using SQLAlchemy), and then try to apply the latest version files (using Alembic). In this mode, the latest version of the Barbican application can create a new database table space updated to the latest schema version, or else it can update an existing database table space to the latest schema revision (called head in the docs).
To bypass this automatic behavior, add db_auto_create = False to the barbican-api.conf file.
Run bin/barbican-db-manage.py -d <Full URL to database, including user/pw> upgrade -v head, which will cause Alembic to apply the changes found in all version files after the version currently written in the target database, up until the latest version file in the linked chain of files.
To upgrade to a specific version, run this command: bin/barbican-db-manage.py -d <Full URL to database, including user/pw> upgrade -v <Alembic-ID-of-version>. The Alembic-ID-of-version is a unique ID assigned to the change such as1a0c2cdafb38.
To downgrade to a specific version, run this command: bin/barbican-db-manage.py -d <Full URL to database, including user/pw> downgrade -v <Alembic-ID-of-version>.