Request for comments: Skeema v2 changes

Throughout Skeema’s nine-year development history, we’ve avoided backwards-incompatible changes in our periodic v1.x feature releases. However, the time has finally come to iron out some old wrinkles in a v2 release. We’re avoiding any overly-drastic breaking changes, but there may be a few minor adjustments which affect your configuration.

These changes aren’t set in stone yet though, as Skeema v2 is still a number of months away! If any of these deprecations will negatively affect you, let us know on the discussion board.

Dropping support for ancient database versions

Historically, Skeema has only added support for new versions of MySQL and MariaDB over time, but never pruned old ones. However, as both database vendors moved to a “new features every quarter” release model in recent years, supporting and testing on an ever-growing range of server versions has become impractical.

As such, Skeema v2 will drop support for MySQL 5.5 (GA Dec 2010, EOL Dec 2018) and MariaDB 10.1 (GA Jun 2014, EOL Oct 2020). Given their age, these server versions are nearly unused in the industry today. Removing them from our codebase will simplify our integration testing and release overhead, and allow the deletion of some old conditional logic. For example, MySQL 5.5 does not even support InnoDB online DDL (ALGORITHM and LOCK clauses) at all.

Skeema v2 will also likely deprecate (but not remove) support for MySQL 5.6, MariaDB 10.2, and possibly other versions that have been EOL for multiple years. These server versions will still be fully supported in Skeema v2.x, but a warning log will be emitted, with full removal occurring in a future Skeema v3. Moving forwards, we anticipate new Skeema major version releases every 12 to 24 months.

Removal of unnecessary options

Skeema’s option list has grown quite large over the years, and some light pruning can help reduce the complexity of the tool. Several options are currently planned for outright removal in Skeema v2. These will first become deprecated in Skeema v1.13, with a warning message logged upon any use of them.

temp-schema-threads

This option is commonly misconfigured with excessively high values, potentially causing workspace connections to contend with each other. Beginning with Skeema v1.13, a better replacement option called temp-schema-mode will offer clearer enum values for different situations along the performance vs load tradeoff axis.

temp-schema-binlog

This option is rarely ever configured, since its default of “auto” is generally ideal already. So in v2, “auto” behavior will always be in effect.

lint-display-width

This linter rule is no longer applicable in modern MySQL: integer display widths were removed in MySQL 8.0.19, released in January 2020. Although MariaDB still supports int display widths, incorrect values are not meaningfully problematic, and this linter rule can generate too much noise.

alter-validate-virtual

The syntax controlled by this option isn’t supported in MariaDB, and using alter-algorithm=copy achieves the same effect anyway in both MySQL and MariaDB.

skip-my-cnf

This option is infrequently used, as it’s generally always beneficial to obtain connectivity options (user, password, SSL options, etc) from .my.cnf if they’re already set there. Only a limited selection of options from .my.cnf are applied anyway, and users can always override those .my.cnf configurations in .skeema option files, or on the command-line.

Behavior changes

Docker workspaces, cleanup, and tmpfs

Recent Skeema releases have massively improved the performance of workspace=docker, including use of an in-memory tmpfs data mount whenever docker-cleanup=destroy is explicitly configured. In v2, we plan to move this tmpfs behavior to a separate option, which will be enabled by default. Since Skeema workspace metadata is always inherently ephemeral, tmpfs can be beneficial regardless of the overall container’s lifecycle.

Additionally, in Skeema v2 the default value for docker-cleanup will change from “none” to “stop”, halting containers upon the skeema process exiting. Skeema’s containerized databases restart quite quickly on modern hardware. Even though with tmpfs that means reinitializing the data directory, this is now a fast operation because we tune down all InnoDB log sizes quite substantially.

Dumping multiple schemas

When running skeema init without a specific --schema, currently all databases are dumped, and a multi-level directory layout is created – even if the server only has one non-system schema. This behavior will become explicitly opt-in (like mysqldump’s use of --all-databases) to reduce confusion for new users.

Similarly, the default for skeema pull will be revised such that --new-schemas now defaults to false, meaning it must be provided explicitly if you wish to auto-detect and dump any new / previously-unexported schemas.

No auto-reformatting

In Skeema v1, by default both skeema lint and skeema pull reformat all CREATE statements to use the database server’s canonical format, unless you supply option --skip-format. In Skeema v2, this default will flip, so that you must explicitly use --format to enable that behavior. The old default reformatting behavior predated the existence of the separate skeema format command, and makes less sense with its presence.

Wording change of “instance” to “server”

Recent Skeema releases have revised the wording of logging, help text, and error messages to prefer “database server” instead of “instance”, for sake of clarity and consistency. However, two remaining uses of “instance” could not be updated previously due to backwards-compatibility concerns, both of which will finally be updated in v2:

• The concurrent-instances option will be renamed to concurrent-servers. To make the transition easier, both names will be available in Skeema v1.13, with the old name being removed in v2.
• The STDOUT output of skeema diff and skeema push contains an -- instance: ... comment, which is intended to help automation users parse Skeema’s output when operating against multiple target servers at once. This line will now change to -- server: ... in v2.

Auto-increment column type linting

The default value for allow-auto-inc has always excluded signed types, since storing negative numbers in an AUTO_INCREMENT column is typically unnecessary. However, over the years we’ve found this default creates far more noise than any other linter rule, so Skeema v2 will adjust the default to include both signed and unsigned types.

Community discussions

We warmly welcome community feedback on all of these proposed changes, as well as suggestions for additional items to consider. Share your thoughts on the GitHub Discussions thread.