关于spark:mlflow-升级upgrade

背景

mlflow 的更新迭代速度还是很快的，均匀一个月一个大版本的更新，截止到 11 月 1 号，曾经更新到了 1.11.0 版本
咱们查看 mlflow release, 就能看到早在 1.10.0 版本，就提供了对 model registry 的更好的 feature 反对，以及可能对试验进行逻辑删除操作，
而这些 features 在 mlflow 1.4.0 是没有的，特地是删除试验的个性，如果试验很多的状况下，咱们看到的试验是横七竖八的，很不不便咱们进行治理，所以咱们进行 mlflow 的降级

降级以及筹备

参照之前 mlflow 的搭建应用，咱们先建设 mlflow 1.4.0 和 mlflow 1.11.0 的 conda 环境
假如你曾经建设好了对应的 conda 环境，且别离为 mlflow-1.4.0 和 mlflow-1.11.0 则执行：

conda activate mlflow-1.11.0

参考 mlflow db upgrade，执行

mlflow db upgrade mysql://user:passwd@host:port/db
如：mlflow db upgrade mysql://root:root@localhost/mlflow

其中

如果执行胜利则会看到如下输入信息:

2020/11/02 10:24:50 INFO mlflow.store.db.utils: Updating database tables
INFO  [alembic.runtime.migration] Context impl MySQLImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade 2b4d017a5e9b -> cfd24bdc0731, Update run status constraint with killed
INFO  [alembic.runtime.migration] Running upgrade cfd24bdc0731 -> 0a8213491aaa, drop_duplicate_killed_constraint
WARNI [0a8213491aaa_drop_duplicate_killed_constraint_py] Failed to drop check constraint. Dropping check constraints may not be supported by your SQL database. Exception content: (MySQLdb._exceptions.ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near'CHECK status'at line 1")
[SQL: ALTER TABLE runs DROP CHECK status]
(Background on this error at: http://sqlalche.me/e/f405)
INFO  [alembic.runtime.migration] Running upgrade 0a8213491aaa -> 728d730b5ebd, add registered model tags table
INFO  [alembic.runtime.migration] Running upgrade 728d730b5ebd -> 27a6a02d2cf1, add model version tags table
INFO  [alembic.runtime.migration] Running upgrade 27a6a02d2cf1 -> 84291f40a231, add run_link to model_version

如果此时再在 mlflow 1.4.0 的环境下 再执行：

mlflow server \
      --backend-store-uri mysql://root:root@localhost/mlflow \
      --host 0.0.0.0 -p 5002 \
      --default-artifact-root s3://mlflow

就会报错：

2020/11/02 10:25:41 ERROR mlflow.cli: Error initializing backend store
2020/11/02 10:25:41 ERROR mlflow.cli: Detected out-of-date database schema (found version 84291f40a231, but expected 2b4d017a5e9b). Take a backup of your database, then run 'mlflow db upgrade <database_uri>' to migrate your database to the latest schema. NOTE: schema migration may result in database downtime - please consult your database's documentation for more detail.
Traceback (most recent call last):
  File "/Users/ljh/opt/miniconda3/envs/mlflow-1.4.0-dev/lib/python3.6/site-packages/mlflow/cli.py", line 263, in server
    initialize_backend_stores(backend_store_uri, default_artifact_root)
  File "/Users/ljh/opt/miniconda3/envs/mlflow-1.4.0-dev/lib/python3.6/site-packages/mlflow/server/handlers.py", line 97, in initialize_backend_stores
    _get_tracking_store(backend_store_uri, default_artifact_root)
  File "/Users/ljh/opt/miniconda3/envs/mlflow-1.4.0-dev/lib/python3.6/site-packages/mlflow/server/handlers.py", line 83, in _get_tracking_store
    _tracking_store = _tracking_store_registry.get_store(store_uri, artifact_root)
  File "/Users/ljh/opt/miniconda3/envs/mlflow-1.4.0-dev/lib/python3.6/site-packages/mlflow/tracking/_tracking_service/registry.py", line 37, in get_store
    return builder(store_uri=store_uri, artifact_uri=artifact_uri)
  File "/Users/ljh/opt/miniconda3/envs/mlflow-1.4.0-dev/lib/python3.6/site-packages/mlflow/server/handlers.py", line 54, in _get_sqlalchemy_store
    return SqlAlchemyStore(store_uri, artifact_uri)
  File "/Users/ljh/opt/miniconda3/envs/mlflow-1.4.0-dev/lib/python3.6/site-packages/mlflow/store/tracking/sqlalchemy_store.py", line 99, in __init__
    mlflow.store.db.utils._verify_schema(self.engine)
  File "/Users/ljh/opt/miniconda3/envs/mlflow-1.4.0-dev/lib/python3.6/site-packages/mlflow/store/db/utils.py", line 52, in _verify_schema
    "more detail." % (current_rev, head_revision))
mlflow.exceptions.MlflowException: Detected out-of-date database schema (found version 84291f40a231, but expected 2b4d017a5e9b). Take a backup of your database, then run 'mlflow db upgrade <database_uri>' to migrate your database to the latest schema. NOTE: schema migration may result in database downtime - please consult your database's documentation for more detail.

这阐明降级胜利

此时再在 mlflow 1.11.0 的 conda 环境下执行：

 mlflow server \
      --backend-store-uri mysql://root:root@localhost/mlflow \
      --host 0.0.0.0 -p 5003 \
      --default-artifact-root s3://mlflow

就能失常的看到页面, 这样 mlflow 从 1.4.0 到 1.11.0 的降级就实现了

注意事项

如果是线上操作，则先备份数据库，因为该降级不肯定能保障降级胜利，如降级失败，可参照失败解决进行解决