事原一次突发的停电事故，当时恰好新办公场地搬迁机房UPS没有到位，办公区域一停电（其他楼层在施工一下就将其中几层的电源关掉了，这也导致我们公司机房直接停电）机房所有服务器都在没有正常关机的情况下被强行关机了。当时还好硬件没有损坏，只是部分服务无法启动，其中就包括Docker和MySQL环境。

其实Docker的那个还比较好办，无非就是Rancher在断电时IO控制单元没有结束，导致在重启的时候该控制单元一直检测IO仍在继续但已经无法读取原有IO线程信息所以无法启动，这个还比较好解决只需要删除相关的配置即可（网上都有解决办法这里就不解释了）。MySQL这边就严重一点，主要是报一个Innodb文件损坏的错误，这有可能会导致整个表乃至库的数据丢失，报错信息如下所示：

2020-06-10 06:29:15 7f16b2294700 InnoDB: Error: Fetch of persistent statistics requested for table "pm"."pm_daily" but the required system tables mysql.innodb_table_stats and mysql.innodb_index_stats are not present or have unexpected structure. Using transient stats instead.
2020-06-10 06:43:31 7f16b2294700 InnoDB: Error: Fetch of persistent statistics requested for table "pm"."pm_member" but the required system tables mysql.innodb_table_stats and mysql.innodb_index_stats are not present or have unexpected structure. Using transient stats instead.
2020-06-10 06:43:41 7f16a9ffb700 InnoDB: Error: Table "mysql"."innodb_table_stats" not found.
2020-06-10 06:43:41 7f16a9ffb700 InnoDB: Recalculation of persistent statistics requested for table "pm"."pm_member" but the required persistent statistics storage is not present or is corrupted. Using transient stats instead.
2020-06-10 06:44:12 7f16b2294700 InnoDB: Error: Table "mysql"."innodb_table_stats" not found.
2020-06-10 06:44:12 7f16b2294700 InnoDB: Error: Fetch of persistent statistics requested for table "pm"."pm_milestone" but the required system tables mysql.innodb_table_stats and mysql.innodb_index_stats are not present or have unexpected structure. Using transient stats instead.
2020-06-10 06:44:52 7f16b2294700 InnoDB: Error: Table "mysql"."innodb_table_stats" not found.
2020-06-10 06:44:52 7f16b2294700 InnoDB: Error: Fetch of persistent statistics requested for table "pm"."pm_notificate" but the required system tables mysql.innodb_table_stats and mysql.innodb_index_stats are not present or have unexpected structure. Using transient stats instead.
2020-06-10 06:45:02 7f16a9ffb700 InnoDB: Error: Table "mysql"."innodb_table_stats" not found.
2020-06-10 06:45:02 7f16a9ffb700 InnoDB: Recalculation of persistent statistics requested for table "pm"."pm_notificate" but the required persistent statistics storage is not present or is corrupted. Using transient stats instead.
2020-06-10 06:45:27 7f16b2294700 InnoDB: 
Error: Table "mysql"."innodb_table_stats" not found.
2020-06-10 06:45:27 7f16b2294700 InnoDB: Error: Fetch of persistent statistics requested for table "pm"."pm_notificate_users" but the required system tables mysql.innodb_table_stats and mysql.innodb_index_stats are not present or have unexpected structure. Using transient stats instead.

......

2020-06-10 06:49:55 7f16a9ffb700 InnoDB: Recalculation of persistent statistics requested for table "pm"."sys_user_level" but the required persistent statistics storage is not present or is corrupted. Using transient stats instead.
2020-06-10 06:50:13 7f16b2294700 InnoDB: Error: Table "mysql"."innodb_table_stats" not found.
2020-06-10 06:50:13 7f16b2294700 InnoDB: Error: Fetch of persistent statistics requested for table "pm"."sys_user_role" but the required system tables mysql.innodb_table_stats and mysql.innodb_index_stats are not present or have unexpected structure. Using transient stats instead.
2020-06-10 06:50:23 7f16a9ffb700 InnoDB: Error: Table "mysql"."innodb_table_stats" not found.
2020-06-10 06:50:23 7f16a9ffb700 InnoDB: Recalculation of persistent statistics requested for table "pm"."sys_user_role" but the required persistent statistics storage is not present or is corrupted. Using transient stats instead.
2020-06-10 07:11:55 1 [Note] mysqld: Normal shutdown

这一看一堆报错基本上都是来自Innodb异常，择取其中一段开看看 ：

2020-06-10 06:49:55 7f16a9ffb700 InnoDB: Recalculation of persistent statistics requested for table "pm"."sys_user_level" but the required persistent statistics storage is not present or is corrupted. Using transient stats instead.

通过上面报错信息得知，MySQL在启动时需要做持久性统计，但是该统计的存储信息已经损坏，所以只能采用瞬时统计模式了。关于这种情况一般来说都会先尝试使用recover模式恢复，实在不行再说。

解决

有见及此先停止当前应用，之后找到mysql.cnf文件在末尾处加入`innodb_force_recovery = 1`配置，之后重新启动服务。

虽然recover模式能够使MySQL正常启动了，但只要将`innodb_force_recovery=0`后系统又恢复原样。也就是说recover模式并没有真正意义上解决修复问题。

既然这样，先使用recover模式进去系统后将受影响的数据库中的所有内容以sql脚本方式导出，这里之所以不用dump导出是因为提示上面说到的，是持久性统计遭到破坏或者丢失了。而dump导出会全环境内容导出，因此这里建议采用sql脚本方式导出。

导出完毕后将当前schema删掉并关闭MySQL服务并将innodb_force_recovery注释掉。之后按照正常的方式启动并通过脚本重建库和表并将数据导入就可以了。