[MYSQL] mysql服务器重启, 却有2个mysqld进程 ?
原创
忘记这是案例几了...
问题现象
昨晚变更完后, 关闭mysql数据库, 然后重启OS(kylin可能有swap问题, 遇到2次了.得手动到后台强制重启OS)后, 检查发现mysqld进程有2个.
说明: 本文所有截图均为测试环境截图.
我这里是使用centos模拟的, 真实环境的父进程是 1
分析
这种现象, 显然是mysqld启动了2次. 我们可以查看mysql的error日志验证
2024-11-15T05:29:22.643542Z 0 [ERROR] InnoDB: Unable to lock /data/mysql_3318/mysqldata/ibdata1 error: 11
2024-11-15T05:29:22.643570Z 0 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
2024-11-15T05:29:22.643573Z 0 [Note] InnoDB: Unable to open the first data file
2024-11-15T05:29:22.643585Z 0 [ERROR] InnoDB: Operating system error number 11 in a file operation.
2024-11-15T05:29:22.643597Z 0 [ERROR] InnoDB: Error number 11 means 'Resource temporarily unavailable'
2024-11-15T05:29:22.643634Z 0 [Note] InnoDB: Some operating system error numbers are described at http://dev.mysql.com/doc/refman/5.7/en/operating-system-error-codes.html
2024-11-15T05:29:22.643644Z 0 [ERROR] InnoDB: Cannot open datafile '/data/mysql_3318/mysqldata/ibdata1'
2024-11-15T05:29:22.643658Z 0 [ERROR] InnoDB: Could not open or create the system tablespace. If you tried to add new data files to the system tablespace, and it failed here, you should now edit innodb_data_file_path in my.cnf back to what it was, and remove the new ibdata files InnoDB created in this failed attempt. InnoDB only wrote those files full of zeros, but did not yet use them in any way. But be careful: do not remove old data files which contain your precious data!
2024-11-15T05:29:22.643662Z 0 [ERROR] InnoDB: Plugin initialization aborted with error Cannot open a file
2024-11-15T05:29:22.844247Z 0 [ERROR] Plugin 'InnoDB' init function returned error.
2024-11-15T05:29:22.844276Z 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2024-11-15T05:29:22.844280Z 0 [ERROR] Failed to initialize builtin plugins.
2024-11-15T05:29:22.844281Z 0 [ERROR] Aborting
2024-11-15T05:29:22.844318Z 0 [Note] Binlog end
2024-11-15T05:29:22.844413Z 0 [Note] Shutting down plugin 'MyISAM'
2024-11-15T05:29:22.844601Z 0 [Note] /soft/mysql_3318/mysqlbase/mysql/bin/mysqld: Shutdown complete发现确实是mysqld在争用ibdata1文件 (mysql打开文件的时候会对文件上锁, 即只允许一个写,就是防止这种情况的). 所以可以确定确实有2个mysqld在启动.
因为是在/etc/rc.local中配置的开机自启, 所以999进程的mysqld_safe和3199进程的mysqld才是我们设计内的自启进程. 那么PID:1085的进程是水启动的呢? 我这里可以看到是1028, 单真实环境均是1, 也就是开机自启的.
能做到开机自启的情况有:
- /etc/rc.local
- crontab (每分钟检查,没启动就执行启动脚本也能实现)
- systemctl/service
- /etc/crontab 这还有个定时任务呢...
首先查看 /etc/rc.local
虽然有看到我们的mysql启动脚本, 但还有个莫名其妙的/etc/start.sh脚本在. 可能存在脚本调脚本的情况. 但太复杂了, 我们就先看后面几种情况.
由于进程是mysql启动的, 所以我们只需要看root和mysql的定时任务就行,遗憾的是并没有相关的定时任务, 也没有配置服务管理, /etc/crontab也是空的. 那应该就是/etc/start.sh里面有调用mysqld_safe了.
我们查看该脚本, 确实发现了启动mysql的命令(我这测试环境的比较简单, 能一眼看出来, 实际环境可能存在多层调用的情况, 最离谱的是父进程都变成1了...)
问题基本上就是找到了. 询问相关人员,发现是沟通问题导致的,于是开发就把mysql的启动也加进去了...
解决办法
注释掉非标准的mysql启动, 然后重启mysqld服务(建议重启下OS,怕还有其它地方也有这种脚本....)
原创声明：本文系作者授权腾讯云开发者社区发表，未经许可，不得转载。
如有侵权，请联系 cloudcommunity@tencent.com 删除。
原创声明：本文系作者授权腾讯云开发者社区发表，未经许可，不得转载。
如有侵权，请联系 cloudcommunity@tencent.com 删除。
腾讯云开发者
