mariadb server mysteriously seems to be crashing
hello - the last couple of days my server seems to have mysteriously stopped. simply doing:
systemctl stop mariadb.service ; systemctl start mariadb.service ;
seems to temporarily fix it. i am tempted to have something like this run in a cron job every hour just to be safe.
Server version: 10.2.27-MariaDB MariaDB Server
# systemctl status mariadb.service ; ● mariadb.service - MariaDB 10.2.27 database server Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled) Drop-In: /etc/systemd/system/mariadb.service.d └─migrated-from-my.cnf-settings.conf Active: failed (Result: timeout) since Sun 2019-10-06 15:28:21 EDT; 2h 27min ago Docs: man:mysqld(8) https://mariadb.com/kb/en/library/systemd/ Process: 20375 ExecStartPost=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS) Process: 20343 ExecStart=/usr/sbin/mysqld $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION (code=killed, signal=KILL) Process: 14181 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= || VAR=`/usr/bin/galera_recovery`; [ $? -eq 0 ] && systemct l set-environment _WSREP_START_POSITION=$VAR || exit 1 (code=killed, signal=TERM) Process: 14154 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS) Main PID: 20343 (code=killed, signal=KILL) CGroup: /system.slice/mariadb.service
EDIT:
here is what i am seeing now:
# systemctl status -l mariadb.service ; ## restart mariadb ● mariadb.service - MariaDB 10.2.27 database server Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled) Drop-In: /etc/systemd/system/mariadb.service.d └─migrated-from-my.cnf-settings.conf Active: activating (start) since Sun 2019-10-06 18:31:14 EDT; 43s ago Docs: man:mysqld(8) https://mariadb.com/kb/en/library/systemd/ Process: 10240 ExecStartPost=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS) Process: 13399 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= || VAR=`/usr/bin/galera_recovery`; [ $? -eq 0 ] && systemctl set-environment _WSREP_START_POSITION=$VAR || exit 1 (code=exited, status=0/SUCCESS) Process: 13320 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS) Main PID: 13455 (mysqld) CGroup: /system.slice/mariadb.service └─13455 /usr/sbin/mysqld Oct 06 18:31:37 edwardsmark.info mysqld[13455]: 2019-10-06 18:31:32 140379062454464 [Note] InnoDB: Using Linux native AIO Oct 06 18:31:37 edwardsmark.info mysqld[13455]: 2019-10-06 18:31:32 140379062454464 [Note] InnoDB: Number of pools: 1 Oct 06 18:31:37 edwardsmark.info mysqld[13455]: 2019-10-06 18:31:32 140379062454464 [Note] InnoDB: Using SSE2 crc32 instructions Oct 06 18:31:37 edwardsmark.info mysqld[13455]: 2019-10-06 18:31:32 140379062454464 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M Oct 06 18:31:37 edwardsmark.info mysqld[13455]: 2019-10-06 18:31:32 140379062454464 [Note] InnoDB: Completed initialization of buffer pool Oct 06 18:31:37 edwardsmark.info mysqld[13455]: 2019-10-06 18:31:32 140378200946432 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). Oct 06 18:31:37 edwardsmark.info mysqld[13455]: 2019-10-06 18:31:34 140379062454464 [Note] InnoDB: Highest supported file format is Barracuda. Oct 06 18:31:37 edwardsmark.info mysqld[13455]: 2019-10-06 18:31:35 140379062454464 [Note] InnoDB: Starting crash recovery from checkpoint LSN=38062514322 Oct 06 18:31:13 edwardsmark.info systemd[1]: Stopped MariaDB 10.2.27 database server. Oct 06 18:31:14 edwardsmark.info systemd[1]: Starting MariaDB 10.2.27 database server...
AND ANOTHER CRASH!
# systemctl status -l mariadb.service ; ● mariadb.service - MariaDB 10.2.27 database server Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled) Drop-In: /etc/systemd/system/mariadb.service.d └─migrated-from-my.cnf-settings.conf Active: failed (Result: timeout) since Sun 2019-10-06 18:38:58 EDT; 6min ago Docs: man:mysqld(8) https://mariadb.com/kb/en/library/systemd/ Process: 10240 ExecStartPost=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS) Process: 14236 ExecStart=/usr/sbin/mysqld $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION (code=exited, status=0/SUCCESS) Process: 14026 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= || VAR=`/usr/bin/galera_recovery`; [ $? -eq 0 ] && systemctl set-environment _WSREP_START_POSITION=$VAR || exit 1 (code=exited, status=0/SUCCESS) Process: 13898 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS) Main PID: 14236 (code=exited, status=0/SUCCESS) Status: "MariaDB server is down" Oct 06 18:38:53 edwardsmark.info mysqld[14236]: 2019-10-06 18:38:53 139696513103616 [Note] InnoDB: FTS optimize thread exiting. Oct 06 18:38:54 edwardsmark.info mysqld[14236]: 191006 18:38:54 server_audit: STOPPED Oct 06 18:38:56 edwardsmark.info mysqld[14236]: 2019-10-06 18:38:56 139696652347136 [Note] InnoDB: Starting shutdown... Oct 06 18:38:56 edwardsmark.info mysqld[14236]: 2019-10-06 18:38:56 139696403367680 [Note] InnoDB: Buffer pool(s) load completed at 191006 18:38:56 Oct 06 18:38:56 edwardsmark.info mysqld[14236]: 2019-10-06 18:38:56 139696403367680 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/mysql/ib_buffer_pool Oct 06 18:38:56 edwardsmark.info mysqld[14236]: 2019-10-06 18:38:56 139696403367680 [Note] InnoDB: Buffer pool(s) dump completed at 191006 18:38:56 Oct 06 18:38:58 edwardsmark.info mysqld[14236]: 2019-10-06 18:38:57 139696652347136 [Note] InnoDB: Shutdown completed; log sequence number 38062514359 Oct 06 18:38:58 edwardsmark.info systemd[1]: Failed to start MariaDB 10.2.27 database server. Oct 06 18:38:58 edwardsmark.info systemd[1]: Unit mariadb.service entered failed state. Oct 06 18:38:58 edwardsmark.info systemd[1]: mariadb.service failed.
Content reproduced on this site is the property of its respective owners,
and this content is not reviewed in advance by MariaDB. The views, information and opinions
expressed by this content do not necessarily represent those of MariaDB or any other party.