Galera Cluster 2 nodes - Unable to restart MySQL server on node 1
I was trying to learn more about Galera Cluster so i have one built with 2 nodes and i am experiencing this issue, similar to this one: Unable to restart MySQL server but i not sure how to proceed so i am asking to the community especially with someone with more experience than me on Galera Cluster. I'll try to summarize:
Configuration:
Galera Cluster 2 nodes - Every node is an Ubuntu 16.04 and has Mariadb 10.2.17 version.
Issue:
One of the node (node1) is in fault, unfortunately there is no error-log or general-log configured but on journalctl i can see that the error is something like "mariadb innodb assertion failure in file" and it's suggest to try innodb_force_recovery (1 to 6) but i don't know how the Galera sync works, or if it's an active\active configuration so i am not confident to eventually start a node not synchronized since days risking a split brain situation. Also, i see on the datadir that a file called "sst_in_progress" is present.
Consideration:
Will be ok to eventually delete the datadir on the fault node and restart the mysql service? Could it be enough to make it starts to sync with the node2 replicating the data without touching datas on node2 who is currently delivering service to my clients virtual machines? Also as far as i understand Galera cluster doesn't replicate system tables so i should export mysql.user table from node2 and import on node1 to have all the users and permission back. Thanks, i hope i succeed to explain the issue, if it's not clear please tell me.