多源(主)复制
Contents
MariaDB starting with 10.0
多源复制意味着一个服务器能从多个从服务器上复制。这是即将发布的MariaDB 10.0的一个新特性
新语法
在命令指定你想起作用的master的连接名或者设置@@default_master_connection
变量为你想生效的连接。
连接名可以是任意的不超过64个字符的字符串.连接名对比的时候不考虑大小写(大小写不敏感)。 你应当保持连接名段(使用固定的连接名),因为他将用来作relay和master信息的index文件的前缀。
新语法介绍如何操作多个连接:
CHANGE MASTER ["connection_name"] ...
FLUSH RELAY LOGS ["connection_name"]
MASTER_POS_WAIT(....,["connection_name"])
RESET SLAVE ["connection_name"]
SHOW RELAYLOG ["connection_name"] EVENTS
SHOW SLAVE ["connection_name"] STATUS
SHOW ALL SLAVES STATUS
START SLAVE ["connection_name"...]]
START ALL SLAVES ...
STOP SLAVE ["connection_name"] ...
STOP ALL SLAVES ...
原始的老式的连接是一个空字符串''
。你如果不想用的话可以不用这个连接。
用CHANGE MASTER来创建新的master连接.
用 RESET SLAVE "connection_name" ALL 来永久的删除连接。
多源复制的变量
新复制变量 @@default_master_connection
指定了如果你不指定一个连接的话那个链接将会被命令和变量使用。默认这个值是 ''
(默认连接名)。下面的复制变量是针对局部的连接的(换句话说,他们显示了 @@default_master_connection
的值)。我们正致力于将重要的对于连接局部化。
类型 | 名称 | 描述 |
---|---|---|
变量 | Max_relay_log_size | relay log的最大值. 如果是0的话,那么在启动的时候就会被设置成 max_binlog_size 的大小 |
状态 | Slave_heartbeat_period | 多久从master请求一个心跳包 (以秒计算). |
状态 | Slave_received_heartbeats | 我们从master收到了多少个心跳包. |
状态 | Slave_running | 显示slave是否正在运行。YESS表示 sql 和 IO 线程是活动的。No表示其中任意一个没在运行中。""表示 @@default_master_connection 不存在。 |
变量 | Sql_slave_skip_counter | 复制日志要忽略多少个条目(主要用在日志中有错误的情况下)。 |
你可以用 SESSION
或者是 GLOBAL
访问以上所有的变量。
注意,和MySQL形成对比的是,所有的变量总是显示正确的有效的值!
例如:
set @@default_master_connection=''; show status like 'Slave_running'; set @@default_master_connection='other_connection'; show status like 'Slave_running';
如果 @@default_master_connection
包含一个不存在的名称,你将收到一个警告.
所有其他的master相关的变量都是全局的,并且影响 "" 连接或者是所有的连接。例如,
Slave_retried_transactions
现在显示所有的slave的重试事务次数。
新添加的状态变量:
名称 | 描述 |
---|---|
Com_start_all_slaves | 执行 START ALL SLAVES 命令的次数。 |
Com_start_slave | 执行 START SLAVE 命令的次数。取代了 Com_slave_start . |
Com_stop_slave | 执行 STOP SLAVE 命令的次数。取代了 Com_slave_stop . |
Com_stop_all_slaves | 执行 STOP ALL SLAVES 命令的次数 |
SHOW ALL SLAVES STATUS
有以下的新的列:
名称 | 描述 |
---|---|
Connection_name | master的连接名。 这是第一个变量 |
Slave_SQL_State | SQL 线程的状态 |
Retried_transactions | 这个连接重试事务的次数。 |
Max_relay_log_size | 这个连接的最大的relay日志的大小。 |
Executed_log_entries | slave已经指向了多少个日志条目。 |
Slave_received_heartbeats | 从master获得了多少个心跳。 |
Slave_heartbeat_period | 多久从master请求一个心跳包(以秒计算)。 |
新文件
被多源复制使用的新文件的基本准则是:他们有在扩展名前被冠以 连接名
前缀的和原来的中继日志文件类似的名字。 主要的例外是,保存所有连接名字的 文件master-info-file
被简单的命名为带有 multi-
前缀的 master-info-file
。
当你使用多源复制的时候,下面的文件将被创建:
名称 | 描述 |
---|---|
multi-master-info-file | master-info-file (一般是 master.info ) 带上了 multi- 前缀。这里面是所有使用中的master连接信息。 |
master-info-file -connection_name.extension | 包含了当前master应用到slave的位置。扩展名一般是 .info |
relay-log -connection_name.xxxxx | relay-log 有了一个 connection_name 的前缀. xxxxx 是 relay log 的编号。这里面保存的是从master读取的复制数据。 |
relay-log-index -connection_name.extension | 包含可用的 relay-log -connection_name.xxxxx 文件的名字。扩展名一般是 .index |
relay-log-info-file -connection_name.extension | 包含relay日志中的当前master的位置。扩展名一般是 .info |
当创建这些文件的时候,连接名被转化成小写的,并且其中所有的特殊字符都被转化了,就和mysql表名中的特殊字符被转化一样。这样做是为了方便文件名可以在不同系统上移植。
提示:
你只需要指定 --log-base-name
,所有的其他变量将会使用这个作为前缀,而不用为mysqld
指定 --relay-log
, --relay-log-index
, --relay-log-index
, --general-log
, --slow-log
,
--log-bin
, --log-bin-index
这些的名字。
Other things
- All error messages from a slave with a connection name, that are written to the error log, are prefixed with
Master 'connection_name':
. This makes it easy to see from where an error originated. - Errors
ER_MASTER_INFO
andWARN_NO_MASTER_INFO
now includes connection_name. - There is no conflict resolution. The assumption is that there are no conflicts in data between the different masters.
- All executed commands are stored in the normal binary log (nothing new here).
- If the server variable
log_warnings
> 1 then you will get some information in the log about how the multi-master-info file is updated (mainly for debugging). - SHOW [FULL] SLAVE STATUS has one line per connection and more columns than before. Note that the first column is the
connection_name
! RESET SLAVE
now deletes all relay-log files.
Typical use cases
- You are partitioning your data over many masters and would like to get it all together on one machine to do analytical queries on all data.
- You have many databases spread over many MariaDB/MySQL servers and would like to have all of them on one machine as an extra backup.
Limitations
- You can for now only have 64 masters (trivial to increase if necessary).
- Each active connection will create 2 threads (as is normal for MariaDB replication).
- You should ensure that all master have different
server-id
's. If you don't do this, you will get into trouble if you try to replicate from the multi-source slave back to your masters. - One can change
max_relay_log_size
for any active connection, but new connections will always use the server startup value formax_relay_log_size
, which can't be changed at runtime. - Option
innodb-recovery-update-relay-log
(xtradb feature to store and restore relay log position for slaves) only works for the default connection ''. As this option is not really safe and can easily cause loss of data if you use storage engines other than InnoDB, we don't recommend this option to be used. Slave_net_timeout
affects all connections. We don't check anymore if it's less thanSlave_heartbeat_period
, as this doesn't make sense in a multi-source setup.
TODO
- Semisync replication ('semisync_slave.so') doesn't yet work with multi-source. Will be fixed for next release.
- All open tasks and known bugs for multi-source can be found here.
- allow to replicate from one master to one slave in many threads
Incompatibilities with MariaDB/MySQL 5.5
max_relay_log_size
is now (almost) a normal variable and not automatically changed ifmax_binlog_size
is changed. To keep things compatible with old config files, we set it tomax_binlog_size
at startup if its value is 0.- You can now access replication variables that depend on the active connection with either
GLOBAL
orSESSION
. - We only write information about relay log positions for recovery if
innodb-recovery-update-relay-log
is set. Slave_retried_transaction
now shows the total count of retried transactions over all slaves.- The status variable
Com_slave_start
is replaced withCom_start_slave
. - The status variable
Com_slave_stop
is replaced withCom_stop_slave
. FLUSH RELAY LOGS
are not replicated anymore. This is not safe as connection names may be different on the slave.
See also:
- The work in MariaDB is based on the project description at MDEV-253.
- The original code base comes from Taobao, developed by Peng Lixun. A big thanks to them for this important feature!