ColumnStore Monitor
ColumnStore Monitor
The ColumnStore monitor, csmon
, is a monitor module for MariaDB ColumnStore
servers. It supports multiple UM nodes and can detect the correct server for
DML/DDL statements which will be labeled as the master. Other UM nodes will be
used for reads.
Required Grants
The credentials defined with the user
and password
parameters must have all
grants on the infinidb_vtable
database.
For example, to create a user for this monitor with the required grants execute the following SQL.
CREATE USER 'maxscale'@'maxscalehost' IDENTIFIED BY 'maxscale-password'; GRANT ALL ON infinidb_vtable.* TO 'maxscale'@'maxscalehost';
Master Selection
The ColumnStore Monitor in MaxScale 2.5 supports ColumnStore 1.0, 1.2 and 1.5, and the master selection is done differently for each version.
- If the version is 1.0, the master server must be specified using the
primary
parameter. - If the version is 1.2, the master server is selected automatically using
the ColumnStore function
mcsSystemPrimary()
. - If the version is 1.5, the master server is selected automatically by querying the ColumnStore daemon running on each node.
Configuration
Read the Monitor Common document for a list of supported common monitor parameters.
version
With this mandatory parameter the used ColumnStore version is specified.
The allowed values are 1.0
, 1.2
and 1.5
.
primary
Required and only allowed when the value of version
is 1.0
.
The primary
parameter controls which server is chosen as the master
server.
If the server pointed to by this parameter is available and is ready to process queries, it receives the Master status.
admin_port
Allowed only when the value of version is 1.5
.
This optional parameter specifies the port of the ColumnStore administrative
daemon. The default value is 8640
. Note that the daemons of all nodes must
be listening on the same port.
admin_base_path
Allowed only when the value of version is 1.5
.
This optional parameter specifies the base path of the ColumnStore
administrative daemon. The default value is /cmapi/0.4.0
.
api_key
Allowed only when the value of version is 1.5
.
This optional parameter specifies the API key to be used in the
communication with the ColumnStore administrative daemon. If no
key is specified, then a key will be generated and stored to the
file api_key.txt
in the directory with the same name as the
monitor in data directory of MaxScale. Typically that will
be /var/lib/maxscale/<monitor-section>/api_key.txt
.
Note that ColumnStore will store the first key provided and thereafter require it, so changing the key requires the resetting of the key on the ColumnStore nodes as well.
local_address
Allowed only when the value of version is 1.5
.
With this parameter it is specified what IP MaxScale should
tell the ColumnStore nodes it resides at. Either it or
local_address
at the global level in the MaxScale
configuration file must be specified. If both have been
specified, then the one specified for the monitor overrides.
Commands
The ColumnStore monitor provides module commands using which the ColumnStore cluster can be managed. The commands can be invoked using the REST-API with a client such as curl or using maxctrl.
All commands require the monitor instance name as the first parameters. Additional parameters must be provided depending on the command.
Note that as maxctrl itself has a timeout of 10 seconds, if a timeout larger than that is provided to any command, the timeout of maxctrl must also be increased. For instance:
maxctrl --timeout 30s call command csmon shutdown CsMonitor 20s
Here a 30 second timeout is specified for maxctrl to ensure that it does not expire before the timeout of 20s provided for the shutdown command possibly does.
The output is always a JSON object.
In the following, assume a configuration like this:
[CsNode1] type=server ... [CsNode2] type=server ... [CsMonitor] type=monitor module=csmon servers=CsNode1,CsNode2 ...
start
Starts the ColumnStore cluster.
call command csmon start <monitor-name> <timeout>
Example
call command csmon start CsMonitor 20s
shutdown
Shuts down the ColumnStore cluster.
call command csmon shutdown <monitor-name> <timeout>
Example
call command csmon shutdown CsMonitor 20s
status
Get the status of the ColumnStore cluster.
call command csmon status <monitor-name> [<server>]
Returns the status of the cluster or the status of a specific server.
Example
call command csmon status CsMonitor call command csmon status CsMonitor CsNode1
mode-set
Sets the mode of the cluster.
call command csmon mode-set <monitor-name> (readonly|readwrite) <timeout>
Example
call command csmon mode-set CsMonitor readonly 20s
config-get
Returns the cluster configuration.
call command csmon config-get <monitor-name> [<server-name>]
If no server is specified, the configuration is fetched from the first server in the monitor configuration, otherwise from the specified server.
Note that if everything is in order, the returned configuration should be identical regardless of the server it is fetched from.
Example
call command csmon config-get CsMonitor CsNode2
add-node
Adds a new node located on the server at the hostname or IP host to the ColumnStore cluster.
call command csmon add-node <monitor-name> <host> <timeout>
Example
call command csmon add-node CsMonitor mcs2 20s
For a more complete example, please refer to adding a node.
remove-node
Remove the node located on the server at the hostname or IP host from the ColumnStore cluster.
call command csmon remove-node <monitor-name> <host> <timeout>
Example
call command csmon remove-node CsMonitor mcs2 20s
For a more complete example, please refer to removing a node.
Example
The following is an example of a csmon
configuration.
[CSMonitor] type=monitor module=csmon version=1.5 servers=CsNode1,CsNode2 user=myuser password=mypwd monitor_interval=5000 api_key=somekey1234
Adding a Node
Adding a new node to a ColumnStore cluster can be performed dynamically at runtime, but it must be done in two steps. First, the node is added to ColumnStore and then, the corresponding server object (that possibly has to be created) in the MaxScale configuration is added to the ColumnStore monitor.
In the following, assume a two node ColumnStore cluster and an initial MaxScale configuration like.
[CsNode1] type=server ... [CsNode2] type=server ... [CsMonitor] type=monitor module=csmon servers=CsNode1,CsNode2 ...
Invoking maxctrl list servers
will now show:
┌─────────┬─────────────┬──────┬─────────────┬─────────────────┬──────┐ │ Server │ Address │ Port │ Connections │ State │ GTID │ ├─────────┼─────────────┼──────┼─────────────┼─────────────────┼──────┤ │ CsNode1 │ 10.10.10.10 │ 3306 │ 0 │ Master, Running │ │ ├─────────┼─────────────┼──────┼─────────────┼─────────────────┼──────┤ │ CsNode2 │ 10.10.10.11 │ 3306 │ 0 │ Slave, Running │ │ └─────────┴─────────────┴──────┴─────────────┴─────────────────┴──────┘
If we now want to add a new ColumnStore node, located at mcs3/10.10.10.12
to the cluster, the steps are as follows.
First the node is added
maxctrl --timeout 30s call command csmon add-node CsMonitor mcs3 20s
After a while the following is output:
{ "links": { "self": "http://localhost:8989/v1/maxscale/modules/csmon/add-node" }, "meta": { "message": "Node mcs3 successfully added to cluster.", "result": { "node_id": "mcs3", "timestamp": "2020-08-07 10:03:49.474539" }, "success": true } }
At this point, the ColumnStore cluster consists of three nodes. However, the ColumnStore monitor is not yet aware of the new node.
First we need to create the corresponding server object.
maxctrl create server CsNode3 10.10.10.12
Invoking maxctrl list servers
will now show:
┌─────────┬─────────────┬──────┬─────────────┬─────────────────┬──────┐ │ Server │ Address │ Port │ Connections │ State │ GTID │ ├─────────┼─────────────┼──────┼─────────────┼─────────────────┼──────┤ │ CsNode3 │ 10.10.10.12 │ 3306 │ 0 │ Down │ │ ├─────────┼─────────────┼──────┼─────────────┼─────────────────┼──────┤ │ CsNode1 │ 10.10.10.10 │ 3306 │ 0 │ Master, Running │ │ ├─────────┼─────────────┼──────┼─────────────┼─────────────────┼──────┤ │ CsNode2 │ 10.10.10.11 │ 3306 │ 0 │ Slave, Running │ │ └─────────┴─────────────┴──────┴─────────────┴─────────────────┴──────┘
The server CsNode3
has been created, but its state is Down
since
it is not yet being monitored.
┌───────────┬─────────┬──────────────────┐ │ Monitor │ State │ Servers │ ├───────────┼─────────┼──────────────────┤ │ CsMonitor │ Running │ CsNode1, CsNode2 │ └───────────┴─────────┴──────────────────┘
It must now be added to the monitor.
maxctrl link monitor CsMonitor CsNode3
Now the server is monitored and maxctrl list monitors
shows:
┌───────────┬─────────┬───────────────────────────┐ │ Monitor │ State │ Servers │ ├───────────┼─────────┼───────────────────────────┤ │ CsMonitor │ Running │ CsNode1, CsNode2, CsNode3 │ └───────────┴─────────┴───────────────────────────┘
The state of the new node is now also set correctly, as shown by
maxctrl list servers
.
┌─────────┬─────────────┬──────┬─────────────┬─────────────────┬──────┐ │ Server │ Address │ Port │ Connections │ State │ GTID │ ├─────────┼─────────────┼──────┼─────────────┼─────────────────┼──────┤ │ CsNode3 │ 10.10.10.12 │ 3306 │ 0 │ Slave, Running │ │ ├─────────┼─────────────┼──────┼─────────────┼─────────────────┼──────┤ │ CsNode1 │ 10.10.10.10 │ 3306 │ 0 │ Master, Running │ │ ├─────────┼─────────────┼──────┼─────────────┼─────────────────┼──────┤ │ CsNode2 │ 10.10.10.11 │ 3306 │ 0 │ Slave, Running │ │ └─────────┴─────────────┴──────┴─────────────┴─────────────────┴──────┘
Note that the MaxScale server object can be created at any point, but
it must not be added to the monitor before the node has been added to
the ColumnStore cluster using call command csmon add-node
.
Removing a Node
Removing a node should be performed in the reverse order of how a node was added. First, the MaxScale server should be removed from the monitor. Then, the node should be removed from the ColumnStore cluster.
Suppose we want to remove the ColumnStore node at mcs2/10.10.10.12
and the current situation is as:
┌───────────┬─────────┬───────────────────────────┐ │ Monitor │ State │ Servers │ ├───────────┼─────────┼───────────────────────────┤ │ CsMonitor │ Running │ CsNode1, CsNode2, CsNode3 │ └───────────┴─────────┴───────────────────────────┘
First, the server is removed from the monitor.
maxctrl unlink monitor CsMonitor CsNode3
Checking with maxctrl list monitors
we see that the server has
indeed been removed.
┌───────────┬─────────┬──────────────────┐ │ Monitor │ State │ Servers │ ├───────────┼─────────┼──────────────────┤ │ CsMonitor │ Running │ CsNode1, CsNode2 │ └───────────┴─────────┴──────────────────┘
Now the node can be removed from the cluster itself.
maxctrl --timeout 30s call command csmon remove-node CsMonitor mcs3 20s { "links": { "self": "http://localhost:8989/v1/maxscale/modules/csmon/remove-node" }, "meta": { "message": "Node mcs3 removed from the cluster.", "result": { "node_id": "mcs3", "timestamp": "2020-08-07 11:41:36.573425" }, "success": true } }