Spider Storage Engine Introduction
Contents
Overview
MariaDB Enterprise Spider is a storage engine for MariaDB Enterprise Server that supports table federation, table sharding, and ODBC data sources.
MariaDB Enterprise Spider creates "virtual" Spider Tables on a Spider Node that clients and applications can query as if they were regular tables. For Federated Spider Tables and Sharded Spider Tables, the Enterprise Spider storage engine uses a MariaDB foreign data wrapper to query a Data Table on one or more Data Nodes. For ODBC Spider Tables, the Enterprise Spider storage engine uses an ODBC foreign data wrapper to query an ODBC Data Source.
Benefits
MariaDB Enterprise Spider:
- Supports a MariaDB foreign data wrapper. The MariaDB foreign data wrapper can be used to replace the older Federated and FederatedX storage engines.
- Supports an ODBC foreign data wrapper in MariaDB Enterprise Server 10.5 and later. ODBC foreign data wrapper is beta maturity.
The maturity can be confirmed by querying the information_schema.SPIDER_WRAPPER_PROTOCOLS table.
ES Versions | Spider Version |
---|---|
ES 10.5 | MariaDB Enterprise Spider 3.4 |
Available Platforms
- AlmaLinux 8 (x86_64, ARM64)
- AlmaLinux 9 (x86_64, ARM64)
- CentOS Linux 7 (x86_64)
- CentOS Linux 8 (x86_64, ARM64)
- Debian 9 (x86_64, ARM64)
- Debian 10 (x86_64, ARM64)
- Debian 11 (x86_64, ARM64)
- Debian 12 (x86_64, ARM64)
- Microsoft Windows (x86_64)
- Red Hat Enterprise Linux 7 (x86_64)
- Red Hat Enterprise Linux 8 (x86_64, ARM64)
- Red Hat UBI 8 (x86_64, ARM64)
- Rocky Linux 8 (x86_64, packages, Red, Linux, Hat, Enterprise, ARM64, 8)
- SUSE Linux Enterprise Server 12 (x86_64, ARM64)
- SUSE Linux Enterprise Server 15 (x86_64, ARM64)
- Ubuntu 18.04 LTS (x86_64, ARM64)
- Ubuntu 20.04 LTS (x86_64, ARM64)
- Ubuntu 22.04 LTS (x86_64, ARM64)
- Ubuntu 24.04 LTS (x86_64, ARM64)
Topologies
MariaDB Enterprise Spider supports multiple topologies. Several options are described below. MariaDB Enterprise Spider can be deployed in other topologies. The topologies on this page are representative of basic product capabilities.
For additional information, choose an Enterprise Spider topology:
Topology | Diagram | Use Cases |
---|---|---|
Spider Federated | Read from and write to tables on remote ES nodes Migrate tables from remote ES node Query tables on remote ES node JOIN local tables with tables on remote ES node | |
Spider Sharded | Shard tables for horizontal scalability Consolidate tables from remote ES nodes into one "virtual" table Partition a large table across multiple remote ES nodes Use regular partitioning syntax to define shards | |
Spider ODBC | Read from and write to tables on ODBC Data Sources Migrate data from an ODBC Data Source Query data from an ODBC Data Source JOIN local tables with data from an ODBC Data Source ES 10.5+ |
Deployment Instructions
To deploy the Enterprise Spider storage engine with MariaDB Enterprise Server, choose an Enterprise Spider topology and an ES version:
Topology | Diagram | Use Cases |
---|---|---|
Enterprise Server 10.5 Enterprise Server 10.6 Enterprise Server 11.4 | *Read from and write to tables on remote ES nodes* Spider Node uses Spider storage engine for Federated Spider Tables Federated Spider Table is a "virtual" table Spider uses MariaDB foreign data wrapper to query Data Table on Data Node Data Node uses non-Spider storage engine for Data Tables Supports transactions, Enterprise Spider | |
Enterprise Server 10.5 Enterprise Server 10.6 Enterprise Server 11.4 | *Shard tables for horizontal scalability* Spider Node uses Spider storage engine for Sharded Spider Tables Sharded Spider Table is a partitioned "virtual" table Spider uses MariaDB foreign data wrapper to query Data Tables on Data Nodes for each partition Data Node uses non-Spider storage engine for Data Tables Supports transactions Enterprise Spider |
Term Definitions
Term | Definition |
---|---|
Data Node | A Data Node is a MariaDB Enterprise Server node that contains one or more Data Tables. |
Data Table | A Data Table stores data for a Spider Table. When a Spider Table is queried, the Enterprise Spider storage engine uses the MariaDB foreign data wrapper to read from and write to the Data Table on a Data Node. The Data Table must be created on the Data Node with the same structure as the Spider Table. The Data Table must use a non-Spider storage engine, such as InnoDB or ColumnStore. |
ODBC Data Source | An ODBC Data Source relies on an ODBC Driver and an ODBC Driver Manager to query an external data source. |
ODBC Driver | An ODBC Driver is a library that integrates with a ODBC Driver Manager to query an external data source. |
ODBC Driver Manager | An ODBC Driver Manager allows applications to use ODBC Drivers. |
Spider Node | A Spider Node is a MariaDB Enterprise Server node that contains one or more Spider Tables. |
Spider Table | A Spider Table is a virtual table that does not store data. When a Spider Table is queried, the Enterprise Spider storage engine uses foreign data wrappers to read from and write to Data Tables on Data Nodes or ODBC Data Sources. |
Resources
ES Information Schema Reference
ES Plugin Reference
ES SQL Statement Reference
ES System Table Reference
- spider_link_failed_log
- spider_link_mon_servers
- spider_table_crd
- spider_table_position_for_recovery
- spider_tables
- spider_table_sts
- spider_xa_failed_log
- spider_xa_member
- spider_xa
-
MariaDB Enterprise Spider Operations
Overview MariaDB Enterprise Spider supports different operations for diffe... -
MariaDB Enterprise Spider Schema Design
Create Tables Spider Tables can be created with MariaDB Enterprise Spider ...