Step 9: Import Data
Contents
Overview
This page details step 9 of the 9-step procedure "Deploy ColumnStore Object Storage Topology".
This step bulk imports data to Enterprise ColumnStore.
Interactive commands are detailed. Alternatively, the described operations can be performed using automation.
Import the Schema
Before data can be imported into the tables, create a matching schema.
On the primary server, create the schema:
1. For each database that you are importing, create the database with the CREATE DATABASE statement:
CREATE DATABASE inventory;
2. For each table that you are importing, create the table with the CREATE TABLE statement:
CREATE TABLE inventory.products ( product_name varchar(11) NOT NULL DEFAULT '', supplier varchar(128) NOT NULL DEFAULT '', quantity varchar(128) NOT NULL DEFAULT '', unit_cost varchar(128) NOT NULL DEFAULT '' ) ENGINE=Columnstore DEFAULT CHARSET=utf8;
Import the Data
Enterprise ColumnStore supports multiple methods to import data into ColumnStore tables.
Interface | Method | Benefits |
---|---|---|
Shell | cpimport | SQL access is not required |
SQL | LOAD DATA INFILE | Shell access is not required |
Remote Database | Remote Database Import | • Use normal database client • Avoid dumping data to intermediate filed |
cpimport
MariaDB Enterprise ColumnStore includes cpimport, which is a command-line utility designed to efficiently load data in bulk. Alternative methods are available.
To import your data from a TSV (tab-separated values) file, on the primary server run cpimport:
$ sudo cpimport -s '\t' inventory products /tmp/inventory-products.tsv
LOAD DATA INFILE
When data is loaded with the LOAD DATA INFILE statement, MariaDB Enterprise ColumnStore loads the data using cpimport, which is a command-line utility designed to efficiently load data in bulk. Alternative methods are available.
To import your data from a TSV (tab-separated values) file, on the primary server use LOAD DATA INFILE statement:
LOAD DATA INFILE '/tmp/inventory-products.tsv' INTO TABLE inventory.products;
Import from Remote Database
MariaDB Enterprise ColumnStore can also import data directly from a remote database. A simple method is to query the table using the SELECT statement, and then pipe the results into cpimport, which is a command-line utility that is designed to efficiently load data in bulk. Alternative methods are available.
To import your data from a remote MariaDB database:
$ mariadb --quick \ --skip-column-names \ --execute="SELECT * FROM inventory.products" \ | cpimport -s '\t' inventory products
Next Step
Navigation in the procedure "Deploy ColumnStore Object Storage Topology":
This page was step 9 of 9.
This procedure is complete.