The Ultimate Guide to High Availability with MariaDB
This is a read-only copy of the MariaDB Knowledgebase generated on 2024-11-17. For the latest, interactive version please visit https://mariadb.com/kb/.

cpimport error

Hi, I had an error while using cpimport command. I configured separate modules with 2 PMs and 1 UM. The type of data storage is external. I executed cpimport command on PM #1(parent OAM). Load file(test_1) contains 10,000,000 rows data.

Error messages are belows:

[root@cm1 ~]# cpimport input_test mf2ae_log test_1 
2017-03-31 02:58:17 (11610) INFO : Running distributed import (mode 1) on all PMs...
2017-03-31 02:59:07 (11610) ERR  : Received a Cpimport Failure from PM1
2017-03-31 02:59:07 (11610) INFO : Please verify error log files in PM1
2017-03-31 02:59:07 (11610) INFO : Canceling outstanding cpimports
2017-03-31 03:00:12 (11610) INFO : Table input_test.mf2ae_log: (OID-3000) was NOT successfully loaded.
2017-03-31 03:00:12 (11610) INFO : Bulk load completed, total run time : 115.004 seconds

Log messages(err.log, debug.log):

[root@cm1 ~]# cat /var/log/mariadb/columnstore/err.log 
Mar 31 02:59:04 cm1 writeengineserver[3499]: 04.404181 |0|0|0| E 32 CAL0000: pushing data : PIPE error .........Broken pipe         
Mar 31 02:59:07 cm1 writeengineserver[3499]: 07.440615 |0|0|0| E 32 CAL0000: 1905 : cpimport exit on failure (signal 9)         
Mar 31 02:59:07 cm1 writeenginesplit[11610]: 07.440784 |0|0|0| E 33 CAL0000: #033[0;31mReceived a Cpimport Failure from PM1#033[0m         
Mar 31 02:59:07 cm1 writeenginesplit[11610]: 07.491808 |0|0|0| E 33 CAL0087: BulkLoad Error: #033[0;31mReceived a Cpimport Failure from PM1#033[0m

/var/log/mariadb/columnstore/debug.log

Mar 31 02:58:17 cm1 writeengineserver[3499]: 17.153209 |0|0|0| D 32 CAL0000: 1905 : onReceiveMode() Setting fMode = 1         
Mar 31 02:58:17 cm1 writeengineserver[3499]: 17.154965 |0|0|0| D 32 CAL0000: 1905 : onReceiveMode() DbRoot Count = 1         
Mar 31 02:58:17 cm1 writeengineserver[3499]: 17.155080 |0|0|0| D 32 CAL0000: 1905 : CMD LINE ARGS came in /usr/local/mariadb/columnstore/bin/cpimport.bin -R /tmp/columnstore_tmp_files/BrmRpt023102581711610.rpt -m 1 -P pm1-11610 -u6a3eae55-54ee-4b59-9114-512e37f363bf input_test mf2ae_log         
Mar 31 02:58:17 cm1 writeengineserver[3499]: 17.157575 |0|0|0| D 32 CAL0000: 1905 : Brm Rpt Filename Arrived /tmp/columnstore_tmp_files/BrmRpt023102581711610.rpt         
Mar 31 02:58:17 cm1 writeengineserver[3499]: 17.157896 |0|0|0| D 32 CAL0000: 1905 : Start Cpimport command reached!!         
Mar 31 02:59:07 cm1 writeengineserver[3499]: 07.324109 |0|0|0| D 32 CAL0000: 1905 : onReceiveEOD : child ID = 11644         
Mar 31 02:59:07 cm1 writeengineserver[3499]: 07.324183 |0|0|0| D 32 CAL0000: 1905 : Message Queue is empty; Stopping CF Thread         
Mar 31 02:59:07 cm1 writeengineserver[3499]: 07.465547 |0|0|0| D 32 CAL0000: 1905 : onCpimportFailure BrmReport Send         
Mar 31 02:59:07 cm1 writeengineserver[3499]: 07.465674 |0|0|0| D 32 CAL0000: 1905 : onReceiveEOD : child ID = 0         
Mar 31 02:59:07 cm1 writeengineserver[3499]: 07.465759 |0|0|0| D 32 CAL0000: 1905 : onReceiveEOD : child ID = 0         
Mar 31 03:00:09 cm1 writeengineserver[3499]: 09.655294 |0|0|0| D 32 CAL0000: 1905 : OnReceiveRollback arrived         
Mar 31 03:00:11 cm1 writeengineserver[3499]: 11.998767 |0|0|0| D 32 CAL0000: 1905 : OnReceiveCleanup arrived

Could you let me know why this error occured? Thank you!

Answer Answered by David Thompson in this comment.

More complete logs from all nodes would help. If you can reproduce this still can you file a jira with the output of columnStoreSupport: https://mariadb.com/kb/en/mariadb/system-troubleshooting-mariadb-columnstore/#mariadb-columnstore-support-tool

I'd take a guess that pm2 is somehow crashing causing the broken pipe. If you have very low memory and the import is large that could be the reason.

Content reproduced on this site is the property of its respective owners, and this content is not reviewed in advance by MariaDB. The views, information and opinions expressed by this content do not necessarily represent those of MariaDB or any other party.