Importing large amount of data to database

Hello,
What version of MySQL are you running?

Have you set any specific configuration options in my.cnf? I posted some recommendations here – https://wpbenchmark.io/improve-wordpress-speed/ – I follow them myself.

Hi Anton–

I also have this issue and have mostly configured things per your recommendations.

I am using MySQL 5.7. Appreciate any insight!

A bit unsure about MySQL 5.7, but I have just discovered, that the following settings help a lot for MySQL 5.8.30+

# especially for MySQL 8.0
innodb_redo_log_capacity=512M # only if you have MySQL 8.0.30+
innodb_log_buffer_size=512M
innodb_log_write_ahead_size=128K
disable_log_bin

Can you try adding them to your my.cnf? Especially the “disable_log_bin”. The first parameter might not work for you.

Hello, sorry I didn′t see email for reply. My settings:

[client]
#password	= your_password
port		= 3306
socket		= /tmp/mysql.sock

[mysqld]
port		= 3306
socket		= /tmp/mysql.sock
datadir = /www/server/data
default_storage_engine = InnoDB
performance_schema_max_table_instances = 400
table_definition_cache = 400
skip-external-locking
key_buffer_size = 128M
max_allowed_packet = 100G
table_open_cache = 512
sort_buffer_size = 2M
net_buffer_length = 4K
read_buffer_size = 2M
read_rnd_buffer_size = 256K
myisam_sort_buffer_size = 32M
thread_cache_size = 64
query_cache_size = 64M
tmp_table_size = 64M
sql-mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES

#skip-name-resolve
max_connections = 500
max_connect_errors = 100
open_files_limit = 65535

#log-bin=mysql-bin
#binlog_format=mixed
server-id = 1
expire_logs_days = 10
slow_query_log=1
slow-query-log-file=/www/server/data/mysql-slow.log
long_query_time=3
#log_queries_not_using_indexes=on

innodb_data_home_dir = /www/server/data
innodb_data_file_path = ibdata1:10M:autoextend
innodb_log_group_home_dir = /www/server/data
innodb_buffer_pool_size = 512M
innodb_log_file_size = 256M
innodb_log_buffer_size = 64M
innodb_flush_log_at_trx_commit = 1
innodb_lock_wait_timeout = 50
innodb_max_dirty_pages_pct = 90
innodb_read_io_threads = 4
innodb_write_io_threads = 4

[mysqldump]
quick
max_allowed_packet = 500M

[mysql]
no-auto-rehash

[myisamchk]
key_buffer_size = 128M
sort_buffer_size = 2M
read_buffer = 2M
write_buffer = 2M

[mysqlhotcopy]
interactive-timeout

—————-
Today I test you configs.

The following settings can slow things. They have their meaning, but speed will be the price for it.

innodb_flush_log_at_trx_commit = 1

I would increase these, but personally have not seen huge impact

innodb_read_io_threads = 4
innodb_write_io_threads = 4
thread_cache_size = 64

If you have enough memory, increase this:
innodb_buffer_pool_size = 512M

innodb_flush_log_at_trx_commit = 1

This change I read, but not risk:
https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_flush_log_at_trx_commit