MySQL导入数据时内存溢出是指在将大量数据导入MySQL数据库时,由于内存不足导致系统崩溃或导入失败的现象。MySQL在导入数据时会将数据加载到内存中进行处理,如果数据量过大,内存可能无法承受,从而导致内存溢出。
innodb_buffer_pool_size
,以适应导入的数据量。以下是一个分批导入数据的示例代码:
import mysql.connector
def import_data_in_batches(file_path, batch_size=1000):
conn = mysql.connector.connect(user='user', password='password', host='host', database='database')
cursor = conn.cursor()
with open(file_path, 'r') as file:
batch = []
for line in file:
batch.append(tuple(line.strip().split(',')))
if len(batch) >= batch_size:
insert_query = "INSERT INTO table_name (column1, column2, column3) VALUES (%s, %s, %s)"
cursor.executemany(insert_query, batch)
conn.commit()
batch = []
if batch:
insert_query = "INSERT INTO table_name (column1, column2, column3) VALUES (%s, %s, %s)"
cursor.executemany(insert_query, batch)
conn.commit()
cursor.close()
conn.close()
import_data_in_batches('data.csv')
通过以上方法可以有效解决MySQL导入数据时内存溢出的问题。
领取专属 10元无门槛券
手把手带您无忧上云