- 对比:使用
pandas
存储数据 VS 使用写文本 方式存储数据
import pandas as pd
import time
def pandasWrite():t0 = time.time()colname = [str(i) for i in range(550)]df = pd.DataFrame(columns=colname)for i in range(100):df.loc[len(df)] = dict(zip(colname, range(550)))t1 = time.time()df.to_csv("temp.csv")print("pandas 存储数据用时:", t1-t0)# print(df)def fileWrite():t0 = time.time()colname = [str(i) for i in range(550)]with open("temp1.txt", 'w', encoding='utf-8') as f:f.write('\t'.join(x for x in colname))for i in range(100):f.write('\t'.join(str(x) for x in range(550))+'\n')t1 = time.time()print("写文本 存储数据用时:", t1-t0)pandasWrite()
fileWrite()
输出:
pandas 存储数据用时: 4.545027494430542
写文本 存储数据用时: 0.03499293327331543
写文本方式,快了 接近 130 倍
工作当中踩过的坑,浪费了大半天时间,大家注意!