问题情境:
项目中存在web页面点击一键导出,导出所有数据对应的pdf文件,由于有些pdf文件是实时生成的,之前最简答的写法for循环处理速度太慢,超过了nginx配置的最大响应时间了,且对用户交互体验上很不友好,所以进一步进行优化。
解决方案:
1.对大量的数据进行分段处理;(向上取余)
代码示例:
public List<List<FrApplyGuaranteeInfo>> segmentList(List<FrApplyGuaranteeInfo> list ,int segmentSize) {int limit = (list.size() + segmentSize - 1) / segmentSize;List<List<FrApplyGuaranteeInfo>> segmentList =Stream.iterate(0, n -> n + 1).limit(limit).parallel().map(a -> list.stream().skip(a * segmentSize).limit(segmentSize).parallel().collect(Collectors.toList())).collect(Collectors.toList());return segmentList;}
2.通过threadPoolTaskExcetor.submit()方法进行多线程任务处理
示例代码:
第一步分段处理后的list
lists.forEach({list->{threadPoolExecutor.submit(new xxxxTask());
})
第二步:具体的task需要实现callalbe
示例代码如下:
@Slf4j
public class DownloadLetterZipTask implements Callable {// 需要预审的记录 private List<FrApplyGuaranteeInfo> dataList;private ZipOutputStream zipOutputStream;private CountDownLatch countDownLatch;private IFrApplyGuaranteeInfoService applyGuaranteeInfoService;public DownloadLetterZipTask(List<FrApplyGuaranteeInfo> dataList, ZipOutputStream zipOutputStream, CountDownLatch countDownLatch, IFrApplyGuaranteeInfoService applyGuaranteeInfoService){ this.dataList = dataList;this.zipOutputStream = zipOutputStream; this.countDownLatch = countDownLatch; this.applyGuaranteeInfoService = applyGuaranteeInfoService; }@Overridepublic Object call() throws Exception {log.info("start--------------" + Thread.currentThread().getName()); try { for (FrApplyGuaranteeInfo applyGuaranteeInfo : dataList) {if (!StringUtils.isEmpty(applyGuaranteeInfo.getAcceptNo())) {try {// 这一步为具体的将文件转为字节数组输出流ByteArrayOutputStream waterOutputStream = applyGuaranteeInfoService.getLetterPdfByteStream(applyGuaranteeInfo.getAcceptNo());byte[] xmpMetadata = waterOutputStream.toByteArray();synchronized (zipOutputStream) {zipOutputStream.putNextEntry(new ZipEntry(applyGuaranteeInfo.getGenerateeLetterNo() + ".pdf"));zipOutputStream.write(xmpMetadata);zipOutputStream.closeEntry();}} catch (Exception e) {log.error("[一键导出]---acceptNo为{}生成pdf失败", applyGuaranteeInfo.getAcceptNo());}}}} catch (Exception e) {e.printStackTrace();log.error("[xxxx]-批量下载zip失败");} finally {countDownLatch.countDown();}return null;}}