今日接到一个小需求，一张表有类似这样的数据：（下面是一行）

但是每行的数据，主要是key不一样，我们想把所有的key取出来，并且做个去重。

首先我先在mysql中，

SELECT GROUP_CONCAT(REPLACE(REPLACE(JSON_KEYS(字段名),'[',''),']',''),',') FROM `表名` 

取出每一行的key并拼在一行中，然后就用java把他们都切割并且去重。这里用java是考虑到mysql切割成数组以后，有大量数据的话，难以取到每个值去做处理。

java代码如下：

import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashSet;
import java.io.FileOutputStream;
import org.apache.poi.ss.usermodel.*;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;public class TextFileProcessor {public static void main(String[] args) {try {// 读取文本文件String filePath = "xxxx.txt";String content = readTextFile(filePath).replace("\"","");// 切割文本为数组String[] wordsArray = content.split(",");// 去除重复元素String[] uniqueWordsArray = removeDuplicates(wordsArray);// 打印结果System.out.println("Original Array: " + Arrays.toString(wordsArray));String excelFilePath = "output.xlsx";writeArrayToExcel(uniqueWordsArray, excelFilePath);} catch (IOException e) {e.printStackTrace();}}private static String readTextFile(String filePath) throws IOException {StringBuilder content = new StringBuilder();try (BufferedReader reader = new BufferedReader(new FileReader(filePath))) {String line;while ((line = reader.readLine()) != null) {content.append(line).append("\n");}}return content.toString();}private static String[] removeDuplicates(String[] array) {// 使用HashSet去除重复元素HashSet<String> uniqueSet = new HashSet<>(Arrays.asList(array));// 转换回数组return uniqueSet.toArray(new String[0]);}private static void writeArrayToExcel(String[] array, String excelFilePath) throws IOException {try (Workbook workbook = new XSSFWorkbook()) {Sheet sheet = workbook.createSheet("Array Data");// 创建行Row row = sheet.createRow(0);// 将数组中的值写入Excel的列for (int i = 0; i < array.length; i++) {Cell cell = row.createCell(i);cell.setCellValue(array[i]);}// 将Workbook写入到文件try (FileOutputStream outputStream = new FileOutputStream(excelFilePath)) {workbook.write(outputStream);}}}
}

pom.xml需要加一个：

<dependency><groupId>org.apache.poi</groupId><artifactId>poi</artifactId><version>4.1.2</version> <!-- Replace with the latest version -->
</dependency>
<dependency><groupId>org.apache.poi</groupId><artifactId>poi-ooxml</artifactId><version>4.1.2</version> <!-- Replace with the latest version -->
</dependency>