spark 写入mysql 中文数据 显示?? 或者 乱码

目录

前言

Spark报错：

解决办法：

总结一下：

报错：

解决：

前言

用spark写入mysql中，查看中文数据 显示?? 或者 乱码

Spark报错：

Sat Nov 23 19:15:59 CST 2024 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Sat Nov 23 19:16:02 CST 2024 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Sat Nov 23 19:16:02 CST 2024 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2024-11-23 19:16:02,903{yy/MM/dd java.sql.BatchUpdateException: Incorrect string value: '\xE5\xBE\x85\xE6\x9C\xBA' for column 'ChangeRecordState' at row 1at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)at java.lang.reflect.Constructor.newInstance(Constructor.java:423)at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)at com.mysql.jdbc.Util.getInstance(Util.java:408)at com.mysql.jdbc.SQLError.createBatchUpdateException(SQLError.java:1163)at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1823)at com.mysql.jdbc.PreparedStatement.executeBatchInternal(PreparedStatement.java:1307)at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:970)at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.savePartition(JdbcUtils.scala:692)at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.$anonfun$saveTable$1(JdbcUtils.scala:856)at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.$anonfun$saveTable$1$adapted(JdbcUtils.scala:854)at org.apache.spark.rdd.RDD.$anonfun$foreachPartition$2(RDD.scala:1020)at org.apache.spark.rdd.RDD.$anonfun$foreachPartition$2$adapted(RDD.scala:1020)at org.apache.spark.SparkContext.$anonfun$runJob$5(SparkContext.scala:2242)at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)at org.apache.spark.scheduler.Task.run(Task.scala:131)at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:497)at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1439)at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:500)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.SQLException: Incorrect string value: '\xE5\xBE\x85\xE6\x9C\xBA' for column 'ChangeRecordState' at row 1at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:965)at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3978)at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3914)at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2530)at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2683)at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2495)at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1903)at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2124)at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1801)... 16 more
2024-11-23 19:16:02,921{yy/MM/dd Sat Nov 23 19:16:02 CST 2024 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 76 in stage 12.0 failed 1 times, most recent failure: Lost task 76.0 in stage 12.0 (TID 242) (LAPTOP-P70BBG66 executor driver): java.sql.BatchUpdateException: Incorrect string value: '\xE5\xBE\x85\xE6\x9C\xBA' for column 'ChangeRecordState' at row 1at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)at java.lang.reflect.Constructor.newInstance(Constructor.java:423)at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)at com.mysql.jdbc.Util.getInstance(Util.java:408)at com.mysql.jdbc.SQLError.createBatchUpdateException(SQLError.java:1163)at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1823)at com.mysql.jdbc.PreparedStatement.executeBatchInternal(PreparedStatement.java:1307)at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:970)at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.savePartition(JdbcUtils.scala:692)at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.$anonfun$saveTable$1(JdbcUtils.scala:856)at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.$anonfun$saveTable$1$adapted(JdbcUtils.scala:854)at org.apache.spark.rdd.RDD.$anonfun$foreachPartition$2(RDD.scala:1020)at org.apache.spark.rdd.RDD.$anonfun$foreachPartition$2$adapted(RDD.scala:1020)at org.apache.spark.SparkContext.$anonfun$runJob$5(SparkContext.scala:2242)at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)at org.apache.spark.scheduler.Task.run(Task.scala:131)at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:497)at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1439)at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:500)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.SQLException: Incorrect string value: '\xE5\xBE\x85\xE6\x9C\xBA' for column 'ChangeRecordState' at row 1at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:965)at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3978)at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3914)at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2530)at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2683)at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2495)at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1903)at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2124)at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1801)... 16 moreDriver stacktrace:at org.apache.spark.scheduler.DAGScheduler.failJobAndIndependentStages(DAGScheduler.scala:2253)at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2(DAGScheduler.scala:2202)at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2$adapted(DAGScheduler.scala:2201)at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:2201)at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1(DAGScheduler.scala:1078)at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1$adapted(DAGScheduler.scala:1078)at scala.Option.foreach(Option.scala:274)at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:1078)at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2440)at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2382)at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2371)at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:868)at org.apache.spark.SparkContext.runJob(SparkContext.scala:2202)at org.apache.spark.SparkContext.runJob(SparkContext.scala:2223)at org.apache.spark.SparkContext.runJob(SparkContext.scala:2242)at org.apache.spark.SparkContext.runJob(SparkContext.scala:2267)at org.apache.spark.rdd.RDD.$anonfun$foreachPartition$1(RDD.scala:1020)at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)at org.apache.spark.rdd.RDD.withScope(RDD.scala:414)at org.apache.spark.rdd.RDD.foreachPartition(RDD.scala:1018)at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.saveTable(JdbcUtils.scala:854)at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:63)at org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:46)at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)at org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:90)at org.apache.spark.sql.execution.SparkPlan.$anonfun$execute$1(SparkPlan.scala:180)at org.apache.spark.sql.execution.SparkPlan.$anonfun$executeQuery$1(SparkPlan.scala:218)at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:215)at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:176)at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:132)at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:131)at org.apache.spark.sql.DataFrameWriter.$anonfun$runCommand$1(DataFrameWriter.scala:989)at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$5(SQLExecution.scala:103)at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:163)at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:90)at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:772)at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:64)at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:989)at org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:438)at org.apache.spark.sql.DataFrameWriter.saveInternal(DataFrameWriter.scala:415)at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:301)at org.apache.spark.sql.DataFrameWriter.jdbc(DataFrameWriter.scala:817)at com._02.gy.GZ.zhibiao.Test01$.main(Test01.scala:61)at com._02.gy.GZ.zhibiao.Test01.main(Test01.scala)
Caused by: java.sql.BatchUpdateException: Incorrect string value: '\xE5\xBE\x85\xE6\x9C\xBA' for column 'ChangeRecordState' at row 1at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)at java.lang.reflect.Constructor.newInstance(Constructor.java:423)at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)at com.mysql.jdbc.Util.getInstance(Util.java:408)at com.mysql.jdbc.SQLError.createBatchUpdateException(SQLError.java:1163)at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1823)at com.mysql.jdbc.PreparedStatement.executeBatchInternal(PreparedStatement.java:1307)at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:970)at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.savePartition(JdbcUtils.scala:692)at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.$anonfun$saveTable$1(JdbcUtils.scala:856)at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.$anonfun$saveTable$1$adapted(JdbcUtils.scala:854)at org.apache.spark.rdd.RDD.$anonfun$foreachPartition$2(RDD.scala:1020)at org.apache.spark.rdd.RDD.$anonfun$foreachPartition$2$adapted(RDD.scala:1020)at org.apache.spark.SparkContext.$anonfun$runJob$5(SparkContext.scala:2242)at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)at org.apache.spark.scheduler.Task.run(Task.scala:131)at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:497)at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1439)at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:500)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.SQLException: Incorrect string value: '\xE5\xBE\x85\xE6\x9C\xBA' for column 'ChangeRecordState' at row 1at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:965)at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3978)at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3914)at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2530)at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2683)at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2495)at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1903)at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2124)at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1801)... 16 more
Sat Nov 23 19:16:03 CST 2024 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Sat Nov 23 19:16:03 CST 2024 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Sat Nov 23 19:16:03 CST 2024 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Sat Nov 23 19:16:03 CST 2024 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2024-11-23 19:16:03,282{yy/MM/dd java.lang.NullPointerExceptionat org.apache.spark.scheduler.Task.$anonfun$run$2(Task.scala:152)at org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1419)at org.apache.spark.scheduler.Task.run(Task.scala:150)at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:497)at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1439)at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:500)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)at java.lang.Thread.run(Thread.java:745)
Sat Nov 23 19:16:03 CST 2024 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2024-11-23 19:16:03,330{yy/MM/dd java.lang.NullPointerExceptionat org.apache.spark.scheduler.Task.$anonfun$run$2(Task.scala:152)at org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1419)at org.apache.spark.scheduler.Task.run(Task.scala:150)at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:497)at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1439)at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:500)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)at java.lang.Thread.run(Thread.java:745)
Sat Nov 23 19:16:03 CST 2024 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2024-11-23 19:16:03,386{yy/MM/dd java.lang.NullPointerExceptionat org.apache.spark.scheduler.Task.$anonfun$run$2(Task.scala:152)at org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1419)at org.apache.spark.scheduler.Task.run(Task.scala:150)at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:497)at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1439)at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:500)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)at java.lang.Thread.run(Thread.java:745)Process finished with exit code 1

解决办法：

我试了网上的很多办法：在saprk里面更改编码，直接把这一列更改为UTF-8

.withColumn("ChangeRecordState", decode($"ChangeRecordState", "UTF-8"))

使用新的driver，然后更改maven里面的jdbc链接版本

properties.setProperty("driver", "com.mysql.cj.jdbc.Driver")

等等....都没有解决问题

所以我开始着手自己解决，问题的核心无非就是编码冲突问题，因为已经写入进去了，所以连接配置啥的都没问题，主要解决编码问题

spark中可以打印出中文字段所以spark基本没有什么问题，我还是用了老方法

mysqlURL的配置项上加了?useUnicode=true&characterEncoding=UTF-8

val mysqlUrl = "jdbc:mysql://bigdata1:3306/shtd_industry?useUnicode=true&characterEncoding=UTF-8"

主要解决的是mysql，所以我开始查询mysql的默认编码是什么，我找到了命令

SHOW TABLE STATUS LIKE 'machine_state_time';

找到Collation字段对应的就是默认的编码  --  latin1_swedish_ci

找到了默认编码不是UTF-8,所以这里问题很明显了，就是把 latin1_swedish_ci 变成 UTF-8

所以我又接着查询，把表转换编码的命令

alter table machine_state_time character set utf8;

结果还是不行，当我试着找出问题，才发现我只把表改成了UTF-8，没有把字段改成UTF-8

alter table machine_state_time convert to character set utf8;

这样mysql的问题也解决了，运行一下还是报错

找到问题，scala语言也需要设置编码方式，所以我又加了一条配置项

System.setProperty("file.encoding", "UTF-8")

这样就可以了，运行成功，然后查看一下mysql中的内容

总结一下：

报错：

主要原因是因为编码冲突，scala，spark，mysql-connect连接，mysql三个编码不一致

解决：

设置scala的编码：

System.setProperty("file.encoding", "UTF-8")

设置spark的编码：

saprk本身就是UTF-8 不用设置

设置mysqlURL的编码：

val mysqlUrl = "jdbc:mysql://bigdata1:3306/shtd_industry?useUnicode=true&characterEncoding=UTF-8"

设置mysql本身的编码：

用两条命令，分别是：

//修改目标表所有字段编码格式为utf8
alter table machine_state_time convert to character set utf8;
//修改目标表表编码格式为utf8
alter table machine_state_time character set utf8;

好的，今天的分享就到这里了，希望对你有帮助，有帮助可以点点赞，评论评论

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：http://www.mzph.cn/bicheng/61762.shtml

如若内容造成侵权/违法违规/事实不符，请联系多彩编程网进行投诉反馈email:809451989@qq.com，一经查实，立即删除！

相关文章

电子应用设计方案-20：智能电冰箱系统方案设计

电子应用设计方案-20：智能电冰箱系统方案设计

智能电冰箱系统方案设计 一、系统概述 本智能电冰箱系统旨在提供更便捷、高效、智能化的食品存储和管理解决方案，通过集成多种传感器、智能控制技术和联网功能，实现对冰箱内部环境的精确监测和控制，以及与用户的互动和远程管理。 二、系统组成…
阅读更多...
实验四：构建园区网（OSPF 动态路由）

实验四：构建园区网（OSPF 动态路由）

目录 一、实验简介 二、实验目的 三、实验需求 四、实验拓扑 五、实验步骤 1、在 eNSP 中部署网络 2、设计全网 IP 地址 3、配置二层交换机 4、配置路由交换机并测试通信 5、配置路由接口地址 6、配置 OSPF 动态路由，实现全网互通 一、实验简介 使用路由…
阅读更多...
【大数据学习 | Spark-Core】yarn-client与yarn-cluster的区别

【大数据学习 | Spark-Core】yarn-client与yarn-cluster的区别

1. yarn的提交命令 # yarn的提交命令参数 --master yarn #执行集群 --deploy-mode # 部署模式 --class #指定运行的类 --executor-memory #指定executor的内存 --executor-cores # 指定核数 --num-executors # 直接指定executor的数量 --queue # 指定队列 2. yarn-client模式…
阅读更多...
WEB攻防-通用漏洞文件上传二次渲染.htaccess变异免杀

WEB攻防-通用漏洞文件上传二次渲染.htaccess变异免杀

知识点： 1、文件上传-二次渲染 2、文件上传-简单免杀变异 3、文件上传-.htaccess妙用 4、文件上传-PHP语言特性 1、上传后门时，文件内容带.就不行 这时可以上传一个转换后的ip地址，ip地址对应网站包含后门代码 转换后的int会在访问的时候…
阅读更多...
【汽车制动】汽车制动相关控制系统

【汽车制动】汽车制动相关控制系统

目录 1.ABS (Anti-lock Brake System，防抱死制动系统) 2.EBD（Electronic Brake-force Distribution，电子制动力分配系统） 3.TCS（Traction Control System，牵引力控制系统） 4.VDC&#xff08…
阅读更多...
【通俗理解】边际化技巧在概率论中的应用——从公式到实例

【通俗理解】边际化技巧在概率论中的应用——从公式到实例

【通俗理解】边际化技巧在概率论中的应用——从公式到实例 关键词提炼 #边际化技巧 #概率论 #联合概率 #条件概率 #积分计算 #概率分布 #贝叶斯推断 第一节：边际化技巧的类比与核心概念【尽可能通俗】 边际化技巧，就像是你在一个复杂的概率迷宫中&am…
阅读更多...
C#里怎么样使用线程暂停？

C#里怎么样使用线程暂停？

C#里怎么样使用线程暂停？ 如果一个线程没有任务在处理，并且又不进行暂停， 这时候，这个线程就会把当前这个CPU占满，即是所谓的死循环。 因此我们设计线程时，一定要知道线程在什么时候没有工作处理时， 就需要进入等待状态，不能再进行下去，否则会导致死循环， 只是耗费…
阅读更多...
Tri Mode Ethernet MAC IP核详解

Tri Mode Ethernet MAC IP核详解

本文对 Vivado 的三速 MAC IP 核（Tri Mode Ethernet MAC，TEMAC）进行介绍。 在自行实现三速以太网 MAC 控制器时，GMII/RGMII 接口可以通过 IDDR、ODDR 原语实现，然而实际使用中自己实现的模块性能不是很稳定&#xff08…
阅读更多...
项目实战：基于Vue3实现一个小相册

项目实战：基于Vue3实现一个小相册

相册的示例效果图 注意看注释... CSS部分&#xff1a; <style>/* 伪元素选择器&#xff0c;用于在具有clear_ele类的元素内部的末尾添加一个新的元素 */.clear_ele::after{ content: ""; /* 这个伪元素的内容属性必须有 *//* 块级元素&#xff0c;便于占据一…
阅读更多...
STM32F4----DCA数字量转换成模拟量

STM32F4----DCA数字量转换成模拟量

STM32F4----DCA数字量转换成模拟量 基本原理 上一节讲诉了ADC的具体原理与程序搭建https://blog.csdn.net/qq_35970934/article/details/143999874?spm1001.2014.3001.5501。这节讲DAC的原理和程序&#xff0c;在实际应用中&#xff0c;我们经常需要调节电压的输出大小&…
阅读更多...
Linux下通过DRM操作屏幕，发生行对齐 (stride)问题

Linux下通过DRM操作屏幕，发生行对齐 (stride)问题

前言 Linux下使用LVGL操作屏幕&#xff0c;屏幕尺寸是[280*1424]&#xff0c;不管如何设置LVGL的参数&#xff0c;屏幕的显示均为花屏&#xff0c;能看到有图像显示&#xff0c;但是图像是行错乱的。 ubuntu桌面系统显示正常 打印DRM看输出 drm: 280x1424 (0mm X 0mm) pixel …
阅读更多...
HTMLCSS：比赛记分卡

HTMLCSS：比赛记分卡

效果演示 这段 HTML 和 CSS 代码创建了一个卡片式的体育比赛信息展示组件&#xff0c;用于显示篮球比赛的两个队伍名称、比赛时间、比分以及一些装饰性的视觉元素。 HTML <div class"card"><div data-status"inprogress" class"teams"…
阅读更多...
Paper -- 3D建筑模型生成 -- GABLE: 基于高分辨率卫星影像的全国尺度精细3D建筑模型

Paper -- 3D建筑模型生成 -- GABLE: 基于高分辨率卫星影像的全国尺度精细3D建筑模型

基本信息 论文题目: GABLE: A first fine-grained 3D building model of China on a national scale from very high resolution satellite imagery 中文题目: GABLE: 基于高分辨率卫星影像的全国尺度精细3D建筑模型 作者及单位: – 多数作者来自中国科学院空间信息研究所&…
阅读更多...
高级java每日一道面试题-2024年11月21日-数据结构篇-红黑树有哪几个特征?

高级java每日一道面试题-2024年11月21日-数据结构篇-红黑树有哪几个特征?

如果有遗漏,评论区告诉我进行补充 面试官: 红黑树有哪几个特征? 我回答: 红黑树&#xff08;Red-Black Tree&#xff09;是一种自平衡二叉查找树&#xff08;Self-Balancing Binary Search Tree&#xff09;&#xff0c;它在插入和删除操作后能够自动保持树的高度平衡。红黑…
阅读更多...
蓝桥杯嵌入式再学习（2）基础框架的构建

蓝桥杯嵌入式再学习（2）基础框架的构建

1&#xff1a;打开工程以后我们需要进行一些配置如图&#xff1a; 第一步&#xff1a;将core里面的对勾取消掉 勾选Microlib 优化等级选择level 0 将我们自己的文件夹添加到路径里面去 这个样子我们就基本将框架搭建好了我们然后需要编写各个底层的驱动了
阅读更多...
矩阵/矩阵乘法/特征征/特征向量的讲解

矩阵/矩阵乘法/特征征/特征向量的讲解

线性代数里有很多的概念&#xff0c;很多概念是有几何意义的&#xff0c;了解了几何意义可能会更好的理解各种概念及其相互之间的关系。 矩阵&#xff1a; 矩阵是一个变换&#xff0c;一个坐标系到另一个坐标第的变换。矩阵里的各个参数&#xff0c;代表了如何进行变换。 矩阵…
阅读更多...
Flink Transformation - 转换算子全面解析

Flink Transformation - 转换算子全面解析

Flink Transformation - 转换算子全面解析 一、引言 在Flink的数据流处理中&#xff0c;转换算子&#xff08;Transformation Operators&#xff09;扮演着极为关键的角色。它们能够对输入的数据流进行各种处理和转换操作&#xff0c;以满足不同的业务需求。本文将详细介绍Fl…
阅读更多...
Easyexcel（5-自定义列宽）

Easyexcel（5-自定义列宽）

相关文章链接 Easyexcel&#xff08;1-注解使用&#xff09;Easyexcel&#xff08;2-文件读取&#xff09;Easyexcel&#xff08;3-文件导出&#xff09;Easyexcel&#xff08;4-模板文件&#xff09;Easyexcel&#xff08;5-自定义列宽&#xff09; 注解 ColumnWidth Data…
阅读更多...
C++ 中的模板特化和偏特化

C++ 中的模板特化和偏特化

C中的模板特化和偏特化是C模板编程中的两种重要技术&#xff0c;用于在特定情况下提供更优化的代码实现。‌ 模板特化 模板特化是指在模板参数为特定类型时&#xff0c;提供一种特定的实现方式。模板特化分为‌函数模板特化‌和‌类模板特化‌。 函数模板特化‌&#xff1a; …
阅读更多...
短信发送业务

短信发送业务

Override public void sendCode(String phone) {// 通过正则判断手机号的合法性if (!phone.matches("^1[3-9]\\d{9}$")) {throw new RuntimeException("请输入合法的手机号");}// 判断3次// String.format("code:%s", phone)String numKey Stri…
阅读更多...
最新文章

Copyright @ 2022~2023