shardingsphere从4.1.1升级到5.2.1但是还没有完结,因为在执行存储过程的时候,系统提示错误如下。shardingsphere是不支持存储过程呢,但项目中不能避免使用存储过程,因为有大量的数据需要初始化,这种情况该如何应对?
### SQL: {call init_data( ?, ?, ? ) }
Caused by: java.sql.SQLFeatureNotSupportedException: prepareCallat org.apache.shardingsphere.driver.jdbc.unsupported.AbstractUnsupportedOperationConnection.prepareCall(AbstractUnsupportedOperationConnection.java:43)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at org.apache.ibatis.logging.jdbc.ConnectionLogger.invoke(ConnectionLogger.java:55)at com.sun.proxy.$Proxy170.prepareCall(Unknown Source)at org.apache.ibatis.executor.statement.CallableStatementHandler.instantiateStatement(CallableStatementHandler.java:87)at org.apache.ibatis.executor.statement.BaseStatementHandler.prepare(BaseStatementHandler.java:88)at org.apache.ibatis.executor.statement.RoutingStatementHandler.prepare(RoutingStatementHandler.java:59)at sun.reflect.GeneratedMethodAccessor358.invoke(Unknown Source)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at org.apache.ibatis.plugin.Invocation.proceed(Invocation.java:49)
既然存储过程并不涉及分片的表,为何shardingsphere还会干预呢?
使用Hikari
可以看到下面的日志
JDBC Connection [HikariProxyConnection@289895964 wrapping com.mysql.cj.jdbc.ConnectionImpl@164ef602] will not be managed by Spring
如果被shardingsphere管理则可以看到下面的日志
JDBC Connection [org.apache.shardingsphere.driver.jdbc.core.connection.ShardingSphereConnection@6e64e072] will not be managed by Spring
因为shardingsphere不支持存储过程及一些语句,所以产生了动态数据源的需求,在网上找的例子配置,Mybatis-plus@DS实现动态切换数据源应用
import com.baomidou.dynamic.datasource.DynamicRoutingDataSource;
import com.baomidou.dynamic.datasource.provider.AbstractDataSourceProvider;
import com.baomidou.dynamic.datasource.provider.DynamicDataSourceProvider;
import com.baomidou.dynamic.datasource.spring.boot.autoconfigure.DataSourceProperty;
import com.baomidou.dynamic.datasource.spring.boot.autoconfigure.DynamicDataSourceAutoConfiguration;
import com.baomidou.dynamic.datasource.spring.boot.autoconfigure.DynamicDataSourceProperties;
import org.apache.shardingsphere.driver.jdbc.adapter.AbstractDataSourceAdapter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.SpringBootConfiguration;
import org.springframework.boot.autoconfigure.AutoConfigureBefore;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;import org.springframework.context.annotation.Lazy;
import javax.annotation.Resource;
import javax.sql.DataSource;
import java.util.Map;@Configuration
@AutoConfigureBefore({DynamicDataSourceAutoConfiguration.class,SpringBootConfiguration.class})
public class DataSourceConfig {/*** 分表数据源名称*/private static final String SHARDING_DATA_SOURCE_NAME = "acc_sharding";@Autowiredprivate DynamicDataSourceProperties properties;@Lazy@Resource(name = "shardingDataSource")AbstractDataSourceAdapter shardingDataSource;@Beanpublic DynamicDataSourceProvider dynamicDataSourceProvider() {Map<String, DataSourceProperty> datasourceMap = properties.getDatasource();return new AbstractDataSourceProvider() {@Overridepublic Map<String, DataSource> loadDataSources() {Map<String, DataSource> dataSourceMap = createDataSourceMap(datasourceMap);// 将 shardingjdbc 管理的数据源也交给动态数据源管理dataSourceMap.put(SHARDING_DATA_SOURCE_NAME, shardingDataSource);return dataSourceMap;}};}@Primary@Beanpublic DataSource dataSource(DynamicDataSourceProvider dynamicDataSourceProvider) {DynamicRoutingDataSource dataSource = new DynamicRoutingDataSource();dataSource.setPrimary(properties.getPrimary());dataSource.setStrict(properties.getStrict());dataSource.setStrategy(properties.getStrategy());dataSource.setProvider(dynamicDataSourceProvider);dataSource.setP6spy(properties.getP6spy());dataSource.setSeata(properties.getSeata());return dataSource;}}
上面的java配置,还有一个问题没有解决,那就是shardingDataSource
这个数据源是怎么初始化的。因为shardingsphere5.2.1
没有默认数据源,因此只好先降到5.1.1
,Shardingsphere5.1.1 整合druid动态数据源,
在5.1.1
中需要去掉下面的配置,因为不支持
shardingsphere5.x整合springboot+dynamic-datasource多数据源实战
切到5.1.1
上面的动态数据源代码是有问题的,因为5.1.1版本自动装载的shardingSphereDataSource beanName=“shardingSphereDataSource”
动态数据源是好的,但带来的问题却是数据库初始化连接过多,如下面的配置,每个系统就有两份数据库连接池配置,这个需要注意,在k8s扩展服务的时候应该考虑到这一点
@Lazy@Autowiredprivate DataSource shardingDataSource;
如果按照这个结论,那么将版本再切到5.2.1
会怎么样呢?猜想有些地方应该还是会继承把。这个猜想是对的。
配置如下
spring:datasource:dynamic:hikari: # 全局hikariCP参数pool-name: Retail_HikariCP #连接池名称minimum-idle: 5 #最小空闲连接数量idle-timeout: 120000 #空闲连接存活最大时间,默认600000(10分钟)maximum-pool-size: 10 #连接池最大连接数,默认是10auto-commit: true #此属性控制从池返回的连接的默认自动提交行为,默认值:truemax-lifetime: 1800000 #此属性控制池中连接的最长生命周期,值0表示无限生命周期,默认1800000即30分钟connection-timeout: 30000 #数据库连接超时时间,默认30秒,即30000connection-test-query: SELECT 1datasource:master:type: com.zaxxer.hikari.HikariDataSourcedriver-class-name: com.mysql.cj.jdbc.Driverusername: rootpassword: ENC(xxx)url: jdbc:mysql://127.0.0.1:3306/acc?useUnicode=true&characterEncoding=UTF-8&useSSL=false&serverTimezone=Asia/Shanghai&allowMultiQueries=true# 数据库配置shardingsphere: mode:type: Standalonerepository:type: JDBCdatasource:names: acc1acc1:type: com.zaxxer.hikari.HikariDataSourcedriver-class-name: com.mysql.cj.jdbc.Driverusername: rootpassword: ENC(xxx)jdbcUrl: jdbc:mysql://127.0.0.1:3306/acc?useUnicode=true&characterEncoding=UTF-8&useSSL=false&serverTimezone=Asia/Shanghai&allowMultiQueries=truepool-name: Retail_HikariCP #连接池名称minimum-idle: 5 #最小空闲连接数量#idle-timeout: 600000 #空闲连接存活最大时间,默认600000(10分钟)maximum-pool-size: 10 #连接池最大连接数,默认是10auto-commit: true #此属性控制从池返回的连接的默认自动提交行为,默认值:true#max-lifetime: 600000 #此属性控制池中连接的最长生命周期,值0表示无限生命周期,默认1800000即30分钟connection-timeout: 300000 #数据库连接超时时间,默认30秒,即30000connection-test-query: SELECT 1props:sql:show: truesharding:default-data-source-name: acc1rules:sharding:tables:# 科目acc_account_subject:actual-data-nodes: acc1.acc_account_subject_$->{0..49}table-strategy:standard:sharding-column: as_idsharding-algorithm-name: acc_account_subject-inline# 科目余额acc_account_balance:actual-data-nodes: acc1.acc_account_balance_$->{0..49}table-strategy:standard:sharding-column: as_idsharding-algorithm-name: acc_account_balance-inline# 期初acc_initial_balance:actual-data-nodes: acc1.acc_initial_balance_$->{0..49}table-strategy:standard:sharding-column: as_idsharding-algorithm-name: acc_initial_balance-inline# 凭证acc_voucher:actual-data-nodes: acc1.acc_voucher_$->{0..49}table-strategy:standard:sharding-column: as_idsharding-algorithm-name: acc_voucher-inline# 凭证分录acc_voucher_entry:actual-data-nodes: acc1.acc_voucher_entry_$->{0..49}table-strategy:standard:sharding-column: as_idsharding-algorithm-name: acc_voucher_entry-inlinebindingTables:- acc_account_subject,acc_account_balance,acc_initial_balance,acc_voucher,acc_voucher_entrysharding-algorithms:acc_account_subject-inline:type: inlineprops:algorithm-expression: acc_account_subject_$->{as_id%50}acc_account_balance-inline:type: inlineprops:algorithm-expression: acc_account_balance_$->{as_id%50}acc_initial_balance-inline:type: inlineprops:algorithm-expression: acc_initial_balance_$->{as_id%50}acc_voucher-inline:type: inlineprops:algorithm-expression: acc_voucher_$->{as_id%50}acc_voucher_entry-inline:type: inlineprops:algorithm-expression: acc_voucher_entry_$->{as_id%50}
在java侧,针对分表的还需要加上注解@DS
,一般用在mapper、service上
否则会提示
### Cause: java.sql.SQLSyntaxErrorException: Table 'acc.acc_account_subject' doesn't exist
; bad SQL grammar []; nested exception is java.sql.SQLSyntaxErrorException: Table 'acc.acc_account_subject' doesn't existat java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1592)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)at java.lang.Thread.run(Thread.java:748)
Caused by: org.springframework.jdbc.BadSqlGrammarException:
### Error querying database. Cause: java.sql.SQLSyntaxErrorException: Table 'acc.acc_account_subject' doesn't exist
### The error may exist in com/whty/acc/setting/dao/AccStatementAsMapper.java (best guess)
分表的数据直接在service上加上注解
关联方,在具体的方法上加,这些第一感觉有些侵入性,但是也还好,提醒其他工程师,这个代码要注意分表,这样好像也没有错。
再怎么原生的mybatis,比shardingsphere再转义一道,还是要快些。