一、前言
在当前系统开发过程中,单体架构的应用正在走向消亡,越来越多的应用开始分布式架构,在分布式架构设计过程中,有一个问题往往避免不了,就是分布式ID生成问题
。
在常见解决方案中,有使用雪花算法
或者自建统一ID生成服务
,雪花算法
是一个很好的分布式ID生成方案,不过雪花算法
的递增规律可能看起来不太明显,自建统一ID生成服务
面对中小型系统来说又太过于复杂了,那除了这些方法之外还有什么好的解决方法没有呢?
这次我们介绍一个解决方案,基于数据库号段的解决方案
。
二、技术实现
1. 原理解析
我们本次介绍的基于数据库号段的解决方案
方案的原理大体如下:
-
数据库中新建一张表,用于记录号段的使用情况,每个序列号的号段信息都有唯一标识用于区分;
-
应用第一次获取ID的时候,先根据序号标识从数据库中获取并更新号段信息,将获取的号段信息缓存到应用中,在应用中根据号段信息和指定的ID生成属性生成ID;
-
应用后续生成ID时,直接通过缓存在应用内的号段信息生成,如果生成的ID超过号段限制了,再去更新数据库并重新获取号段信息,进行ID生成;
-
为了防止号段一直更新导致溢出,增加号段日切方案,即:每次生成的ID可以携带当前日期信息,应用日期发生日切时,数据库号段信息重新置0,简单来说就是新的一天,序列号又从1开始,由于携带了当前日期信息系,所以也不会重复。
示意架构如下:
生成序列号ID的逻辑嵌入到每个应用中,是去中心化
的模式,号段信息维护依赖数据库,更新时依靠数据库的锁机制保障号段的递增性,防止由于号段覆盖产生的序号ID重复,应用内真正生成ID时,会使用Java的锁机制进行应用内的序号生成唯一性保证。
2. 编码实现
好了,上面介绍了我们数据库号段模式序列号组件大概原来,下面进行实战阶段吧。
首先,我们需要在数据库中创建一张表,由于记录数据库中的号段信息,表信息不用很复杂,建表语句如下:
CREATE TABLE `db_sequence`
(`sequence_key` varchar(64) NOT NULL COMMENT '序列号key,应用通过不同的key可以获取不同序号信息',`start_index` bigint(20) COMMENT '号段的起始值',`generator_date` datetime COMMENT '当前序号的生成日期',PRIMARY KEY (`sequence_key`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
建好表以后就正式进入编码阶段了。
-
新建一个spring boot项目,导入如下依赖:
<dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency><groupId>org.mybatis.spring.boot</groupId><artifactId>mybatis-spring-boot-starter</artifactId><version>2.3.0</version> </dependency> <dependency><groupId>com.mysql</groupId><artifactId>mysql-connector-j</artifactId> </dependency> <dependency><groupId>org.projectlombok</groupId><artifactId>lombok</artifactId><optional>true</optional> </dependency> <dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-configuration-processor</artifactId><optional>true</optional> </dependency>
-
创建序列号功能的配置文件属性接收类
为了使我们使用序列号功能更加灵活,我们创建一个属性配置接收实体类:
import lombok.Getter; import lombok.Setter; import lombok.ToString; import org.springframework.boot.context.properties.ConfigurationProperties;import java.io.Serializable;@ConfigurationProperties(prefix = DBSequenceProperties.KEY_PREFIX) @Getter @Setter @ToString public class DBSequenceProperties implements Serializable {public static final String KEY_PREFIX = "db.sequence";/*** 是否启用组件*/private boolean enable;/*** 是否日切,默认日切,即:每日生成的序列号会重置从1开始,同时生成的序列号会默认添加当前应用日志,* 如果关闭则一直使用序列号生成,有溢出的风险*/private boolean dailyCutting = true;/*** 从数据库获取序号信息时,默认的key名称*/private String defaultKey = "sys_default";/*** 数据库号段的步长*/private Integer stepLength = 10;/*** 生成的序号长度,长度不够时,默认前面进行补0操作*/private Integer sequenceLength = 16;/*** 序号是否拼接日期字符串*/private boolean dateStr = true; }
配置信息比较简单,核心就是号段的大小和生成的序号长度,
号段的大小直接关乎序列号生成的性能
,毕竟是依赖数据库保存号段信息,如果号段设置过小会导致数据库锁竞争频繁,影响性能,如果设置过大,应用宕机又有序号浪费的问题;同时,一般针对序号的生成为了使用方便都有长度要求,所以我们也要设置合理的序号长度。 -
创建序列号功能的缓存信息保存类
前面已经介绍了,应用获取了号段之后需要缓存到应用中,这样下次获取的时候就不用频繁访问数据库了,我们需要构建一个可以用于缓存序号信息的类。
import lombok.Getter; import lombok.Setter; import lombok.ToString;import java.util.Date;/*** 数据库序列号信息*/ @Getter @Setter @ToString public class DBSequenceContent {/*** 序列号key*/private String sequenceKey;/*** 当前序列号*/private Long currentIndex;/*** 最大序列号*/private Long maxId;/*** 序列号生成时间*/private Date sequenceGeneratorDate;/*** 序列号生成时间字符串*/private String sequenceGeneratorDateStr; }
-
创建序列号功能的生成器
前面做好准备工作以后,就可以真正准备序列号的生成逻辑了,整个生成逻辑比较简单,注释在代码中已经写了。
import com.j.sequence.support.DBSequenceProperties; import lombok.extern.slf4j.Slf4j; import org.springframework.util.StringUtils;import javax.sql.DataSource; import java.util.Date; import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ConcurrentMap;/*** 数据库序号组件生成器*/ @Slf4j public class DBSequenceGenerator {/*** 缓存序列号信息*/private static final ConcurrentMap<String, DBSequenceContent> SEQUENCE_CONTENT_MAP = new ConcurrentHashMap();/*** 数据源对象*/private DataSource dataSource;/*** 组件配置信息*/private DBSequenceProperties dbSequenceProperties;public DBSequenceGenerator(DataSource dataSource, DBSequenceProperties dbSequenceProperties) {this.dataSource = dataSource;this.dbSequenceProperties = dbSequenceProperties;}/*** 获取默认key的序号信息** @return*/public String getId() {return getId(dbSequenceProperties.getDefaultKey());}/*** 获取指定sequenceKey的序列号** @param sequenceKey* @return*/public String getId(String sequenceKey) {// 校验sequenceKeyif (!StringUtils.hasLength(sequenceKey)) {throw new IllegalArgumentException("sequenceKey must not be null!");}Date appDate = new Date();// 判断当前应用内是否已经缓存了DBSequenceContent dbSequenceContent = SEQUENCE_CONTENT_MAP.get(sequenceKey);if (dbSequenceContent == null) { // 内存中没有,需要从数据库中加载信息synchronized (sequenceKey.intern()) { // 将锁的粒度细化到sequenceKeydbSequenceContent = SEQUENCE_CONTENT_MAP.get(sequenceKey);if (dbSequenceContent == null) { // 双重检查,防止其他线程已经初始化了dbSequenceContentdbSequenceContent = DBSequenceDBHandler.loadSequenceContent(dataSource, sequenceKey, dbSequenceProperties, appDate);updateSequenceContentMap(dbSequenceContent, sequenceKey);}}}if (dbSequenceProperties.isDailyCutting()) { // 开启了日切模式if (DBSequenceDBHandler.compareDate(appDate, dbSequenceContent.getSequenceGeneratorDate()) > 0) { // 当前应用时间大于了序列号变动时间了synchronized (sequenceKey.intern()) {dbSequenceContent = SEQUENCE_CONTENT_MAP.get(sequenceKey);if (DBSequenceDBHandler.compareDate(appDate, dbSequenceContent.getSequenceGeneratorDate()) > 0) { // 同样防止其他线程更新了dbSequenceContentdbSequenceContent = DBSequenceDBHandler.reloadSequenceContent(dataSource, sequenceKey, dbSequenceProperties, appDate);updateSequenceContentMap(dbSequenceContent, sequenceKey);}}}}return doGeneratorSequence(dataSource, sequenceKey, dbSequenceProperties, appDate);}/*** 生成序列号** @param dataSource* @param sequenceKey* @param appDate* @return*/private String doGeneratorSequence(DataSource dataSource, String sequenceKey, DBSequenceProperties dbSequenceProperties, Date appDate) {long tempId;String dateStr;synchronized (sequenceKey.intern()) {DBSequenceContent dbSequenceContent = SEQUENCE_CONTENT_MAP.get(sequenceKey);long sequence = dbSequenceContent.getCurrentIndex() + 1;if (sequence > dbSequenceContent.getMaxId()) { // 超过了最大值,重新从数据库中获取号段信息dbSequenceContent = DBSequenceDBHandler.reloadSequenceContent(dataSource, sequenceKey, dbSequenceProperties, appDate);updateSequenceContentMap(dbSequenceContent, sequenceKey);sequence = dbSequenceContent.getCurrentIndex() + 1;}dbSequenceContent.setCurrentIndex(sequence);tempId = sequence;dateStr = dbSequenceContent.getSequenceGeneratorDateStr();}String idStr = String.valueOf(tempId);int sequenceLength = dbSequenceProperties.getSequenceLength();int idLength = idStr.length();StringBuilder idSb = new StringBuilder();if (dbSequenceProperties.isDateStr()) {idSb.append(dateStr);idLength += idSb.length();}if (sequenceLength >= idLength) { // 位数不够需要进行补0操作int length = sequenceLength - idLength;for (int i = 0; i < length; i++) {idSb.append("0");}} else {throw new IllegalArgumentException("idLength more than sequenceLength limit!");}idSb.append(tempId);return idSb.toString();}/*** 更新dbSequenceContent属性** @param dbSequenceContent* @param sequenceKey*/private void updateSequenceContentMap(DBSequenceContent dbSequenceContent, String sequenceKey) {if (dbSequenceContent == null || dbSequenceContent.getCurrentIndex() == null) {SEQUENCE_CONTENT_MAP.remove(sequenceKey); // 移除缓存中的信息,方便下次从数据库中获取throw new RuntimeException(String.format("get %s info error, please check db!", sequenceKey));}SEQUENCE_CONTENT_MAP.put(sequenceKey, dbSequenceContent);}/*** 清理缓存中的sequenceKey信息,清理以后,下次获取会重新从数据库中查询** @param sequenceKeys* @return*/public boolean clearCacheSequence(String... sequenceKeys) {if (sequenceKeys == null || sequenceKeys.length == 0) {synchronized (this) {SEQUENCE_CONTENT_MAP.clear();}} else {for (int i = 0; i < sequenceKeys.length; i++) {String key = sequenceKeys[i];synchronized (key.intern()) {SEQUENCE_CONTENT_MAP.remove(key);}}}return true;}}
-
实现序列号功能的数据库操作逻辑
DBSequenceGenerator
类中的逻辑主要专注于ID生成的整个逻辑流转,涉及真正的数据库操作,我们可以放到另一个类中,这样核心代码看起来会简洁一些:import com.j.sequence.support.DBSequenceProperties; import lombok.extern.slf4j.Slf4j;import javax.sql.DataSource; import java.sql.*; import java.text.SimpleDateFormat; import java.util.Date;/*** 数据库序号组件数据库操作处理器*/ @Slf4j public class DBSequenceDBHandler {/*** 加载数据库中序列号信息,没有的话则保存** @param dataSource* @param sequenceKey* @param dbSequenceProperties* @param appDate* @return*/public static DBSequenceContent loadSequenceContent(DataSource dataSource, String sequenceKey, DBSequenceProperties dbSequenceProperties, Date appDate) {DBSequenceContent dbSequenceContent;Connection connection = null;Boolean autoCommit = null;try {connection = dataSource.getConnection();// 都是简单操作SQL,为了适配不同ORM框架,只需要注入DataSource对象就行,所以SQL写死在代码中,数据库操作使用原生的JDBCString sql = "SELECT start_index, generator_date FROM db_sequence where sequence_key = ? ";PreparedStatement ps = connection.prepareStatement(sql);ps.setString(1, sequenceKey);ResultSet rs = ps.executeQuery();autoCommit = connection.getAutoCommit();connection.setAutoCommit(false);if (rs != null && rs.next()) { // 数据库中已经存在该条记录dbSequenceContent = updateDBSequenceContent(connection, sequenceKey, dbSequenceProperties, appDate);} else { // 数据库中不存在数据需要新增sql = "INSERT INTO db_sequence (sequence_key, start_index, generator_date) VALUES(?, ?, ?)";PreparedStatement psSave = connection.prepareStatement(sql);psSave.setString(1, sequenceKey);psSave.setInt(2, dbSequenceProperties.getStepLength());psSave.setTimestamp(3, new Timestamp(appDate.getTime()));psSave.executeUpdate();psSave.close();dbSequenceContent = new DBSequenceContent();dbSequenceContent.setSequenceKey(sequenceKey);dbSequenceContent.setSequenceGeneratorDate(appDate);dbSequenceContent.setSequenceGeneratorDateStr(new SimpleDateFormat("yyyyMMdd").format(appDate));dbSequenceContent.setCurrentIndex(0L);dbSequenceContent.setMaxId(dbSequenceProperties.getStepLength() * 1L);}rs.close();ps.close();connection.commit();} catch (SQLException sqlException) {if (connection != null) {try {connection.rollback();} catch (SQLException se) {log.error("connection rollback error!", se);}}log.error("add sequenceKey: {} error!", sequenceKey, sqlException);// 可能是其他应用已经save过了,此时插入报主键冲突了,所以重试一下log.info("retry get dbSequenceContent by reloadSequenceContentByDailyCutting start!");dbSequenceContent = reloadSequenceContent(dataSource, sequenceKey, dbSequenceProperties, appDate);if (dbSequenceContent != null && dbSequenceContent.getCurrentIndex() != null) {log.info("retry get dbSequenceContent by reloadSequenceContentByDailyCutting successes!");} else {log.error("retry get dbSequenceContent by reloadSequenceContentByDailyCutting error!");}} finally {closeConnection(connection, autoCommit);}return dbSequenceContent;}private static DBSequenceContent updateDBSequenceContent(Connection connection, String sequenceKey, DBSequenceProperties dbSequenceProperties, Date appDate) throws SQLException {String sql = "SELECT start_index, generator_date FROM db_sequence where sequence_key = ? for update "; // 存在该条记录再进行上锁PreparedStatement psLock = connection.prepareStatement(sql);psLock.setString(1, sequenceKey);ResultSet rsLock = psLock.executeQuery();DBSequenceContent dbSequenceContent = new DBSequenceContent();if (rsLock.next()) {long startIndex = rsLock.getLong("start_index");Date generatorDate = rsLock.getDate("generator_date");dbSequenceContent.setSequenceKey(sequenceKey);if (dbSequenceProperties.isDailyCutting() && compareDate(generatorDate, appDate) < 0) { //如果序列号需要日切// 数据库中日期晚于应用日期,需要进行日切操作sql = "update db_sequence set start_index=?, generator_date=? where sequence_key = ? ";final PreparedStatement psUpdateSIDate = connection.prepareStatement(sql);psUpdateSIDate.setInt(1, dbSequenceProperties.getStepLength());psUpdateSIDate.setTimestamp(2, new Timestamp(appDate.getTime()));psUpdateSIDate.setString(3, sequenceKey);psUpdateSIDate.executeUpdate();psUpdateSIDate.close();dbSequenceContent.setSequenceGeneratorDate(appDate);dbSequenceContent.setSequenceGeneratorDateStr(new SimpleDateFormat("yyyyMMdd").format(appDate));dbSequenceContent.setCurrentIndex(0L);dbSequenceContent.setMaxId(dbSequenceProperties.getStepLength() * 1L);} else {sql = "update db_sequence set start_index=start_index+? where sequence_key = ? ";final PreparedStatement psUpdateSI = connection.prepareStatement(sql);psUpdateSI.setInt(1, dbSequenceProperties.getStepLength());psUpdateSI.setString(2, sequenceKey);psUpdateSI.executeUpdate();psUpdateSI.close();dbSequenceContent.setSequenceGeneratorDate(generatorDate);dbSequenceContent.setSequenceGeneratorDateStr(new SimpleDateFormat("yyyyMMdd").format(generatorDate));dbSequenceContent.setCurrentIndex(startIndex);dbSequenceContent.setMaxId(startIndex + dbSequenceProperties.getStepLength());}} else {log.error("sequenceKey: {} record maybe delete, please check db!", sequenceKey);}rsLock.close();psLock.close();return dbSequenceContent;}/*** 更新数据库号段信息** @param dataSource* @param sequenceKey* @param dbSequenceProperties* @param appDate* @return*/public static DBSequenceContent reloadSequenceContent(DataSource dataSource, String sequenceKey, DBSequenceProperties dbSequenceProperties, Date appDate) {DBSequenceContent dbSequenceContent = null;Connection connection = null;Boolean autoCommit = null;try {connection = dataSource.getConnection();autoCommit = connection.getAutoCommit();connection.setAutoCommit(false);dbSequenceContent = updateDBSequenceContent(connection, sequenceKey, dbSequenceProperties, appDate);connection.commit();} catch (SQLException sqlException) {dbSequenceContent = null;log.error("reloadSequenceContentByDailyCutting error!", sqlException);} finally {closeConnection(connection, autoCommit);}return dbSequenceContent;}/*** 比较日期,只比较年月日** @param date0* @param date1* @return*/public static int compareDate(Date date0, Date date1) {SimpleDateFormat simpleDateFormat = new SimpleDateFormat("yyyyMMdd");int date0Int = Integer.parseInt(simpleDateFormat.format(date0));int date1Int = Integer.parseInt(simpleDateFormat.format(date1));return date0Int > date1Int ? 1 : (date0Int < date1Int ? -1 : 0);}/*** 关闭connection资源** @param connection* @param autoCommit*/private static void closeConnection(Connection connection, Boolean autoCommit) {if (connection != null) {if (autoCommit != null) {try {connection.setAutoCommit(autoCommit);} catch (SQLException sqlException) {log.error("connection set autoCommit error!", sqlException);}}try {connection.close();} catch (SQLException sqlException) {log.error("connection close error!", sqlException);}}} }
-
创建配置类进行功能加载
在上面核心功能编码实现以后,为了适配
spring boot
项目,我们可以准备一个Configuration
进行配置加载操作,简化功能使用。import com.j.sequence.core.DBSequenceGenerator; import com.j.sequence.support.DBSequenceProperties; import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; import org.springframework.boot.context.properties.EnableConfigurationProperties; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration;import javax.sql.DataSource;/*** 数据库序列号组件配置类*/ @Configuration @EnableConfigurationProperties(DBSequenceProperties.class) /*** 条件加载,需要显示启用设置 db.sequence.enable=true*/ @ConditionalOnProperty(prefix = DBSequenceProperties.KEY_PREFIX, name = "enable", matchIfMissing = true) public class DBSequenceConfiguration {@Bean("com.j.sequence.boot.DBSequenceConfiguration.dbSequenceGenerator")public DBSequenceGenerator dbSequenceGenerator(DataSource dataSource, DBSequenceProperties dbSequenceProperties) {return new DBSequenceGenerator(dataSource, dbSequenceProperties);}}
3. 编码总结
编码实现阶段到此就结束了,代码的核心的逻辑都在注释中描述了,这里我们简单总结一个核心编码逻辑。
- 获取ID时,优先从缓存中获取缓存的号段信息;
- 如果号段信息不存在则需要在数据库中新增
sequenceKey
对应信息号段信息,为了防止其他应用进行了新增,防止主键冲突,程序会先进行是否存在的判断,如果存在则会使用for update
关键字进行行锁,然后进行数据更新,缓存更新操作;否则才会添加,同样为了防止其他应用抢先进行了新增,在新增失败以后,会进行一次直接获取的重试操作,如果这次操作也失败,才会返回空的缓存信息,结束ID获取; - 经历步骤2以后,程序再往下运行,号段信息就一定存在了,此时判断是否发生了日切,如果需要日切则将数据库中的序列号信息重置;
- 经历步骤3以后,应用中的号段缓存信息此时已经可以用于最后的ID生成了,如果ID位数不够就进行补0操作,最后ID生成格式为:
年年年年月月日日[n个0]递增的序号
(n可以为0)。
三、功能测试
在application.yaml
配置文件中添加配置:
spring:application:name: db-sequence-demodatasource:driver-class-name: com.mysql.cj.jdbc.Driverusername: xxxpassword: xxxurl: jdbc:mysql://xx.8.xx.xx:3306/xxxdb:sequence:enable: truedefaultKey: myDBSeqdailyCutting: truestepLength: 9 #号段为9,一次缓存最多生成9个,超过以后要从数据库中重新获取sequenceLength: 12date-str: true
在编码完成以后我们需要进行功能,为了方便我们直接在应用中编写测试代码,启动工程进行测试。
1. 简单测试
简单测试,我们主要测试生成的序列号是否正确并且连续。
- 测试代码
import com.j.sequence.core.DBSequenceGenerator;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;@RestController
public class SeqController {@Autowiredprivate DBSequenceGenerator dbSequenceGenerator;@RequestMapping("/getId")public List<String> getId() {List<String> list = new ArrayList<>();for (int i = 0; i < 10; i++) {list.add(dbSequenceGenerator.getId());}return list;}
}
-
测试请求
可以看到,序列号正常生成了,同时设置的序号号段为9,自动更新获取为10也没有发生任何问题,测试通过。
2. 多线程测试
多线程测试,主要是模拟多个线程并发请求获取ID的时候,ID是否可以正常生成并获取。
-
测试代码
import com.j.sequence.core.DBSequenceGenerator; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RestController;import java.util.Collections; import java.util.HashSet; import java.util.Set; import java.util.concurrent.CountDownLatch; import java.util.concurrent.TimeUnit;@RestController public class SeqController {@Autowiredprivate DBSequenceGenerator dbSequenceGenerator;private Set<String> ids = Collections.synchronizedSet(new HashSet<>());@RequestMapping("getIdByMultiThread")public String getIdByMultiThread() {int threadNum = 5; // 线程数量int idNum = 10;//每个线程获取ID数量CountDownLatch countDownLatch = new CountDownLatch(threadNum);for (int i = 0; i < threadNum; i++) {new Thread(() -> {try {Thread.sleep(200L);} catch (InterruptedException e) {e.printStackTrace();}countDownLatch.countDown();for (int i1 = 0; i1 < idNum; i1++) {ids.add(dbSequenceGenerator.getId());}}).start();}try {TimeUnit.SECONDS.sleep(2L); // 暂停等待线程执行完成,本地测试2s够了,如果不够可自行调整} catch (InterruptedException e) {e.printStackTrace();}return ids.size() == threadNum * idNum ? "生成ID数量符合预期:" + ids.size() : "生成ID重复导致集合数量错误:" + ids.size();}}
-
测试请求
本次测试案例中,我们主要使用Set集合测试多线程情况下ID生成的正确性,我们使用5个线程,每个线程生成10个序号的方式进行测试,预期会生成50个序号,最后测试结果符合预期,测试通过。
3. 多实例测试
多实例测试时,我们打算使用5个实例进行测试,为了测试简单,并不会真正部署5个实例节点,为了方便,修改一下DBSequenceGenerator
类,去掉static
修饰符,使成员变量都是类级别的,如下:
/*** 数据库序号组件生成器*/
@Slf4j
public class DBSequenceGenerator {/*** 缓存序列号信息*/private /*static*/ final ConcurrentMap<String, DBSequenceContent> SEQUENCE_CONTENT_MAP = new ConcurrentHashMap();// ........................其他代码不变
-
测试代码
import com.j.sequence.core.DBSequenceGenerator; import com.j.sequence.support.DBSequenceProperties; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RestController;import javax.sql.DataSource; import java.util.Collections; import java.util.HashSet; import java.util.Set; import java.util.concurrent.CountDownLatch; import java.util.concurrent.TimeUnit;@RestController public class SeqController {@Autowiredprivate DataSource dataSource;@Autowiredprivate DBSequenceProperties dbSequenceProperties;private Set<String> ids = Collections.synchronizedSet(new HashSet<>());@RequestMapping("getIdByMultiInstance")public String getIdByMultiInstance() {int threadNum = 5; // 线程数量int idNum = 10;//每个线程获取ID数量CountDownLatch countDownLatch = new CountDownLatch(threadNum);for (int i = 0; i < threadNum; i++) {new Thread(() -> {// 构建多个DBSequenceGenerator,模拟多个实例DBSequenceGenerator dbSequenceGenerator = new DBSequenceGenerator(dataSource, dbSequenceProperties);try {Thread.sleep(200L);} catch (InterruptedException e) {e.printStackTrace();}countDownLatch.countDown();for (int i1 = 0; i1 < idNum; i1++) {ids.add(dbSequenceGenerator.getId());}}).start();}try {TimeUnit.SECONDS.sleep(2L); // 暂停等待线程执行完成,本地测试2s够了,如果不够可自行调整} catch (InterruptedException e) {e.printStackTrace();}return ids.size() == threadNum * idNum ? "生成ID数量符合预期:" + ids.size() : "生成ID重复导致集合数量错误:" + ids.size();}}
-
测试请求
本次测试过程中,模拟多实例请求,因为DBSequenceGenerator
对象是通过注入spring容器方式提供的,用户在一个实例中使用的时候,只需要通过spring提供的依赖注入就行,所以多实例测试,我们模拟使用5个线程,每个线程都单独创建DBSequenceGenerator
对象去获取10个序号,预期可以获取到50个序号,最后测试结果也符合我们预期,测试通过。
4. 数据库信息核对
按照我们的测试流程,每次测试都会重新重启,我们可以计算一下数据库最终的号段偏移量:
-
简单测试:
10/9=1
,10%9=1<9
,偏移:1*9 + 1*9=18
-
多线程测试:
50/9=5
,50%9=5<9
,偏移:5*9+1*9=54
-
多实例测试:参考简单测试计算方法:
18*5=90
最终:18+54+90=162
查看数据库记录:
通过数据库记录可以确定,号段变化符合我们预期,测试通过。
四、写在最后
通过上面的编码我们实行了一个基于数据库号段去中心化的分布式ID生成方案
,该组件生成的序列号可以保证有序递增,且递增规律比较明显,不过由于号段信息存储在数据库中,多个实例去获取时,只能保证每次获取号段以后,单个实例里面生成的序号是递增的,但是不能保证单个实例里面的序号是连续的,这个需要注意。
一般情况下应用数据库还是很稳定的,合理的设置号段也可以避免数据库的压力,可以把改功能封装成一个可以复用的SDK
,不过针对该方案来说也有很多可以完善的地方,比如号段回收等优化机制,建议用于生产之前还是需要进行严格功能测试和性能测试。