hadoop loadBalance源码分析

项目hbase数据库出现很诡异的assignment ，region移动的src和dest都是同一台regionserver，不过时间戳不同，启动的只有一个regionserver，不知道怎么出现了两个时间戳

分析下源码解决一下

loadbalance只有一个实现 org.apache.hadoop.hbase.master.DefaultLoadBalancer

在HMaster中会启动一个线程 org.apache.hadoop.hbase.Chore，按照设定的hbase.balancer.period（默认300000ms，五分钟），遍历所有表，根据每个表在regionserver中的region数量做balance，有一个平衡系数hbase.regions.slop（默认0.2），根据region总数算出平均region值，avg×0.8 取整作为最小值，avg×1.2取整作为最大值，regionserver上超过最大值要移走，小于最小值要移动region过来。否则打印目前的平衡状态。

assignmentManager 根据上述步骤生成的RegionPlan, 从src移动region到desc src和desc都是ServerName对象

HMaster启动时会等待region servers注册到serverManager

// Wait for region servers to report in.
this.serverManager.waitForRegionServers(status);
// Check zk for regionservers that are up but didn't register
for (ServerName sn: this.regionServerTracker.getOnlineServers()) {
    if (!this.serverManager.isServerOnline(sn)) {
    // Not registered; add it.LOG.info("Registering server found up in zk but who has not yet " +"reported in: " + sn);this.serverManager.recordNewServer(sn, HServerLoad.EMPTY_HSERVERLOAD);}
}

serverManager线程sleep一定时间，等待HRegionServer注册

HRegionServer.java:

      // Try and register with the Master; tell it we are here.  Break if// server is stopped or the clusterup flag is down or hdfs went wacky.while (keepLooping()) {MapWritable w = reportForDuty();if (w == null) {LOG.warn("reportForDuty failed; sleeping and then retrying.");this.sleeper.sleep();} else {handleReportForDutyResponse(w);break;}}

HRegionServer 注册之后进入mainloop

 // The main run loop.while (!this.stopped && isHealthy()) {
　　long now = System.currentTimeMillis();

　　　if ((now - lastMsg) >= msgInterval) {
　　　　　　doMetrics();
　　　　　　tryRegionServerReport();
　　　　　　lastMsg = System.currentTimeMillis();
　　　　}

}

每隔hbase.regionserver.msginterval时间（默认3秒），进行一次注册尝试,如果服务器ip和端口不在已注册列表中，则添加ServerName进map

ServerManager.java

 void regionServerReport(ServerName sn, HServerLoad hsl)throws YouAreDeadException, PleaseHoldException {checkIsDead(sn, "REPORT");if (!this.onlineServers.containsKey(sn)) {// Already have this host+port combo and its just different start code?
      checkAlreadySameHostPort(sn);// Just let the server in. Presume master joining a running cluster.// recordNewServer is what happens at the end of reportServerStartup.// The only thing we are skipping is passing back to the regionserver// the ServerName to use. Here we presume a master has already done// that so we'll press on with whatever it gave us for ServerName.
      recordNewServer(sn, hsl);} else {this.onlineServers.put(sn, hsl);}}

recordNewServer 会打印 ServerName对象的ip 端口和时间戳信息

同一个region server注册的ServerName对象会拥有同样的时间戳

this.startcode = System.currentTimeMillis();...result = this.hbaseMaster.regionServerStartup(port, this.startcode, now);...this.serverNameFromMasterPOV = new ServerName(hostnameFromMasterPOV,
this.isa.getPort(), this.startcode);...this.hbaseMaster.regionServerReport(this.serverNameFromMasterPOV.getVersionedBytes(), hsl);