聊聊nacos-RaftCore的MasterElection

序

本文主要研究一下 nacos RaftCore 的 MasterElection

RaftCore

nacos-1.1.3/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/raft/RaftCore.java

@Component
public class RaftCore {
    
    //......

    @PostConstruct
    public void init() throws Exception {Loggers.RAFT.info("initializing Raft sub-system");

        executor.submit(notifier);

        long start = System.currentTimeMillis();

        raftStore.loadDatums(notifier, datums);

        setTerm(NumberUtils.toLong(raftStore.loadMeta().getProperty("term"), 0L));

        Loggers.RAFT.info("cache loaded, datum count: {}, current term: {}", datums.size(), peers.getTerm());

        while (true) {if (notifier.tasks.size() <= 0) {break;}
            Thread.sleep(1000L);
        }

        initialized = true;

        Loggers.RAFT.info("finish to load data from disk, cost: {} ms.", (System.currentTimeMillis() - start));

        GlobalExecutor.registerMasterElection(new MasterElection());
        GlobalExecutor.registerHeartbeat(new HeartBeat());

        Loggers.RAFT.info("timer started: leader timeout ms: {}, heart-beat timeout ms: {}",
            GlobalExecutor.LEADER_TIMEOUT_MS, GlobalExecutor.HEARTBEAT_INTERVAL_MS);
    }

    //......
}

• RaftCore 的 init 方法通过 GlobalExecutor.registerMasterElection(new MasterElection()) 注册了 MasterElection

GlobalExecutor

nacos-1.1.3/naming/src/main/java/com/alibaba/nacos/naming/misc/GlobalExecutor.java

public class GlobalExecutor {

    //......

    public static final long TICK_PERIOD_MS = TimeUnit.MILLISECONDS.toMillis(500L);

    public static void registerMasterElection(Runnable runnable) {executorService.scheduleAtFixedRate(runnable, 0, TICK_PERIOD_MS, TimeUnit.MILLISECONDS);
    }

    //......
}

• registerMasterElection 方法每隔 TICK_PERIOD_MS 毫秒调度一次 runnable

MasterElection

nacos-1.1.3/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/raft/RaftCore.java

    public class MasterElection implements Runnable {
        @Override
        public void run() {
            try {if (!peers.isReady()) {return;}

                RaftPeer local = peers.local();
                local.leaderDueMs -= GlobalExecutor.TICK_PERIOD_MS;

                if (local.leaderDueMs > 0) {return;}

                // reset timeout
                local.resetLeaderDue();
                local.resetHeartbeatDue();

                sendVote();} catch (Exception e) {Loggers.RAFT.warn("[RAFT] error while master election {}", e);
            }

        }

        public void sendVote() {RaftPeer local = peers.get(NetUtils.localServer());
            Loggers.RAFT.info("leader timeout, start voting,leader: {}, term: {}",
                JSON.toJSONString(getLeader()), local.term);

            peers.reset();

            local.term.incrementAndGet();
            local.voteFor = local.ip;
            local.state = RaftPeer.State.CANDIDATE;

            Map<String, String> params = new HashMap<>(1);
            params.put("vote", JSON.toJSONString(local));
            for (final String server : peers.allServersWithoutMySelf()) {final String url = buildURL(server, API_VOTE);
                try {HttpClient.asyncHttpPost(url, null, params, new AsyncCompletionHandler<Integer>() {
                        @Override
                        public Integer onCompleted(Response response) throws Exception {if (response.getStatusCode() != HttpURLConnection.HTTP_OK) {Loggers.RAFT.error("NACOS-RAFT vote failed: {}, url: {}", response.getResponseBody(), url);
                                return 1;
                            }

                            RaftPeer peer = JSON.parseObject(response.getResponseBody(), RaftPeer.class);

                            Loggers.RAFT.info("received approve from peer: {}", JSON.toJSONString(peer));

                            peers.decideLeader(peer);

                            return 0;
                        }
                    });
                } catch (Exception e) {Loggers.RAFT.warn("error while sending vote to server: {}", server);
                }
            }
        }
    }

• MasterElection 实现了 Runnable 方法，其 run 方法在 peers 都是 ready 而且 local.leaderDueMs 减去 TICK_PERIOD_MS 小于等于 0 的时候会开始选举；它首先 resetLeaderDue 及 resetHeartbeatDue，然后执行 sendVote 方法；sendVote 方法首先重置 peers，递增 localPeer 的 term，并设置 voteFor 为自己，然后更新 state 为 RaftPeer.State.CANDIDATE，最后遍历 peers.allServersWithoutMySelf()，将自己的 vote 信息异步 post 给其他 peer；如果其他 peer 返回成功则执行 peers.decideLeader(peer)，返回 1，否则返回 0

小结

RaftCore 的 init 方法通过 GlobalExecutor.registerMasterElection(new MasterElection()) 注册了 MasterElection；registerMasterElection 方法每隔 TICK_PERIOD_MS 毫秒调度一次；MasterElection 实现了 Runnable 方法，其 run 方法在 peers 都是 ready 而且 local.leaderDueMs 减去 TICK_PERIOD_MS 小于等于 0 的时候会开始选举；它首先 resetLeaderDue 及 resetHeartbeatDue，然后执行 sendVote 方法

doc

• RaftCore