乐趣区

关于java:重写-Nacos-服务发现多个服务器如何跨命名空间访问公共服务

一、问题背景

在开发某个公共利用时,笔者发现该公共利用的数据是所有测试环境(假如存在 dev/dev2/dev3)通用的。

这就意味着只需部署一个利用,就能满足所有测试环境的需要;也意味着所有测试环境都须要调用该公共利用,而不同测试环境的利用注册在不同的 Nacos 命名空间。

二、两种解决方案

如果所有测试环境都须要调用该公共利用,有两种可行的计划。第一种,将该公共服务同时注册到不同的测试环境所对应的命名空间中。

第二种,将公共利用注册到独自的命名空间,不同的测试环境可能跨命名空间拜访该利用。

三、具体的问题解决过程

后行交代笔者的版本号配置。Nacos 客户端版本号为 NACOS 1.4.1;Java 我的项目的 Nacos 版本号如下。

最后想法是将该公共利用同时注册到多个命名空间下。在查找材料的过程中,团队成员在 GitHub 上发现了一篇相似问题的博客分享:Registration Center: Can services in different namespaces be called from each other? #1176。

01 注册多个命名空间

从该博客中,咱们看到其余程序员敌人也遇到了相似的公共服务的需要。在本篇文章中,笔者将进一步分享实现思路以及示例代码。

阐明:以下代码内容来自用户 chuntaojun 的分享。

shareNamespace={namespaceId[:group]},{namespaceId[:group]} 
@RunWith(SpringRunner.class)
@SpringBootTest(classes = NamingApp.class, properties = {"server.servlet.context-path=/nacos"},
    webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class SelectServiceInShareNamespace_ITCase {

    private NamingService naming1;
    private NamingService naming2;
    @LocalServerPort
    private int port;
    @Before
    public void init() throws Exception{NamingBase.prepareServer(port);
        if (naming1 == null) {Properties properties = new Properties();
            properties.setProperty(PropertyKeyConst.SERVER_ADDR, "127.0.0.1"+":"+port);
            properties.setProperty(PropertyKeyConst.SHARE_NAMESPACE, "57425802-3058-4507-9a73-3229b9f00a36");
            naming1 = NamingFactory.createNamingService(properties);

            Properties properties2 = new Properties();
            properties2.setProperty(PropertyKeyConst.SERVER_ADDR, "127.0.0.1"+":"+port);
            properties2.setProperty(PropertyKeyConst.NAMESPACE, "57425802-3058-4507-9a73-3229b9f00a36");
            naming2 = NamingFactory.createNamingService(properties2);
        }
        while (true) {if (!"UP".equals(naming1.getServerStatus())) {Thread.sleep(1000L);
                continue;
            }
            break;
        }
    }

    @Test
    public void testSelectInstanceInShareNamespaceNoGroup() throws NacosException, InterruptedException {String service1 = randomDomainName();
        String service2 = randomDomainName();
        naming1.registerInstance(service1, "127.0.0.1", 90);
        naming2.registerInstance(service2, "127.0.0.2", 90);

        Thread.sleep(1000);

        List<Instance> instances = naming1.getAllInstances(service2);
        Assert.assertEquals(1, instances.size());
        Assert.assertEquals(service2, NamingUtils.getServiceName(instances.get(0).getServiceName()));
    }

    @Test
    public void testSelectInstanceInShareNamespaceWithGroup() throws NacosException, InterruptedException {String service1 = randomDomainName();
        String service2 = randomDomainName();
        naming2.registerInstance(service1, groupName, "127.0.0.1", 90);
        naming3.registerInstance(service2, "127.0.0.2", 90);

        Thread.sleep(1000);

        List<Instance> instances = naming3.getAllInstances(service1);
        Assert.assertEquals(1, instances.size());
        Assert.assertEquals(service1, NamingUtils.getServiceName(instances.get(0).getServiceName()));
        Assert.assertEquals(groupName, NamingUtils.getServiceName(NamingUtils.getGroupName(instances.get(0).getServiceName())));
    }

}

进一步思考后发现该解决方案可能不太符合以后遇到的问题。公司目前的开发测试环境有很多个,并且不确定当前会不会持续减少。

如果每减少一个环境,都须要批改一次公共服务的配置,并且重启一次公共服务,着实太麻烦了。倒不如反其道而行,让其余的服务器实现跨命名空间拜访公共服务。

02 跨命名空间拜访

针对实际问题查找材料时,咱们找到了相似的参考分享《重写 Nacos 服务发现逻辑动静批改近程服务 IP 地址》。

跟着博客思路看代码,笔者理解到服务发现的次要相干类是 NacosNamingService, NacosDiscoveryProperties, NacosDiscoveryAutoConfiguration

而后,笔者将博客的示例代码复制过去,试着进行如下调试:

@Slf4j
@Configuration
@ConditionalOnNacosDiscoveryEnabled
@ConditionalOnProperty(name = {"spring.profiles.active"},
        havingValue = "dev"
)
@AutoConfigureBefore({NacosDiscoveryClientAutoConfiguration.class})
public class DevEnvironmentNacosDiscoveryClient {

    @Bean
    @ConditionalOnMissingBean
    public NacosDiscoveryProperties nacosProperties() {return new DevEnvironmentNacosDiscoveryProperties();
    }

    static class DevEnvironmentNacosDiscoveryProperties extends NacosDiscoveryProperties {

        private NamingService namingService;

        @Override
        public NamingService namingServiceInstance() {if (null != this.namingService) {return this.namingService;} else {Properties properties = new Properties();
                properties.put("serverAddr", super.getServerAddr());
                properties.put("namespace", super.getNamespace());
                properties.put("com.alibaba.nacos.naming.log.filename", super.getLogName());
                if (super.getEndpoint().contains(":")) {int index = super.getEndpoint().indexOf(":");
                    properties.put("endpoint", super.getEndpoint().substring(0, index));
                    properties.put("endpointPort", super.getEndpoint().substring(index + 1));
                } else {properties.put("endpoint", super.getEndpoint());
                }

                properties.put("accessKey", super.getAccessKey());
                properties.put("secretKey", super.getSecretKey());
                properties.put("clusterName", super.getClusterName());
                properties.put("namingLoadCacheAtStart", super.getNamingLoadCacheAtStart());

                try {this.namingService = new DevEnvironmentNacosNamingService(properties);
                } catch (Exception var3) {log.error("create naming service error!properties={},e=,", this, var3);
                    return null;
                }

                return this.namingService;
            }
        }

    }

    static class DevEnvironmentNacosNamingService extends NacosNamingService {public DevEnvironmentNacosNamingService(Properties properties) {super(properties);
        }

        @Override
        public List<Instance> selectInstances(String serviceName, List<String> clusters, boolean healthy) throws NacosException {List<Instance> instances = super.selectInstances(serviceName, clusters, healthy);
            instances.stream().forEach(instance -> instance.setIp("10.101.232.24"));
            return instances;
        }
    }

}

调试后发现博客提供的代码并不能满足笔者的需要,还得进一步深刻摸索。

但侥幸的是,调试过程发现 Nacos 服务发现的要害类是 com.alibaba.cloud.nacos.discovery.NacosServiceDiscovery,其中的要害办法是 getInstances()getServices(),即「返回指定服务 ID 的所有服务实例」和「获取所有服务的名称」。

也就是说,对 getInstances() 办法进行重写必定能实现本次指标——跨命名空间拜访公共服务。

/**
 * Return all instances for the given service.
 * @param serviceId id of service
 * @return list of instances
 * @throws NacosException nacosException
 */
public List<ServiceInstance> getInstances(String serviceId) throws NacosException {String group = discoveryProperties.getGroup();
        List<Instance> instances = discoveryProperties.namingServiceInstance()
                        .selectInstances(serviceId, group, true);
        return hostToServiceInstanceList(instances, serviceId);
}

/**
 * Return the names of all services.
 * @return list of service names
 * @throws NacosException nacosException
 */
public List<String> getServices() throws NacosException {String group = discoveryProperties.getGroup();
        ListView<String> services = discoveryProperties.namingServiceInstance()
                        .getServicesOfServer(1, Integer.MAX_VALUE, group);
        return services.getData();}

03 最终解决思路及代码示例

具体的解决方案思路大抵如下:

  1. 生成一个共享配置类NacosShareProperties,用来配置共享公共服务的 namespacegroup
  2. 重写配置类 NacosDiscoveryProperties(新:NacosDiscoveryPropertiesV2),将新增的共享配置类作为属性放进该配置类,后续会用到;
  3. 重写服务发现类 NacosServiceDiscovery(新:NacosServiceDiscoveryV2),这是最要害的逻辑;
  4. 重写主动配置类 NacosDiscoveryAutoConfiguration,将自定义相干类比 Nacos 原生类更早的注入容器。

最终代码中用到了一些工具类,能够自行补充残缺。

/**
 * <pre>
 *  @description: 共享 nacos 属性
 *  @author: rookie0peng
 *  @date: 2022/8/29 15:22
 *  </pre>
 */
@Configuration
@ConfigurationProperties(prefix = "nacos.share")
public class NacosShareProperties {private final Map<String, Set<String>> NAMESPACE_TO_GROUP_NAME_MAP = new ConcurrentHashMap<>();

    /**
     * 共享 nacos 实体列表
     */
    private List<NacosShareEntity> entities;

    public List<NacosShareEntity> getEntities() {return entities;}

    public void setEntities(List<NacosShareEntity> entities) {this.entities = entities;}

    public Map<String, Set<String>> getNamespaceGroupMap() {safeStream(entities).filter(entity -> nonNull(entity) && nonNull(entity.getNamespace()))
                .forEach(entity -> {Set<String> groupNames = NAMESPACE_TO_GROUP_NAME_MAP.computeIfAbsent(entity.getNamespace(), k -> new HashSet<>());
                    if (nonNull(entity.getGroupNames()))
                        groupNames.addAll(entity.getGroupNames());
                });
        return new HashMap<>(NAMESPACE_TO_GROUP_NAME_MAP);
    }

    @Override
    public String toString() {
        return "NacosShareProperties{" +
                "entities=" + entities +
                '}';
    }

    /**
     * 共享 nacos 实体
     */
    public static final class NacosShareEntity {

        /**
         * 命名空间
         */
        private String namespace;

        /**
         * 分组
         */
        private List<String> groupNames;

        public String getNamespace() {return namespace;}

        public void setNamespace(String namespace) {this.namespace = namespace;}

        public List<String> getGroupNames() {return groupNames;}

        public void setGroupNames(List<String> groupNames) {this.groupNames = groupNames;}

        @Override
        public String toString() {
            return "NacosShareEntity{" +
                    "namespace='" + namespace + '\'' +
                    ", groupNames=" + groupNames +
                    '}';
        }
    }
}
/**
 * @description: naocs 服务发现属性重写
 * @author: rookie0peng
 * @date: 2022/8/30 1:19
 */
public class NacosDiscoveryPropertiesV2 extends NacosDiscoveryProperties {private static final Logger log = LoggerFactory.getLogger(NacosDiscoveryPropertiesV2.class);

    private final NacosShareProperties nacosShareProperties;

    private static final Map<String, NamingService> NAMESPACE_TO_NAMING_SERVICE_MAP = new ConcurrentHashMap<>();

    public NacosDiscoveryPropertiesV2(NacosShareProperties nacosShareProperties) {super();
        this.nacosShareProperties = nacosShareProperties;
    }

    public Map<String, NamingService> shareNamingServiceInstances() {if (!NAMESPACE_TO_NAMING_SERVICE_MAP.isEmpty()) {return new HashMap<>(NAMESPACE_TO_NAMING_SERVICE_MAP);
        }
        List<NacosShareProperties.NacosShareEntity> entities = Optional.ofNullable(nacosShareProperties)
                .map(NacosShareProperties::getEntities).orElse(Collections.emptyList());
        entities.stream().filter(entity -> nonNull(entity) && nonNull(entity.getNamespace()))
                .filter(PredicateUtil.distinctByKey(NacosShareProperties.NacosShareEntity::getNamespace))
                .forEach(entity -> {
                    try {NamingService namingService = NacosFactory.createNamingService(getNacosProperties(entity.getNamespace()));
                        if (namingService != null) {NAMESPACE_TO_NAMING_SERVICE_MAP.put(entity.getNamespace(), namingService);
                        }
                    } catch (Exception e) {log.error("create naming service error! properties={}, e=", this, e);
                    }
                });
        return new HashMap<>(NAMESPACE_TO_NAMING_SERVICE_MAP);
    }

    private Properties getNacosProperties(String namespace) {Properties properties = new Properties();
        properties.put(SERVER_ADDR, getServerAddr());
        properties.put(USERNAME, Objects.toString(getUsername(), ""));
        properties.put(PASSWORD, Objects.toString(getPassword(), ""));
        properties.put(NAMESPACE, namespace);
        properties.put(UtilAndComs.NACOS_NAMING_LOG_NAME, getLogName());
        String endpoint = getEndpoint();
        if (endpoint.contains(":")) {int index = endpoint.indexOf(":");
            properties.put(ENDPOINT, endpoint.substring(0, index));
            properties.put(ENDPOINT_PORT, endpoint.substring(index + 1));
        }
        else {properties.put(ENDPOINT, endpoint);
        }

        properties.put(ACCESS_KEY, getAccessKey());
        properties.put(SECRET_KEY, getSecretKey());
        properties.put(CLUSTER_NAME, getClusterName());
        properties.put(NAMING_LOAD_CACHE_AT_START, getNamingLoadCacheAtStart());

//        enrichNacosDiscoveryProperties(properties);
        return properties;
    }
}
/**
 * @description: naocs 服务发现重写
 * @author: rookie0peng
 * @date: 2022/8/30 1:10
 */
public class NacosServiceDiscoveryV2 extends NacosServiceDiscovery {

    private final NacosDiscoveryPropertiesV2 discoveryProperties;

    private final NacosShareProperties nacosShareProperties;

    private final NacosServiceManager nacosServiceManager;

    public NacosServiceDiscoveryV2(NacosDiscoveryPropertiesV2 discoveryProperties, NacosShareProperties nacosShareProperties, NacosServiceManager nacosServiceManager) {super(discoveryProperties, nacosServiceManager);
        this.discoveryProperties = discoveryProperties;
        this.nacosShareProperties = nacosShareProperties;
        this.nacosServiceManager = nacosServiceManager;
    }

    /**
     * Return all instances for the given service.
     * @param serviceId id of service
     * @return list of instances
     * @throws NacosException nacosException
     */
    public List<ServiceInstance> getInstances(String serviceId) throws NacosException {String group = discoveryProperties.getGroup();
        List<Instance> instances = discoveryProperties.namingServiceInstance()
                .selectInstances(serviceId, group, true);
        if (isEmpty(instances)) {Map<String, Set<String>> namespaceGroupMap = nacosShareProperties.getNamespaceGroupMap();
            Map<String, NamingService> namespace2NamingServiceMap = discoveryProperties.shareNamingServiceInstances();
            for (Map.Entry<String, NamingService> entry : namespace2NamingServiceMap.entrySet()) {
                String namespace;
                NamingService namingService;
                if (isNull(namespace = entry.getKey()) || isNull(namingService = entry.getValue()))
                    continue;
                Set<String> groupNames = namespaceGroupMap.get(namespace);
                List<Instance> shareInstances;
                if (isEmpty(groupNames)) {shareInstances = namingService.selectInstances(serviceId, group, true);
                    if (nonEmpty(shareInstances))
                        break;
                } else {shareInstances = new ArrayList<>();
                    for (String groupName : groupNames) {List<Instance> subShareInstances = namingService.selectInstances(serviceId, groupName, true);
                        if (nonEmpty(subShareInstances)) {shareInstances.addAll(subShareInstances);
                        }
                    }
                }
                if (nonEmpty(shareInstances)) {
                    instances = shareInstances;
                    break;
                }
            }
        }
        return hostToServiceInstanceList(instances, serviceId);
    }

    /**
     * Return the names of all services.
     * @return list of service names
     * @throws NacosException nacosException
     */
    public List<String> getServices() throws NacosException {String group = discoveryProperties.getGroup();
        ListView<String> services = discoveryProperties.namingServiceInstance()
                .getServicesOfServer(1, Integer.MAX_VALUE, group);
        return services.getData();}

    public static List<ServiceInstance> hostToServiceInstanceList(List<Instance> instances, String serviceId) {List<ServiceInstance> result = new ArrayList<>(instances.size());
        for (Instance instance : instances) {ServiceInstance serviceInstance = hostToServiceInstance(instance, serviceId);
            if (serviceInstance != null) {result.add(serviceInstance);
            }
        }
        return result;
    }

    public static ServiceInstance hostToServiceInstance(Instance instance,
                                                        String serviceId) {if (instance == null || !instance.isEnabled() || !instance.isHealthy()) {return null;}
        NacosServiceInstance nacosServiceInstance = new NacosServiceInstance();
        nacosServiceInstance.setHost(instance.getIp());
        nacosServiceInstance.setPort(instance.getPort());
        nacosServiceInstance.setServiceId(serviceId);

        Map<String, String> metadata = new HashMap<>();
        metadata.put("nacos.instanceId", instance.getInstanceId());
        metadata.put("nacos.weight", instance.getWeight() + "");
        metadata.put("nacos.healthy", instance.isHealthy() + "");
        metadata.put("nacos.cluster", instance.getClusterName() + "");
        metadata.putAll(instance.getMetadata());
        nacosServiceInstance.setMetadata(metadata);

        if (metadata.containsKey("secure")) {boolean secure = Boolean.parseBoolean(metadata.get("secure"));
            nacosServiceInstance.setSecure(secure);
        }
        return nacosServiceInstance;
    }

    private NamingService namingService() {
        return nacosServiceManager
                .getNamingService(discoveryProperties.getNacosProperties());
    }
}
/**
 * @description: 重写 nacos 服务发现的主动配置
 * @author: rookie0peng
 * @date: 2022/8/30 1:08
 */
@Configuration(proxyBeanMethods = false)
@ConditionalOnDiscoveryEnabled
@ConditionalOnNacosDiscoveryEnabled
@AutoConfigureBefore({NacosDiscoveryAutoConfiguration.class})
public class NacosDiscoveryAutoConfigurationV2 {

    @Bean
    @ConditionalOnMissingBean
    public NacosDiscoveryPropertiesV2 nacosProperties(NacosShareProperties nacosShareProperties) {return new NacosDiscoveryPropertiesV2(nacosShareProperties);
    }

    @Bean
    @ConditionalOnMissingBean
    public NacosServiceDiscovery nacosServiceDiscovery(NacosDiscoveryPropertiesV2 discoveryPropertiesV2, NacosShareProperties nacosShareProperties, NacosServiceManager nacosServiceManager) {return new NacosServiceDiscoveryV2(discoveryPropertiesV2, nacosShareProperties, nacosServiceManager);
    }
}

本认为问题到这就完结了,但最初自测时发现程序基本不走 Nacos 的服务发现逻辑,而是执行 Ribbon 的负载平衡逻辑com.netflix.loadbalancer.AbstractLoadBalancerRule

不过实现类是 com.alibaba.cloud.nacos.ribbon.NacosRule,持续基于 NacosRule 重写负载平衡。

/**
 * @description: 共享 nacos 命名空间规定
 * @author: rookie0peng
 * @date: 2022/8/31 2:04
 */
public class ShareNacosNamespaceRule extends AbstractLoadBalancerRule {private static final Logger LOGGER = LoggerFactory.getLogger(ShareNacosNamespaceRule.class);

    @Autowired
    private NacosDiscoveryPropertiesV2 nacosDiscoveryPropertiesV2;
    @Autowired
    private NacosShareProperties nacosShareProperties;

    /**
     * 重写 choose 办法
     *
     * @param key
     * @return
     */
    @SneakyThrows
    @Override
    public Server choose(Object key) {
        try {String clusterName = this.nacosDiscoveryPropertiesV2.getClusterName();
            DynamicServerListLoadBalancer loadBalancer = (DynamicServerListLoadBalancer) getLoadBalancer();
            String name = loadBalancer.getName();

            NamingService namingService = nacosDiscoveryPropertiesV2
                    .namingServiceInstance();
            List<Instance> instances = namingService.selectInstances(name, true);
            if (CollectionUtils.isEmpty(instances)) {LOGGER.warn("no instance in service {}, then to get share service's instance", name);
                List<Instance> shareNamingService = this.getShareNamingService(name);
                if (nonEmpty(shareNamingService))
                    instances = shareNamingService;
                else
                    return null;
            }
            List<Instance> instancesToChoose = instances;
            if (org.apache.commons.lang3.StringUtils.isNotBlank(clusterName)) {List<Instance> sameClusterInstances = instances.stream()
                        .filter(instance -> Objects.equals(clusterName,
                                instance.getClusterName()))
                        .collect(Collectors.toList());
                if (!CollectionUtils.isEmpty(sameClusterInstances)) {instancesToChoose = sameClusterInstances;}
                else {
                    LOGGER.warn("A cross-cluster call occurs,name = {}, clusterName = {}, instance = {}",
                            name, clusterName, instances);
                }
            }

            Instance instance = ExtendBalancer.getHostByRandomWeight2(instancesToChoose);

            return new NacosServer(instance);
        }
        catch (Exception e) {LOGGER.warn("NacosRule error", e);
            return null;
        }
    }


    @Override
    public void initWithNiwsConfig(IClientConfig iClientConfig) { }

    private List<Instance> getShareNamingService(String serviceId) throws NacosException {List<Instance> instances = Collections.emptyList();
        Map<String, Set<String>> namespaceGroupMap = nacosShareProperties.getNamespaceGroupMap();
        Map<String, NamingService> namespace2NamingServiceMap = nacosDiscoveryPropertiesV2.shareNamingServiceInstances();
        for (Map.Entry<String, NamingService> entry : namespace2NamingServiceMap.entrySet()) {
            String namespace;
            NamingService namingService;
            if (isNull(namespace = entry.getKey()) || isNull(namingService = entry.getValue()))
                continue;
            Set<String> groupNames = namespaceGroupMap.get(namespace);
            List<Instance> shareInstances;
            if (isEmpty(groupNames)) {shareInstances = namingService.selectInstances(serviceId, true);
                if (nonEmpty(shareInstances))
                    break;
            } else {shareInstances = new ArrayList<>();
                for (String groupName : groupNames) {List<Instance> subShareInstances = namingService.selectInstances(serviceId, groupName, true);
                    if (nonEmpty(subShareInstances)) {shareInstances.addAll(subShareInstances);
                    }
                }
            }
            if (nonEmpty(shareInstances)) {
                instances = shareInstances;
                break;
            }
        }
        return instances;
    }
}

至此问题得以解决。

Nacos 上配置好共享 namespacegroup 后,就可能进行跨命名空间拜访了。

# nacos 共享命名空间配置 示例
nacos.share.entities[0].namespace=e6ed2017-3ed6-4d9b-824a-db626424fc7b
nacos.share.entities[0].groupNames[0]=DEFAULT_GROUP
# 指定服务应用共享的负载平衡规定,service-id 是注册到 nacos 上的服务 id,ShareNacosNamespaceRule 须要写全限定名
service-id.ribbon.NFLoadBalancerRuleClassName=***.***.***.ShareNacosNamespaceRule

留神:如果 Java 我的项目的 nacos discovery 版本用的是 2021.1,则不须要重写 Ribbon 的负载平衡类,因为该版本的 Nacos 不依赖 Ribbon。

2.2.1.RELEASE 版本 nacos discovery 依赖 Ribbon.

2021.1 版本 nacos discovery 不依赖 Ribbon。

四、总结

为了达到共享命名空间的预期,构思、查找材料、实现逻辑、调试,前后一共破费 4 天工夫。成就感满满的同时,笔者也发现该性能仍存在共享服务缓存等可优化空间,留待后续实现。

五、参考文献

[1] Registration Center: Can services in different namespaces be called from each other? [EB/OL]. https://github.com/alibaba/na…, 2019-05-07/2022-11-29.

[2] 重写 Nacos 服务发现逻辑动静批改近程服务 IP 地址 [EB/OL]. https://www.cnblogs.com/chang…, 2021-04-08/2022-11-29.

理解更多麻利开发、项目管理、行业动态等音讯,关注咱们的 sf 账号 -LigaAI~ 或者点击 LigaAI- 新一代智能研发合作平台,在线申请体验咱们的产品。

退出移动版