关于数据库:CoralCache一个提高微服务可用性的中间件

30次阅读

共计 8060 个字符，预计需要花费 21 分钟才能阅读完成。

摘要 ：当数据库出问题时能降级从本地缓存的数据中查问数据，CoralCache 就是这样一个进步微服务可用性的中间件。

背景

有些场景下，微服务依赖数据库中一些配置项或者数量很少的数据，但当数据库自身有问题时候，即便数据量很少，这个服务是不能失常工作；因而须要思考一种能反对全量 + 极少变更的全局数据的场景，当数据库出问题时能降级从本地缓存的数据中查问数据，CoralCache就是这样一个进步微服务可用性的中间件。

架构

CoralCache 中间件架构如下图所示，通过 @EnableLocal 注解开启性能，利用启动后将配置的表数据一次性加载到内存中，内存中的数据逻辑构造和数据库中的逻辑构造一样。

图 1. 架构图

表达式计算引擎

内存查问引擎的原理是数据库查问降级产生后，Intercepter 将拦挡到的原始 SQL 传入查问引擎中，查问引擎解析 SQL 后失去表名、列名、where 条件表达式，遍历 InnerDB 中对应表的数据行，并通过表达式计算引擎计算结果，计算结果为真则增加到后果集中最初返回给调用方。

计算引擎构造如下图所示，将 where 条件表达式转为后缀表达式后顺次遍历后缀表达式，遇到 操作数 间接入栈，遇到 操作符 则依据操作符须要的操作数个数弹栈。

图 2. 表达式计算引擎构造

而后依据操作符和弹出的操作数进行计算，不同操作符对应不同的计算方法，并将计算后的后果从新作为操作数入栈执到遍历实现，外围计算流程代码如下所示：

public Object calc(Expression where, InnerTable table, InnerRow row) {
        try {postTraversal(where);
        } catch (Exception e) {log.warn("calc error: {}", e.getMessage());
            return false;
        }
        for (ExprObj obj : exprList) {switch (obj.exprType()) {
                case ITEM:
                    stack.push(obj);
                    break;
                case BINARY_OP: {ExprObj result = calcBinaryOperation(((ExprOperation) obj).getOperationType(), table, row);
                    stack.push(result);
                    break;
                }
                case UNARY_OP: {ExprObj result = calcSingleOperation(((ExprOperation) obj).getOperationType(), table, row);
                    stack.push(result);
                    break;
                }
                case FUNCTION_OP: {ExprObj result = calcFunctionOperation(((ExprOperation) obj).getOperationType(), table, row);
                    stack.push(result);
                    break;
                }
                default:
                    break;
            }
        }
        return stack.pop();}

常见运算符的实现

逻辑运算

逻辑常见运算符为 <、<=、>、>=、= 等，它们的共性都是须要 2 个操作数并且返回值是布尔类型。

public ExprItem logicalCalculus(InnerTable table, InnerRow row, LogicalOperation logicalOperation) {ExprObj second = stack.pop();
        ExprObj first = stack.pop();

        ExprItem result = new ExprItem();
        result.setItemType(ItemType.T_CONST_OBJ);
        Obj firstObj = getObj((ExprItem) first, table, row);
        Obj secondObj = getObj((ExprItem) second, table, row);
        boolean value = logicalOperation.apply(firstObj, secondObj);
        result.setValue(new Obj(value, ObjType.BOOL));
        return result;
    }

例子，以 ”=” 的实现来展现：

private ExprObj calcBinaryOperation(OperationType type, InnerTable table, InnerRow row) {
        ExprObj result = null;
        switch (type) {
            case T_OP_EQ:
                result = logicalCalculus(table, row, (a, b) -> ObjUtil.eq(a, b)); // 等于符号的实现
                break;
            ...
            default:
                break;
        }
        return result;
 }

public class ObjUtil {private static ObjType resultType(ObjType first, ObjType second) {return ObjType.RESULT_TYPE[first.ordinal()][second.ordinal()];
    }

    public static boolean eq(Obj first, Obj second) {ObjType type = resultType(first.getType(), second.getType());

        switch (type) {
            case LONG: {long firstValue = first.getValueAsLong();
                long secondValue = second.getValueAsLong();
                return firstValue == secondValue;
            }
            case DOUBLE: {double firstValue = first.getValueAsDouble();
                double secondValue = second.getValueAsDouble();
                return Double.compare(firstValue, secondValue) == 0;
            }
            case TIMESTAMP: {java.util.Date firstValue = first.getValueAsDate();
                java.util.Date secondValue = first.getValueAsDate();
                return firstValue.compareTo(secondValue) == 0;
            }
            ...
            default:
                break;
        }
        throw new UnsupportedOperationException(first.getType() + "and" + second.getType() + "not support'='operation.");
    }
}

数学运算

数学运算和逻辑运算的流程都一样，只不过运算后的后果为数字类型。

LIKE 运算符

除了下面说的逻辑运算和数学运算外，还反对进行含糊匹配的非凡操作符LIKE。

LIKE 表达式语法

常见用法如下

LIKE “%HUAWEI” 匹配以 HUAWEI 结尾的字符串
LIKE “HUAWEI%” 匹配以 HUAWEI 结尾的字符串
LIKE “A_B” 匹配以 ”A” 起头且以 ”Z” 为结尾的字串
LIKE “A?B” 同上
LIKE “%[0-9]%” 匹配含有数字的字符串
LIKE “%[a-z]%” 匹配含有小写字母字符串
LIKE “%[!0-9]%” 匹配不含数字的字符串
? 和_都示意单个字符

JAVA 中实现 LIKE 的计划：将 LIKE 的模式转为 JAVA 中的正则表达式。

LIKE 词法定义

expr := wild-card + expr
      | wild-char + expr
      | escape + expr
      | string + expr
      | ""

wild-card := %  
wild-char := _  
escape := [%|_]  
string := [^%_]+ (One or > more characters that are not wild-card or wild-char)

定义 Token 类

public abstract class Token {
    private final String value;

    public Token(String value) {this.value = value;}

    public abstract String convert();

    public String getValue() {return value;}
}

public class ConstantToken extends Token {public ConstantToken(String value) {super(value);
    }

    @Override
    public String convert() {return getValue();
    }
}

public class EscapeToken extends Token {public EscapeToken(String value) {super(value);
    }

    @Override
    public String convert() {return getValue();
    }
}

public class StringToken extends Token {public StringToken(String value) {super(value);
    }

    @Override
    public String convert() {return Pattern.quote(getValue());
    }
}

public class WildcardToken extends Token {public WildcardToken(String value) {super(value);
    }

    @Override
    public String convert() {return ".*";}
}

public class WildcharToken extends Token {public WildcharToken(String value) {super(value);
    }

    @Override
    public String convert() {return ".";}
}

创立 Lexer(Tokenizer)

public class Tokenizer {private Collection<Tuple> patterns = new LinkedList<>();

    public <T extends Token> Tokenizer add(String regex, Function<String, Token> creator) {this.patterns.add(new Tuple<Pattern, Function<String, Token>>(Pattern.compile(regex), creator));
        return this;
    }

    public Collection<Token> tokenize(String clause) throws RuntimeException {Collection<Token> tokens = new ArrayList<>();
        String copy = String.copyValueOf(clause.toCharArray());

        int position = 0;
        while (!copy.equals("")) {
            boolean found = false;
            for (Tuple tuple : this.patterns) {Pattern pattern = (Pattern) tuple.getFirst();
                Matcher m = pattern.matcher(copy);
                if (m.find()) {
                    found = true;
                    String token = m.group(1);
                    Function<String, Token> fn = (Function<String, Token>) tuple.getSecond();
                    tokens.add(fn.apply(token));
                    copy = m.replaceFirst("");
                    position += token.length();
                    break;
                }
            }

            if (!found) {throw new RuntimeException("Unexpected sequence found in input string, at" + position);
            }
        }

        return tokens;

    }
}

创立 LIKE 到正则表达式的转换映射

public class LikeTranspiler {private static Tokenizer TOKENIZER = new Tokenizer()
            .add("^([[^]]*])", ConstantToken::new)
            .add("^(%)", WildcardToken::new)
            .add("^(_)", WildcharToken::new)
            .add("^([^[]%_]+)", StringToken::new);

    public static String toRegEx(String pattern) throws ParseException {StringBuilder sb = new StringBuilder().append("^");
        for (Token token : TOKENIZER.tokenize(pattern)) {sb.append(token.convert());
        }

        return sb.append("$").toString();}
}

间接调用 LikeTranspiler 的 toRegEx 办法将 LIKE 语法转为 JAVA 中的正则表达式。

private ExprObj calcBinaryOperation(OperationType type, InnerTable table, InnerRow row) {
        ExprObj result = null;
        switch (type) {
            . . .
            case T_OP_LIKE:
                result = logicalCalculus(table, row, (a, b) -> ObjUtil.like(a, b));
                break;
            . . .
        }

        return result;
    }
public static boolean like(Obj first, Obj second) {Assert.state(first.getType() == ObjType.STRING, OperationType.T_OP_LIKE + "only support STRING.");
        Assert.state(second.getType() == ObjType.STRING, OperationType.T_OP_LIKE + "only support STRING.");

        String firstValue = (String) first.getRelValue();

        String secondValue = (String) second.getRelValue();

        String regEx = LikeTranspiler.toRegEx(secondValue);

        return Pattern.compile(regEx).matcher(firstValue).matches();}

通过创立词法分析器并应用此办法进行转换，咱们能够避免 LIKE 像这样的子句被转换为正则表达式 %abc[%]%，该子句应将其中的任何子字符串与其中的子字符串匹配，该子句将与子字符串或匹配任何字符串。abc%.abc[.].abc.abc。

类型计算转换

不同数据类型在进行计算时须要转型，具体的转化入下二维数组中。

// 不同类型计算后的类型
ObjType[][] RESULT_TYPE = {
        //UNKNOWN  BYTE     SHORT    INT      LONG     FLOAT    DOUBLE   DECIMAL  BOOL     DATE       TIME       TIMESTAMP  STRING     NULL
        {UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN,   UNKNOWN,   UNKNOWN,   UNKNOWN,   UNKNOWN},// UNKNOWN
        {UNKNOWN, LONG,    LONG,    LONG,    LONG,    DOUBLE,  DOUBLE,  DECIMAL, BOOL,    UNKNOWN,   UNKNOWN,   UNKNOWN,   LONG,      UNKNOWN},// BYTE
        {UNKNOWN, LONG,    LONG,    LONG,    LONG,    DOUBLE,  DOUBLE,  DECIMAL, BOOL,    UNKNOWN,   UNKNOWN,   UNKNOWN,   LONG,      UNKNOWN},// SHORT
        {UNKNOWN, LONG,    LONG,    LONG,    LONG,    DOUBLE,  DOUBLE,  DECIMAL, BOOL,    UNKNOWN,   UNKNOWN,   UNKNOWN,   LONG,      UNKNOWN},// INT
        {UNKNOWN, LONG,    LONG,    LONG,    LONG,    DOUBLE,  DOUBLE,  DECIMAL, BOOL,    UNKNOWN,   UNKNOWN,   UNKNOWN,   LONG,      UNKNOWN},// LONG
        {UNKNOWN, DOUBLE,  DOUBLE,  DOUBLE,  DOUBLE,  DOUBLE,  DOUBLE,  DECIMAL, BOOL,    UNKNOWN,   UNKNOWN,   UNKNOWN,   DOUBLE,    UNKNOWN},// FLOAT
        {UNKNOWN, DOUBLE,  DOUBLE,  DOUBLE,  DOUBLE,  DOUBLE,  DOUBLE,  DECIMAL, BOOL,    UNKNOWN,   UNKNOWN,   UNKNOWN,   DOUBLE,    UNKNOWN},// DOUBLE
        {UNKNOWN, DECIMAL, DECIMAL, DECIMAL, DECIMAL, DECIMAL, DECIMAL, DECIMAL, UNKNOWN, UNKNOWN,   UNKNOWN,   UNKNOWN,   DECIMAL,   UNKNOWN},// DECIMAL
        {UNKNOWN, BOOL,    BOOL,    BOOL,    BOOL,    BOOL,    BOOL,    BOOL,    BOOL,    UNKNOWN,   UNKNOWN,   UNKNOWN,   BOOL,      UNKNOWN},// BOOL
        {UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, TIMESTAMP, TIMESTAMP, TIMESTAMP, TIMESTAMP, UNKNOWN},// DATE
        {UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, TIMESTAMP, TIMESTAMP, TIMESTAMP, TIMESTAMP, UNKNOWN},// TIME
        {UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, TIMESTAMP, TIMESTAMP, TIMESTAMP, TIMESTAMP, UNKNOWN},// TIMESTAMP
        {UNKNOWN, LONG,    LONG,    LONG,    LONG,    DOUBLE,  DOUBLE,  DECIMAL, BOOL,    TIMESTAMP, TIMESTAMP, TIMESTAMP, STRING,    UNKNOWN},// STRING
        {UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN, UNKNOWN,   UNKNOWN,   UNKNOWN,   UNKNOWN,   UNKNOWN},// NULL
};

参考资料

[1] https://codereview.stackexcha…

本文分享自华为云社区《微服务缓存中间件 CoralCache 表达式计算引擎详解》，原文作者：超纯的小白兔。

点击关注，第一工夫理解华为云陈腐技术~

正文完
数据库
发表至： 数据库
2021-02-19
 0
关于数据库:墨天轮2022年度数据库获奖名单
关于数据库:腾讯云TDSQL-MySQL版-开发指南-分布式事务
关于数据库:先行蜀道-四川农信核心系统分布式转型
关于数据库:我们有新名字啦