共计 2655 个字符,预计需要花费 7 分钟才能阅读完成。
OpenMLDB
Summary
本周合并 Pull requests 11 个,新增 Pull requests 9 个,敞开 Issues 14 个,新增 Issues 12 个。总计 57 个文件批改,新增 3002 行代码,删除 539 行代码。
Merged Pull Requests
feat: refine the description of benchmark in readme#405
feat: update the benchmark images for readme#404
feat: remove junit dependencies and covert java cases as scala#396
feat: bump hadoop-common to 2.8.5 to avoid vulnerability#388
fix: fix a bug in union with instance not in window#381
feat: add batchjob module#359
feat: add task manager module#361
feat: enhance plan optimization for group and filter#350
feat: support column query with the same name in window skew optimization#349
feat: add java common lib#347
test: modify test case#368
Open Pull Requests
feat: bump junit from 4.11 to 4.13.1 in /java/openmldb-batchjob#382
WIP : create message table#383
feat: add batch, batchjob and taskmanager as java submodules#386
fix: fix some code implementation in window skew optimization#392
feat: support aggregation over the whole table#393
feat: add integration test#395
feat: support insert multiple rows into a table using a single SQL insert statement#399
feat: add kubernetes java dependencies for taskmanager#400
fix: python test and cicd#401
Close Issues
Remove junit since of the EPL license for OpenMLDB Batch#390
Avoid vulnerability by upgrading hadoop-common dependency#387
rtidb 磁盘占用 #389
feat: engine plan optimization for where and group with the same partition#317
是否反对批量插入 #177
字段含意 #378
表应用大小 #373
Fix the issue to enable window skew optimization for window union case#374
Enable WindowSkewOpt by default and resolve the running issues#335
Add module for OpenMLDB Batch to run custom SQL and submit by TaskManager#351
Add TaskManager service to submit OpenMLDB Batch jobs#360
scripts: package java sdk set cmake type to release#372
Multiple columns with the same name can’t execute when last join and over window#356
Add java common lib#342
Open Issues
feat: improve error message system.#406
feat: support insert multiple rows into a table using a single SQL insert statement.#398
feat: api server support parameterized query#397
Add engine test on performance insensitive mode#394
Bug: SQL INSERT Statement with multi rows does not work as expected#391
Make openmldb-batchjob and openmldb-taskmanager as submodules of openmldb-parent#385
Support AWS S3 for offline data lake storage#384
Support creating database API for NearlineTablet#380
Support hive metastore for NearlineTablet#379
Support submit and manage Flink jobs for TaskManager#376
Support submit and manage Kubernetes jobs for TaskManager#375
scripts: package java sdk set cmake type to release#371
Contributors
dl239 (dl239@126.com)
tobe (tobeg3oogle@gmail.com)
Wang ZeKai (1290561498@qq.com)
wuyou10206 (zw1020688@163.com)
Highlights
本周新的三个 Java 模块,别离是 openmldb-common、openmldb-batchjob 和 openmldb-taskmanager。通过可重用的 Java 模块形象,能够疾速实现新的子模块。BatchJob 模块和 TaskManager 实现了第一个最小功能集版本,提供了根底的批处理工作治理性能。
本周通过拓展 SQL 语法解析器和物理打算优化器,实现了在线执行引擎的全表分组聚合和过滤性能,此性能难点在于须要在 SQL 语法解析阶段就辨别 SQL 中的 UDF 和 UDAF 函数,并最终生成对应的逻辑打算和优化后的物理打算。
欢送更多开发者关注和参加 OpenMLDB 开源我的项目。