介绍
本期配角:ShardingCore 一款 ef-core 下高性能、轻量级针对分表分库读写拆散的解决方案，具备零依赖、零学习老本、零业务代码入侵
dotnet 下惟一一款全自动分表, 多字段分表框架, 领有高性能, 零依赖、零学习老本、零业务代码入侵, 并且反对读写拆散动静分表分库, 同一种路由能够齐全自定义的新星组件, 通过本框架你岂但能够学到很多分片的思维和技巧, 并且更能学到 Expression 的奇思妙用
你的 star 和点赞是我坚持下去的最大能源, 一起为.net 生态提供更好的解决方案
我的项目地址
- github 地址 https://github.com/xuejmnet/s…
- gitee 地址 https://gitee.com/dotnetchina…
背景
因为之前有小伙伴在应用 ShardingCore 的时候问过我是否能够利用 ShardingCore 的分库性能实现多租户呢，我的答复是能够的，然而须要针对分库对象进行路由的编写，相当于我一个我的项目须要实现多租户所有的表都须要实现分库才能够，那么这个在理论利用中将是不切实际的，所以尽管分库能够用来进行多租户然而个别没人会真的这样操作，那么就没有方法在 ShardingCore 应用正当的多租户外加分表分库了吗，针对这个问题 ShardingCore 在新的版本 x.4.x.x+ 中进行了实现
性能
ShardingCorex.4.x.x+ 版本中具体实现了哪些性能呢
- 多配置反对, 能够针对每个租户或者这个配置进行独自的分表分库读写拆散的链接配置
- 多数据库配置，反对多配置下每个配置都能够领有本人的数据库来进行分表分库读写拆散
- 动静多配置，反对动静增加多配置(目前不反对动静删减多配置, 后续会反对如果有须要)
场景
假如咱们有这么一个多租户零碎，这个零碎在咱们创立好账号后会调配给咱们一个独自的数据库和对应的表信息, 之后用户能够利用这个租户配置信息进行操作解决
首先咱们创立一个 AspNetCore 的我的项目
这边才用的.Net6 版本的 webapi
增加依赖
这边咱们增加了三个包, 别离是 ShardingCore,Microsoft.EntityFrameworkCore.SqlServer,Pomelo.EntityFrameworkCore.MySql, 其中ShardingCore 用的是预览版的如果不勾选那么将无奈显示进去, 为什么咱们须要增加额定的两个数据库驱动呢, 起因是因为咱们须要在不同的租户下实现不同的数据库的配置, 比方租户 A 和咱们签订的协定外面有阐明零碎应用开源数据库, 或者心愿应用 Linux 平台那么能够针对租户 A 进行配置 MySql 或者 PgSql, 租户 B 是资深软粉说须要应用MSSQL 那么就能够针对其配置MSSQL. 个别状况下咱们可能不会呈现多数据库的状况然而为了关照到非凡状况咱们这边也针对这种状况进行了反对。
公共用户存储
首先在我还没有创立租户的时候是不存在数据库的所以我的数据自然而然不会存在以后租户下, 这边咱们采纳的是存储到其余数据库中, 假如咱们应用一个公共的数据库作为用户零碎.
创立用户零碎
创立零碎用户和创立零碎用户在数据库内的映射关系
public class SysUser
{public string Id { get; set;}
public string Name {get; set;}
public string Password {get; set;}
public DateTime CreationTime {get; set;}
public bool IsDeleted {get; set;}
}
public class SysUserMap:IEntityTypeConfiguration<SysUser>
{public void Configure(EntityTypeBuilder<SysUser> builder)
{builder.HasKey(o => o.Id);
builder.Property(o => o.Id).IsRequired().IsUnicode(false).HasMaxLength(50);
builder.Property(o => o.Name).IsRequired().HasMaxLength(50);
builder.Property(o => o.Password).IsRequired().IsUnicode(false).HasMaxLength(50);
builder.HasQueryFilter(o => o.IsDeleted == false);
builder.ToTable(nameof(SysUser));
}
}创立这个数据库该有的配置信息表，便于前期启动后重建
public class SysUserTenantConfig
{public string Id { get; set;}
public string UserId {get; set;}
/// <summary>
/// 增加 ShardingCore 配置的 Json 包
/// </summary>
public string ConfigJson {get; set;}
public DateTime CreationTime {get; set;}
public bool IsDeleted {get; set;}
}
public class SysUserTenantConfigMap:IEntityTypeConfiguration<SysUserTenantConfig>
{public void Configure(EntityTypeBuilder<SysUserTenantConfig> builder)
{builder.HasKey(o => o.Id);
builder.Property(o => o.Id).IsRequired().IsUnicode(false).HasMaxLength(50);
builder.Property(o => o.UserId).IsRequired().IsUnicode(false).HasMaxLength(50);
builder.Property(o => o.ConfigJson).IsRequired().HasMaxLength(2000);
builder.HasQueryFilter(o => o.IsDeleted == false);
builder.ToTable(nameof(SysUserTenantConfig));
}
}创立对应的零碎用户存储 DbContext
public class IdentityDbContext:DbContext
{public IdentityDbContext(DbContextOptions<IdentityDbContext> options):base(options)
{ }
protected override void OnModelCreating(ModelBuilder modelBuilder)
{base.OnModelCreating(modelBuilder);
modelBuilder.ApplyConfiguration(new SysUserMap());
modelBuilder.ApplyConfiguration(new SysUserTenantConfigMap());
}
}创立一个租户的 DbContext
public class TenantDbContext:AbstractShardingDbContext,IShardingTableDbContext
{public TenantDbContext(DbContextOptions<TenantDbContext> options) : base(options)
{ }
public IRouteTail RouteTail {get; set;}
}目前咱们先定义好后续进行编写外部的租户代码
创立动静租户参数
动静租户分片配置信息在 ShardingCore 只须要实现 IVirtualDataSourceConfigurationParams<TShardingDbContext> 接口, 然而这个接口有很多参数须要填写, 所以这边框架针对这个接口进行了默认参数的抽象类 AbstractVirtualDataSourceConfigurationParams<TShardingDbContext>。
这边咱们针对配置参数进行配置采纳新建一个配置 json 的对象
public class ShardingTenantOptions
{public string ConfigId { get; set;}
public int Priority {get; set;}
public string DefaultDataSourceName {get; set;}
public string DefaultConnectionString {get; set;}
public DbTypeEnum DbType {get; set;}
}参数外面配置了以后数据库, 这边比较简单咱们就临时应用单表分库的模式来实现, 目前临时不对每个租户分库进行演示。之后并且编写 SqlServer 和MySql的配置反对
public class SqlShardingConfiguration : AbstractVirtualDataSourceConfigurationParams<TenantDbContext>
{
private static readonly ILoggerFactory efLogger = LoggerFactory.Create(builder =>
{builder.AddFilter((category, level) => category == DbLoggerCategory.Database.Command.Name && level == LogLevel.Information).AddConsole();});
public override string ConfigId {get;}
public override int Priority {get;}
public override string DefaultDataSourceName {get;}
public override string DefaultConnectionString {get;}
public override ITableEnsureManager TableEnsureManager {get;}
private readonly DbTypeEnum _dbType;
public SqlShardingConfiguration(ShardingTenantOptions options)
{
ConfigId = options.ConfigId;
Priority = options.Priority;
DefaultDataSourceName = options.DefaultDataSourceName;
DefaultConnectionString = options.DefaultConnectionString;
_dbType = options.DbType;
// 用来疾速判断是否存在数据库中的表
if (_dbType == DbTypeEnum.MSSQL)
{TableEnsureManager = new SqlServerTableEnsureManager<TenantDbContext>();
}
else if (_dbType == DbTypeEnum.MYSQL)
{TableEnsureManager = new MySqlTableEnsureManager<TenantDbContext>();
}
else
{throw new NotImplementedException();
}
}
public override DbContextOptionsBuilder UseDbContextOptionsBuilder(string connectionString,
DbContextOptionsBuilder dbContextOptionsBuilder)
{switch (_dbType)
{
case DbTypeEnum.MSSQL:
{dbContextOptionsBuilder.UseSqlServer(connectionString).UseLoggerFactory(efLogger);
}
break;
case DbTypeEnum.MYSQL:
{dbContextOptionsBuilder.UseMySql(connectionString, new MySqlServerVersion(new Version())).UseLoggerFactory(efLogger);
}
break;
default: throw new NotImplementedException();}
return dbContextOptionsBuilder;
}
public override DbContextOptionsBuilder UseDbContextOptionsBuilder(DbConnection dbConnection,
DbContextOptionsBuilder dbContextOptionsBuilder)
{switch (_dbType)
{
case DbTypeEnum.MSSQL:
{dbContextOptionsBuilder.UseSqlServer(dbConnection).UseLoggerFactory(efLogger);
}
break;
case DbTypeEnum.MYSQL:
{dbContextOptionsBuilder.UseMySql(dbConnection, new MySqlServerVersion(new Version())).UseLoggerFactory(efLogger);
}
break;
default: throw new NotImplementedException();}
return dbContextOptionsBuilder;
}
}编写用户注册接口
[Route("api/[controller]/[action]")]
[ApiController]
[AllowAnonymous]
public class PassportController:ControllerBase
{
private readonly IdentityDbContext _identityDbContext;
public PassportController(IdentityDbContext identityDbContext)
{_identityDbContext = identityDbContext;}
[HttpPost]
public async Task<IActionResult> Register(RegisterRequest request)
{if (await _identityDbContext.Set<SysUser>().AnyAsync(o => o.Name == request.Name))
return BadRequest("user not exists");
var sysUser = new SysUser()
{Id = Guid.NewGuid().ToString("n"),
Name = request.Name,
Password = request.Password,
CreationTime=DateTime.Now
};
var shardingTenantOptions = new ShardingTenantOptions()
{
ConfigId = sysUser.Id,
Priority = new Random().Next(1,10),
DbType = request.DbType,
DefaultDataSourceName = "ds0",
DefaultConnectionString = GetDefaultString(request.DbType,sysUser.Id)
};
var sysUserTenantConfig = new SysUserTenantConfig()
{Id = Guid.NewGuid().ToString("n"),
UserId = sysUser.Id,
CreationTime = DateTime.Now,
ConfigJson = JsonConvert.SerializeObject(shardingTenantOptions)
};
await _identityDbContext.AddAsync(sysUser);
await _identityDbContext.AddAsync(sysUserTenantConfig);
await _identityDbContext.SaveChangesAsync();
// 注册实现后进行配置生成
DynamicShardingHelper.DynamicAppendVirtualDataSourceConfig(new SqlShardingConfiguration(shardingTenantOptions));
return Ok();}
[HttpPost]
public async Task<IActionResult> Login(LoginRequest request)
{var sysUser = await _identityDbContext.Set<SysUser>().FirstOrDefaultAsync(o=>o.Name==request.Name&&o.Password==request.Password);
if (sysUser == null)
return BadRequest("name or password error");
// 秘钥，就是标头，这里用 Hmacsha256 算法，须要 256bit 的密钥
var securityKey = new SigningCredentials(new SymmetricSecurityKey(Encoding.ASCII.GetBytes("123123!@#!@#123123")), SecurityAlgorithms.HmacSha256);
//Claim，JwtRegisteredClaimNames 中预约义了好多种默认的参数名，也能够像上面的 Guid 一样本人定义键名.
//ClaimTypes 也预约义了好多类型如 role、email、name。Role 用于赋予权限，不同的角色能够拜访不同的接口
// 相当于有效载荷
var claims = new Claim[] {new Claim(JwtRegisteredClaimNames.Iss,"https://localhost:5000"),
new Claim(JwtRegisteredClaimNames.Aud,"api"),
new Claim("id",Guid.NewGuid().ToString("n")),
new Claim("uid",sysUser.Id),
};
SecurityToken securityToken = new JwtSecurityToken(
signingCredentials: securityKey,
expires: DateTime.Now.AddHours(2),// 过期工夫
claims: claims
);
var token = new JwtSecurityTokenHandler().WriteToken(securityToken);
return Ok(token);
}
private string GetDefaultString(DbTypeEnum dbType, string userId)
{switch (dbType)
{case DbTypeEnum.MSSQL: return $"Data Source=localhost;Initial Catalog=DB{userId};Integrated Security=True;";
case DbTypeEnum.MYSQL: return $"server=127.0.0.1;port=3306;database=DB{userId};userid=root;password=L6yBtV6qNENrwBy7;";
default: throw new NotImplementedException();}
}
}
public class RegisterRequest
{public string Name { get; set;}
public string Password {get; set;}
public DbTypeEnum DbType {get; set;}
}
public class LoginRequest
{public string Name { get; set;}
public string Password {get; set;}
}简略来阐明一下, 这边咱们采纳的是用户的 id 作为租户 id, 将租户 id 作为数据库配置, 来反对多配置模式。到此为止咱们的用户零碎就曾经实现了是不是非常的简略仅仅几段代码, 用户这边注册实现后将会创立对应的数据库和对应的表, 如果你是分表的那么将会主动创立对应的数据库表等信息。
租户零碎
租户零碎咱们做一个订单的简略演示, 应用订单 id 取模, 取模取 5 来进行分表操作
新增租户零碎的订单信息
public class Order
{public string Id { get; set;}
public string Name {get; set;}
public DateTime CreationTime {get; set;}
public bool IsDeleted {get; set;}
}
public class OrderMap:IEntityTypeConfiguration<Order>
{public void Configure(EntityTypeBuilder<Order> builder)
{builder.HasKey(o => o.Id);
builder.Property(o => o.Id).IsRequired().IsUnicode(false).HasMaxLength(50);
builder.Property(o => o.Name).IsRequired().HasMaxLength(100);
builder.HasQueryFilter(o => o.IsDeleted == false);
builder.ToTable(nameof(Order));
}
}新增订单路由
public class OrderVirtualTableRoute:AbstractSimpleShardingModKeyStringVirtualTableRoute<Order>
{public OrderVirtualTableRoute() : base(2, 5)
{ }
public override void Configure(EntityMetadataTableBuilder<Order> builder)
{builder.ShardingProperty(o => o.Id);
}
}简略的字符串取模
增加租户中间件
增加租户中间件, 在零碎中如果应用多配置那么就必须要指定本次创立的 dbcontext 应用的是哪个配置
public class TenantSelectMiddleware
{
private readonly RequestDelegate _next;
private readonly IVirtualDataSourceManager<TenantDbContext> _virtualDataSourceManager;
public TenantSelectMiddleware(RequestDelegate next, IVirtualDataSourceManager<TenantDbContext> virtualDataSourceManager)
{
_next = next;
_virtualDataSourceManager = virtualDataSourceManager;
}
public async Task Invoke(HttpContext context)
{if (context.Request.Path.ToString().StartsWith("/api/tenant", StringComparison.CurrentCultureIgnoreCase))
{if (!context.User.Identity.IsAuthenticated)
{await _next(context);
return;
}
var tenantId = context.User.Claims.FirstOrDefault((o) => o.Type == "uid")?.Value;
if (string.IsNullOrWhiteSpace(tenantId))
{await DoUnAuthorized(context, "not found tenant id");
return;
}
using (_virtualDataSourceManager.CreateScope(tenantId))
{await _next(context);
}
}
else
{await _next(context);
}
}
private async Task DoUnAuthorized(HttpContext context, string msg)
{
context.Response.StatusCode = 403;
await context.Response.WriteAsync(msg);
}
}该中间件拦挡 /api/tenant 门路下的所有申请并且针对这些申请增加对应的租户信息
配置租户扩大初始化数据
public static class TenantExtension
{public static void InitTenant(this IServiceProvider serviceProvider)
{using (var scope = serviceProvider.CreateScope())
{var identityDbContext = scope.ServiceProvider.GetRequiredService<IdentityDbContext>();
identityDbContext.Database.EnsureCreated();
var sysUserTenantConfigs = identityDbContext.Set<SysUserTenantConfig>().ToList();
if (sysUserTenantConfigs.Any())
{foreach (var sysUserTenantConfig in sysUserTenantConfigs)
{var shardingTenantOptions = JsonConvert.DeserializeObject<ShardingTenantOptions>(sysUserTenantConfig.ConfigJson);
DynamicShardingHelper.DynamicAppendVirtualDataSourceConfig(new SqlShardingConfiguration(shardingTenantOptions));
}
}
}
}
}这边因为咱们针对租户信息进行了初始化而不是硬编码, 所以须要一个在启动的时候对租户信息进行动静增加
配置多租户
启动配置Startup
var builder = WebApplication.CreateBuilder(args);
// Add services to the container.
builder.Services.AddControllers();
builder.Services.AddAuthentication();
#region 用户系统配置
builder.Services.AddDbContext<IdentityDbContext>(o =>
o.UseSqlServer("Data Source=localhost;Initial Catalog=IdDb;Integrated Security=True;"));
// 生成密钥
var keyByteArray = Encoding.ASCII.GetBytes("123123!@#!@#123123");
var signingKey = new SymmetricSecurityKey(keyByteArray);
// 认证参数
builder.Services.AddAuthentication("Bearer")
.AddJwtBearer(o =>
{
o.TokenValidationParameters = new TokenValidationParameters
{
ValidateIssuerSigningKey = true,
IssuerSigningKey = signingKey,
ValidateIssuer = true,
ValidIssuer = "https://localhost:5000",
ValidateAudience = true,
ValidAudience = "api",
ValidateLifetime = true,
ClockSkew = TimeSpan.Zero,
RequireExpirationTime = true,
};
});
#endregion
#region 配置 ShardingCore
builder.Services.AddShardingDbContext<TenantDbContext>()
.AddEntityConfig(op =>
{
op.CreateShardingTableOnStart = true;
op.EnsureCreatedWithOutShardingTable = true;
op.AddShardingTableRoute<OrderVirtualTableRoute>();})
.AddConfig(op =>
{
// 默认配置一个
op.ConfigId = $"test_{Guid.NewGuid():n}";
op.Priority = 99999;
op.AddDefaultDataSource("ds0", "Data Source=localhost;Initial Catalog=TestTenantDb;Integrated Security=True;");
op.UseShardingQuery((conStr, b) =>
{b.UseSqlServer(conStr);
});
op.UseShardingTransaction((conn, b) =>
{b.UseSqlServer(conn);
});
}).EnsureMultiConfig(ShardingConfigurationStrategyEnum.ThrowIfNull);
#endregion
var app = builder.Build();
// Configure the HTTP request pipeline.
app.Services.GetRequiredService<IShardingBootstrapper>().Start();
// 初始化启动配置租户信息
app.Services.InitTenant();
app.UseAuthorization();
app.UseAuthorization();
// 在认证后启用租户抉择中间件
app.UseMiddleware<TenantSelectMiddleware>();
app.MapControllers();
app.Run();
编写租户操作
[Route("api/tenant/[controller]/[action]")]
[ApiController]
[Authorize(AuthenticationSchemes = "Bearer")]
public class TenantController : ControllerBase
{
private readonly TenantDbContext _tenantDbContext;
public TenantController(TenantDbContext tenantDbContext)
{_tenantDbContext = tenantDbContext;}
public async Task<IActionResult> AddOrder()
{var order = new Order()
{Id = Guid.NewGuid().ToString("n"),
CreationTime = DateTime.Now,
Name = new Random().Next(1,100)+"_name"
};
await _tenantDbContext.AddAsync(order);
await _tenantDbContext.SaveChangesAsync();
return Ok(order.Id);
}
public async Task<IActionResult> UpdateOrder([FromQuery]string id)
{var order =await _tenantDbContext.Set<Order>().FirstOrDefaultAsync(o=>o.Id==id);
if (order == null) return BadRequest();
order.Name = new Random().Next(1, 100) + "_name";
await _tenantDbContext.SaveChangesAsync();
return Ok(order.Id);
}
public async Task<IActionResult> GetOrders()
{var orders =await _tenantDbContext.Set<Order>().ToListAsync();
return Ok(orders);
}
}启动我的项目
这边咱们基本上曾经配置好咱们所须要的之后咱们就能够间接启动我的项目了
这边咱们通过接口注册了一个 TenantA 的用户并且抉择了应用 MSSQL, 这边成就帮咱们主动生成好了对应的数据库表构造
接下来我么再注册一个 TenantB 用户抉择 MySql
通过截图咱们能够看到 ShardingCore 也是为咱们创立好了对应的数据库和对应的表信息
登录租户
首先咱们登录
TenantA 用户 token
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJodHRwczovL2xvY2FsaG9zdDo1MDAwIiwiYXVkIjoiYXBpIiwiaWQiOiJkNGMwZjZiNzI5MzE0M2VlYWM0Yjg3NzUwYzE4MWUzOSIsInVpZCI6ImMxMWRkZjFmNTY0MjQwZjc5YTQzNTEzZGMwNmVjZGMxIiwiZXhwIjoxNjQxODI4ODQ0fQ.zJefwnmcIEZm-kizlN7DhwTRgGxiCg52Esa8QmHiEKYTenantB 用户 token
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJodHRwczovL2xvY2FsaG9zdDo1MDAwIiwiYXVkIjoiYXBpIiwiaWQiOiIwNzY4NzUwMmVjYzY0NTMyOGFkNTcwZDRkYjMwNDI3MSIsInVpZCI6ImVkODg4YTc3MzAwYTQ4NjZhYmUyNWY2MTE1NmEwZTQzIiwiZXhwIjoxNjQxODI4ODgxfQ.cL0d010jdXLXNGT8M0wsRMqn3VeIxFnV0keM0H3SPzo接下来咱们别离对两个租户进行穿插解决
AddOrder
租户 A 插入一个订单, 订单 Id：aef6905f512a4f72baac5f149ef32d21
TenantB 用户也插入一个订单, 订单 id:450f5dd0e82442eca33dfcf3d57fafa3
两个用户解决
通过日志打印显著可能感觉进去两者是辨别了不同的数据库
UpdateOrder
GetOrders
总结
通过上述性能的演示置信很多小伙伴应该曾经晓得他具体的运作流程了, 通过配置多个租户信息, 在 ShardingCore 上实现多配置, 动静配置, 来保障在多租户模式下的分表分库读写拆散仍然能够应用, 并且领有跟好的适泛性。
如果你须要开发一个大型程序, 领导上来就是分库分表, 那么在以前大概率是会破费十分多的精力在解决分片这件事件上, 而最终我的项目是否能够做完并且应用还是一个微小的问题, 然而当初不一样了, 毕竟 ShardingCore 之前并没有一款十分好用的分片组件在.net 上, 并且领有十分完满的 orm 作为反对，基本上重来没有一个框架说多租户模式是能够抉择数据库的, 之前市面上所有的多租户你只能抉择一种数据库, 目前.Net 在开源的状态下我置信会有越来越好的组件框架诞生, 毕竟这么好的语言如果配上丰盛的生态那将是所有.Neter 的福音。
最初的最初
demo 地址 https://github.com/xuejmnet/S…
您都看到这边了确定不点个 star 或者赞吗, 一款.Net 不得不学的分库分表解决方案, 简略了解为 sharding-jdbc 在.net 中的实现并且反对更多个性和更优良的数据聚合, 领有原生性能的 97%, 并且无业务侵入性, 反对未分片的所有 efcore 原生查问
- github 地址 https://github.com/xuejmnet/s…
- gitee 地址 https://gitee.com/dotnetchina…