应用 rebloom 遇到了报错:redis.exceptions.ResponseError: Maximum expansions reached
具体内容如下:
File "/usr/local/lib/python3.10/site-packages/redisbloom/client.py", line 411, in cfAddNX
return self.execute_command(self.CF_ADDNX, *params)
│ │ │ │ └ ['crawler', 'https://power.in-en.com/html/power-2402142.shtml']
│ │ │ └ 'CF.ADDNX'
│ │ └ Client<ConnectionPool<Connection<host=172.16.36.108,port=6379,db=0>>>
│ └ <function Redis.execute_command at 0x7f49110785e0>
└ Client<ConnectionPool<Connection<host=172.16.36.108,port=6379,db=0>>>
File "/usr/local/lib/python3.10/site-packages/redis/client.py", line 901, in execute_command
return self.parse_response(conn, command_name, **options)
│ │ │ │ └ {}
│ │ │ └ 'CF.ADDNX'
│ │ └ Connection<host=172.16.36.108,port=6379,db=0>
│ └ <function Redis.parse_response at 0x7f4911078670>
└ Client<ConnectionPool<Connection<host=172.16.36.108,port=6379,db=0>>>
File "/usr/local/lib/python3.10/site-packages/redis/client.py", line 915, in parse_response
response = connection.read_response()
│ └ <function Connection.read_response at 0x7f49111f3c70>
└ Connection<host=172.16.36.108,port=6379,db=0>
File "/usr/local/lib/python3.10/site-packages/redis/connection.py", line 756, in read_response
raise response
└ ResponseError('Maximum expansions reached')
redis.exceptions.ResponseError: Maximum expansions reached
问题起因:布隆过滤器的容量满了
解决方案:
如果你是用 redis cli,就先删除 key,再从新创立 key。用这个命令创立 CF.RESERVE
,具体参考:CF.RESERVE
比方 CF.RESERVE crawler 100000000
就是创立一个容量为一亿的布隆过滤器
当然,这个容量到底用多少,须要本人评估
如果你要查看以后布隆过滤器的容量大小,就用 CF.INFO 命令,比方:
127.0.0.1:6379> CF.INFO clip_url_hash
1) "Size"
2) "16777272"
3) "Number of buckets"
4) "8388608"
5) "Number of filters"
6) "1"
7) "Number of items inserted"
8) "21436"
9) "Number of items deleted"
10) "0"
11) "Bucket size"
12) "2"
13) "Expansion rate"
14) "1"
15) "Max iterations"
16) "20"
如果你是用 python, 能够依照上面的代码来创立新的 key
from redisbloom.client import Client
import settings
client = Client(
host=settings.REDIS_CONFIG.host,
port=settings.REDIS_CONFIG.port,
db=settings.REDIS_CONFIG.db
)
def create_filter():
client.cfCreate(key, 1000000000)