应用python的futures.ThreadPoolExecutor
是,如果调用submit提交工作
ThreadPoolExecutor的会向执行
self._work_queue.put(w)
其中
self._work_queue = queue.SimpleQueue()
SimpleQueue 是不限度队列大小的,如果提交的工作太多,解决不及时,则导致占用太多内存
能够替换到_work_queue的实现,应用queue.Queue(maxsize=maxsize)
class ThreadPoolExecutorWithQueueSizeLimit(futures.ThreadPoolExecutor): def __init__(self, maxsize=50, *args, **kwargs): super(ThreadPoolExecutorWithQueueSizeLimit, self).__init__(*args, **kwargs) self._work_queue = queue.Queue(maxsize=maxsize)
links:
- https://stackoverflow.com/que...