python错误BrokenProcessPool


报错信息如下

> A process in the process pool was terminated abruptly while the future was running or pending.

Rasie Queue.Full

   # 线程池入口
   def ThreadPool(self, val):
       with ThreadPoolExecutor(max_workers=Thread_number) as executor2:
           tasks = [executor2.submit(self.extract_dict, data) for data in val]
           for future in as_completed(tasks):
               if future.result() is None or future.result() == None:
                   continue
               for result in future.result():
                   print('result: %s' % (result))
           executor2.shutdown()

   # 进程池入口
   def ProcessPool(self, redis_list_key):
       '''
       :param Process_number: 即开启线程的个数
       :return: None
       '''
       result_list = self.get_list(redis_list_key)
       q = queue.Queue()
       q.put(result_list)
       with ProcessPoolExecutor(max_workers=Process_number) as executor:
           while not q.empty():
               val = q.get()
               for result in executor.map(self.ThreadPool, val):
                   if result is None or result == None:
                       continue
                   print('ProcessPoolresult: %s' % (result))
               executor.shutdown(wait=True) ```
</font>

<font size=4>问题就在于线程池模块中并没有返回对象 ,也就是我所多线程开启的程序```extract_dict```没有返回return,所以导致下面的```future.result()```
会导致出现异常 爆出```BrokenProcessPool```的错误

```python
if future.result() is None or future.result() == None:
       continue
   for result in future.result():
       print('result: %s' % (result))```
       
       
所以将代码调整为如下所示:
</font>

<font size=4>
   
```python
   # 线程池入口
   def ThreadPool(self, val):
       with ThreadPoolExecutor(max_workers=Thread_number) as executor2:
           tasks = [executor2.submit(self.extract_dict, data) for data in val]
           as_completed(tasks)
           executor2.shutdown()
   
   # 进程池入口
   def ProcessPool(self, redis_list_key):
       '''
       :param Process_number: 即开启线程的个数
       :return: None
       '''
       result_list = self.get_list(redis_list_key)
       q = queue.Queue()
       q.put(result_list)
       with ProcessPoolExecutor(max_workers=Process_number) as executor:
           while not q.empty():
               val = q.get()
               for result in executor.map(self.ThreadPool, val):
                   if result is None or result == None:
                       continue
                   print('ProcessPoolresult: %s' % (result))
               executor.shutdown(wait=True) ```
               
</font>

<font size=4>直接使用 as_completed(tasks)对象,不需要对返回值进行处理
   
as_completed(tasks) 是 concurrent.futures的一个模块
   
```python 
     from concurrent.futures import as_completed```
</font>



```python

文章作者: 旋律JOJO哒
版权声明: 本博客所有文章除特別声明外,均采用 CC BY 4.0 许可协议。转载请注明来源 旋律JOJO哒 !
评论
  目录