U /e@sdZddlmZddlZddlZddlZddlZddlZddlm Z ddl m Z e eZeZddZGdd d e jZdd d Zd dZeZdS)a Modified ThreadPoolExecutor to support threads leaving the thread pool This includes a global `secede` method that a submitted function can call to have its thread leave the ThreadPoolExecutor's thread pool. This allows the thread pool to allocate another thread if necessary and so is useful when a function realises that it is going to be a long-running job that doesn't want to take up space. When the function finishes its thread will terminate gracefully. This code copies and modifies two functions from the `concurrent.futures.thread` module, notably `_worker` and ThreadPoolExecutor._adjust_thread_count` to allow for checking against a global `threading.local` state. These functions are subject to the following license, which is included as a comment at the end of this file: https://docs.python.org/3/license.html ... and are under copyright by the Python Software Foundation Copyright 2001-2016 Python Software Foundation; All Rights Reserved ) annotationsN)_concurrent_futures_thread)timec Csdt_|t_zztjr|jJ|jrb|j\}}|j||jt | W5QRqW5QRXz|j dd}Wnt jk rYqYnX|dk r|~qtjs|dks|jr|dWW2dSq~Wn"tk rtjdddYnXW5t`t`XdS)NTtimeoutzException in worker)exc_info) thread_stateproceedexecutor _rejoin_lock _rejoin_listpop_threadsaddremove threadingcurrent_threadsetgetqueueEmptyrunthread _shutdownput BaseExceptionloggercritical)r Z work_queueZ rejoin_threadZ rejoin_eventZtaskrB/tmp/pip-unpacked-wheel-g426oqom/distributed/threadpoolexecutor.py_worker's4    r!cs6eZdZeZfddZddZd ddZZ S) ThreadPoolExecutorcs0tj||g|_t|_|dd|_dS)NZthread_name_prefixZDaskThreadPoolExecutor)super__init__r rLockr r_thread_name_prefix)selfargskwargs __class__rr r$Js zThreadPoolExecutor.__init__cCs\t|j|jkrXtjt|jdtt |j f||j fd}d|_ |j ||dS)Nz-%d-%d)targetnamer(T)lenrZ _max_workersrThreadr!r&osgetpidnext_counter _work_queuedaemonrstart)r'trrr _adjust_thread_countRs z'ThreadPoolExecutor._adjust_thread_countTNc Cstv|jd|_|jdW5QRX|dk r __classcell__rrr*r r"Fs  r"Tc Cs:dt_t&tjjt|r,tjW5QRXdS)zwHave this thread secede from the ThreadPoolExecutor See Also -------- rejoin : rejoin the thread pool FN) r r r9r rrrrr8)adjustrrr secedens rFc CsXt}t}tj}|j|j||fW5QRX|dd| dt_ dS)aHave this thread rejoin the ThreadPoolExecutor This will block until a new slot opens up in the executor. The next thread to finish a task will leave the pool to allow this one to join. See Also -------- secede : leave the thread pool cSsdS)Nrrrrr zrejoin..TN) rrEventr r r r appendZsubmitr<r )reventerrr rejoin|s rM)T)__doc__ __future__rrBloggingr0rrZ distributedrrZdistributed.metricsr getLoggerr?rlocalr r!r"rFrMr%r9rrrr s     (