athena.utils.data_queue¶
data queue for multi thread
Module Contents¶
Functions¶
test() |
Test data queue. |
-
class
athena.utils.data_queue.DataQueue(generator, capacity=20, num_threads=4, max_index=10000, wait_time=0.0001)¶ Queue for data prefetching :param generator: instance of generator which feed data :type generator: generator :param capacity: maximum data to prefetch :type capacity: int :param num_threads: control concurrency, only take effect when do preprocessing :type num_threads: int :param wait_time: time to sleep when queue is full :type wait_time: float
-
__del__(self)¶
-
get(self)¶
-
stop(self)¶
-
generator_task(self)¶ Enqueue batch data
-
-
athena.utils.data_queue.test()¶ Test data queue. Excpet return:
epoch: %d, nb_batch: %d: finish.