Threading and Queue Example

0 Juan Grasso · April 16, 2015
i just learn about threading and queue and i did this code. I think that it's and easy way to understand how both modules work together. Its comented so i think everyone shoud understand what its going on.


import threading
from queue import Queue
import time
 
links = ['link_1','link_2','link_3','link_4','link_5','link_6','link_7']
 
def downloader(worker): #  The function that 'downlad' the content of the links
    print('Downlading: ',str(worker))
    time.sleep(2) #  Simulate that we are downloading something
    print('You just download: ',str(worker))
 
def threader(): #  Each time a thread is free, it thread the function
    while True:
        worker = q.get() #  The worker get a item of the queue
        downloader(worker) # Call the function Downloader with that specific worker
        q.task_done()  #  Tell that we just finished a task
 
q = Queue() # Create the queue
 
for x in range(3): #  I create 3 threads just because want
    t = threading.Thread(target = threader) #  Create the thread
    t.daemon = True #  Say that the thread is a deamon so it die when Main ends
    t.start() #  Put the thread to work
 
for l in links:
    q.put(l) #  Add each item in links to the queue
 
q.join() #  The program dont contine until all the tasks are donde
 
print ('All the downloads are completed')

I leave my github link if there is someone interested on it. 
https://github.com/juangrassob/LearningPython

 

Post a Reply

Replies

Oldest  Newest  Rating
0 Stian OK · April 27, 2015
Here's an object oriented example that's compatible with both Python 2.x and 3.x


import threading
import time

try:
    from queue import Queue
except ImportError:
    from Queue import Queue

class MyApp:
    def __init__(self):
        self.queue = Queue()

    def add(self, message):
        self.queue.put(message)

    def worker(self):
        while True:
            item = self.queue.get()
            time.sleep(2)
            print(item)
            self.queue.task_done()

    def join(self):
        self.queue.join()

if __name__ == "__main__":
    app = MyApp()
    app.add("Hello, world!")
    app.add("How are you?")
    app.add("Yeaaaaaaaah!")
    app.add("Another message bro..")
    app.add("Yet another message..")

    for i in range(2):
        t = threading.Thread(target=app.worker)
        t.daemon = True
        t.start()
        print("Started worker #%d" % int(i+1))

    app.join()
  • 1

Python

107,320 followers
About

This section is all about snakes! Just kidding.

Links
Moderators
Bucky Roberts Administrator