Skip to main content

Package for using processes which mimics the threading module

Project description

processing is a package for the Python language which supports the spawning of processes using the API of the standard library’s threading module. It runs on both Unix and Windows.

Features:

  • Objects can be transferred between processes using pipes or multi-producer/multi-consumer queues.

  • Objects can be shared between processes using a server process or (for simple data) shared memory.

  • Equivalents of all the synchronization primitives in threading are available.

  • A Pool class makes it easy to submit tasks to a pool of worker processes.

Examples

The processing.Process class follows the API of threading.Thread. For example

from processing import Process, Queue

def f(q):
    q.put('hello world')

if __name__ == '__main__':
    q = Queue()
    p = Process(target=f, args=[q])
    p.start()
    print q.get()
    p.join()

Synchronization primitives like locks, semaphores and conditions are available, for example

>>> from processing import Condition
>>> c = Condition()
>>> print c
<Condition(<RLock(None, 0)>), 0>
>>> c.acquire()
True
>>> print c
<Condition(<RLock(MainProcess, 1)>), 0>

One can also use a manager to create shared objects either in shared memory or in a server process, for example

>>> from processing import Manager
>>> manager = Manager()
>>> l = manager.list(range(10))
>>> l.reverse()
>>> print l
[9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
>>> print repr(l)
<Proxy[list] object at 0x00E1B3B0>

Tasks can be offloaded to a pool of worker processes in various ways, for example

>>> from processing import Pool
>>> def f(x): return x*x
...
>>> p = Pool(4)
>>> result = p.mapAsync(f, range(10))
>>> print result.get(timeout=1)
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

processing-0.52.zip (178.0 kB view hashes)

Uploaded Source

Built Distributions

processing-0.52.win32-py2.5.exe (191.2 kB view hashes)

Uploaded Source

processing-0.52.win32-py2.4.exe (189.7 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page