Python并行處理框架 Jug

Jug 是一個基于任務的并行處理框架，采用 Python 編寫，可用來在不同的機器上運行同一個任務，使用 NFS 做文件系統的通訊。

• Persistent data across runs
• Re-use partial results if you change the algorithms (for example, if you search over a few more parameters for the best, then it will reuse the pre-computed values). Normally, I have a main computation script and then write a second visualisation script to plot out the results or compute some summary statistics and it's good if the second script is easy to write, easy to change, and reuses all computational results seamlessly.
• Supports concurrency with a very flexible system: CPUs can join the computation at any time. This allows it to be used in batch processing systems.
• You can check up on the status of the computation at any time (jug status)
• Two backends: file-based if all the processors share a filesystem (works over NFS too) or redis based if they can all connect to the same redis server.

示例代碼：

from jug import TaskGenerator
from time import sleep

@TaskGenerator
def is_prime(n):
    sleep(1.)
    for j in xrange(2,n-1):
        if (n % j) == 0:
            return False
    return True

primes100 = map(is_prime, xrange(2,101))

項目主頁：http://www.baiduhome.net/lib/view/home/1339246462271