Multiprocessing in Python

There are numerous ways to multiprocess in python. Pools give a very easy way.

Note: If you do not catch keyboard interrups as shown in the code then you will have to wait until all workers have finished even if you send Ctrl+C to the parent process. There are other methods such as map (blocks until each result is received and orders sequentially), apply and apply_async where you can provide a callback function that actually determines what must be done with the result. If the task items are not iterable then you must use apply or apply_async and handle the input in the callback.

from multiprocessing import Pool
from functools import partial

import time
import sys

def doWork(i, x):
    """
    Does the actual work.
    """
    print i
    time.sleep(2)
    return (x * i)
    pass


def multi():
    """
    Does the actual multiprocessing.
    """
    # Create the pool.
    pool = Pool()

    # We must convert the doWork to a function of one argument.
    # This argument will be filled in by an iterator.
    partialWorker = partial(doWork, x=5)

    tasks = range(100)

    p = pool.map_async(partialWorker, tasks)

    try:
        results = p.get(0xFFFF)
    except KeyboardInterrupt:
        print "Received Ctrl-c"
        sys.exit(1)

    pool.close()
    pool.join()          
    pass


if __name__ == "__main__":
    multi()
    

Using lambda functions for sorting in Python

Give a list L = [56, 78, 98, 34] the following one linear will sort the elements in the descending order of their values.
L.sort(lambda x, y: 1 if y > x else -1)

Convert png to mac icons

Use this tool.

Reconnect an ejected USB HDD in a Mac

Sometimes you realize that a file that you need is in an external USB HDD that you just ejected. Of course the external USB is still connected to your USB port. So what would you do to make the USB HDD reappear in your connected drives? Of course, you can physically disconnect the external USB HDD and re-plugin it. But that sounds stupid because the device is already connected to your machine. Just open a terminal and first find out what is the number of your disk by,

$ diskutil list

This will list up all your disk. Let us assume the external USB HDD that you want to reconnect is disk1 (by the way your inbuilt HDD will be disk0), do the following,

$ diskutil mountDisk disk1

Thats it! No more physical unplugging and replugging... 

Checking the site-packages

If you want to know where the python site-packages directory is located in your machine do the following.


from distutils.sysconfig import get_python_lib
print(get_python_lib())

Mac OSX Terminal shortcuts

Here are some useful shortcuts.