GRID job with dependencies

In bioinformatics, it is common submit jobs to grid servers. In many cases, those grid jobs have dependencies on other jobs: one job shall only start after certain jobs have finished. For example, in the Tuxedo workflow for RNA-seq, cufflinks always run after (successful) tophat alignment and cuffdiff always runs after cuffdiff. The -hold_jid option in qsub command help to solve the dependency needs:

$qsub ../scripts/cuffmerge_7_9.qsub 

Your job 737624 (“CM_HCC_7_9”) has been submitted

$qsub -hold_jid 737624 ../scripts/cuffdiff_7_9.qsub

Your job 737625 (“CD_HCC_7_9”) has been submitted

$ q
737624 0.50284 CM_HCC_7_9 me r 01/18/2013 09:53:50 96mem_dev.q@n0204.myclust 8
737625 0.00000 CD_HCC_7_9 me hqw 01/18/2013 09:54:28

Note the “hqw” status in the second job indicates this job is on hold. In case the dependency is on more than one job, a list of jobs can be put in the -hold_jid option.

More information about this option and the grid environment can found from here.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s