linux - How can I use a pipe or redirect in a qsub command? -

there commands i'd run on grid using qsub (sge 8.1.3, centos 5.9) need use pipe (|) or redirect (>). example, let's have parallelize command

echo 'hello world' > hello.txt 

(obviously simplified example: in reality might need redirect output of program bowtie directly samtools). if did:

qsub echo 'hello world' > hello.txt 

the resulting content of hello.txt like

your job 123454321 ("echo") has been submitted 

similarly if used pipe (echo "hello world" | myprogram), message passed myprogram, not actual stdout.

i'm aware write small bash script each contain command pipe/redirect, , qsub ./myscript.sh. however, i'm trying run many parallelized jobs @ same time using script, i'd have write many such bash scripts each different command. when scripting solution can start feel hackish. example of such script in python:

for i, (infile1, infile2, outfile) in enumerate(files):     command = ("bowtie -s %s %s | " +                "samtools view -bs - > %s\n") % (infile1, infile2, outfile)      script = "job" + str(counter) + ".sh"     open(script, "w").write(command)     os.system("chmod 755 %s" % script)     os.system("qsub -cwd ./%s" % script) 

this frustrating few reasons, among them program can't delete many jobxx.sh scripts afterwards clean after itself, since don't know how long job waiting in queue, , script has there when job starts.

is there way provide full echo 'hello world' > hello.txt command qsub without having create file containing command?

you can turning bash -c command, lets put | in quoted statement:

 qsub bash -c "cmd <options> | cmd2 <options>" 

as @spuder has noted in comments, seems in other versions of qsub (not sge 8.1.3, i'm using), 1 can solve problem with:

echo "cmd <options> | cmd2 <options>" | qsub 

as well.