Python script to run shell command -

i'm having bit of trouble. want run shell command within python in specific directory. based on code found on internet need following inclusions:

import os import subprocess import shlex 

and code below

os.chdir('/etc/test/') cmd = 'scrapy crawl test' subprocess.call(shlex.split(cmd)) 

as looks like, trying run command "scrapy crawl test" within /etc/test/ directory. when run manually terminal seems work fine when run python code gives me error:

info exception occured while scraping: [errno 2] no such file or directory

is able tell me if code incorrect, or if going wrong way perhaps.

why using subprocess? common practice run scrapy script use twisted's reactor. taken docs:

from twisted.internet import reactor scrapy.crawler import crawler scrapy.settings import settings scrapy import log testspiders.spiders.followall import followallspider  spider = followallspider(domain='scrapinghub.com') crawler = crawler(settings()) crawler.configure() crawler.crawl(spider) crawler.start() log.start() reactor.run() # script block here 

there plenty of examples out there:

• running scrapy script (beginner)
• how run scrapy within python script
• running scrapy script - hangs
• scrapy crawl script blocks script execution after scraping

hope helps.