I wanted to use put my code on to scrapy cloud and run that lakhs of times in single day and the execution time of my code is taking an avg of 45sec.. is it possible to do in scrapy cloud? If yes can you tell me the procedure?
0 Votes
thriveni posted
almost 7 years ago
AdminBest Answer
Hello,
I wanted to use put my code on to scrapy cloud and run that lakhs of times in single day and the execution time of my code is taking an avg of 45sec.. is it possible to do in scrapy cloud? If yes can you tell me the procedure?
0 Votes
thriveni posted almost 7 years ago Admin Best Answer
Hello,
You can schedule periodic jobs to run the spider at specific intervals as given in https://support.scrapinghub.com/a/solutions/articles/22000200419-scheduling-periodic-jobs. Please note that this feature is available only for Scrapy cloud paid plans. You need to be subscribed to atleast 1 Scrapy Unit to schedule periodic jobs.
0 Votes
1 Comments
thriveni posted almost 7 years ago Admin Answer
Hello,
You can schedule periodic jobs to run the spider at specific intervals as given in https://support.scrapinghub.com/a/solutions/articles/22000200419-scheduling-periodic-jobs. Please note that this feature is available only for Scrapy cloud paid plans. You need to be subscribed to atleast 1 Scrapy Unit to schedule periodic jobs.
0 Votes
Login to post a comment