No recent searches
Popular Articles
Sorry! nothing found for
Posted 9 months ago by Dave Zvenyach
I am getting a seek of closed file error (see below) when I try to export items into S3 using the Gzip postprocessing plugin. I am using the
"FEEDS": { "s3://bucket/key.jsonl.gz": { "format": "jsonlines", "encoding": "utf8", "store_empty": False, "indent": 2, "postprocessing": ["scrapy.extensions.postprocessing.GzipPlugin"], } }
Any help would be welcome!
Thanks!
***
[scrapy.extensions.feedexport] Error storing jsonlines feed (17523 items) in: s3://bucket/key.jsonl.gz
Less
Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/twisted/python/threadpool.py", line 244, in inContext result = inContext.theWork() # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/twisted/python/threadpool.py", line 260, in <lambda> inContext.theWork = lambda: context.call( # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/twisted/python/context.py", line 117, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/twisted/python/context.py", line 82, in callWithContext return func(*args, **kw) ^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/scrapy/extensions/feedexport.py", line 244, in _store_in_thread file.seek(0) File "/usr/local/lib/python3.11/tempfile.py", line 483, in func_wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ ValueError: seek of closed file
0 Votes
0 Comments
Login to post a comment
People who like this
This post will be deleted permanently. Are you sure?
We use cookies to try and give you a better experience in Freshdesk.
You can learn more about what kind of cookies we use, why, and how from our Privacy Policy. If you hate cookies, or are just on a diet, you can disable them altogether too. Just note that the Freshdesk service is pretty big on some cookies (we love the choco-chip ones), and some portions of Freshdesk may not work properly if you disable cookies.
We’ll also assume you agree to the way we use cookies and are ok with it as described in our Privacy Policy, unless you choose to disable them altogether through your browser.
I am getting a seek of closed file error (see below) when I try to export items into S3 using the Gzip postprocessing plugin. I am using the
scrapy:2.11 stack and the following settings:Any help would be welcome!
Thanks!
***
[scrapy.extensions.feedexport] Error storing jsonlines feed (17523 items) in: s3://bucket/key.jsonl.gz
Less
0 Votes
0 Comments
Login to post a comment