Head
2026-05-29 21:58:19 [scrapy.utils.log] INFO: Scrapy 2.10.0 started (bot: retriever)
2026-05-29 21:58:19 [scrapy.utils.log] INFO: Versions: lxml 5.1.0.0, libxml2 2.12.3, cssselect 1.2.0, parsel 1.9.0, w3lib 2.0.0, Twisted 22.4.0, Python 3.10.12 (main, Mar 3 2026, 11:56:32) [GCC 11.4.0], pyOpenSSL 24.1.0 (OpenSSL 3.2.1 30 Jan 2024), cryptography 42.0.5, Platform Linux-6.8.0-1030-aws-x86_64-with-glibc2.35
2026-05-29 21:58:19 [scrapy.addons] INFO: Enabled addons:
[]
2026-05-29 21:58:19 [scrapy.crawler] INFO: Overridden settings:
{'AUTOTHROTTLE_ENABLED': True,
'BOT_NAME': 'retriever',
'HTTPCACHE_EXPIRATION_SECS': 86400,
'IMAGES_STORE_S3_ACL': 'public-read',
'LOG_FILE': '/home/scrapyd/logs/loknow_spiders_prod/ImageSpider/7e187f885ba911f1bdc0c3bbae71ecef.log',
'LOG_FORMATTER': 'loknow_spiders.logging.PoliteLogFormatter',
'LOG_LEVEL': 'INFO',
'NEWSPIDER_MODULE': 'loknow_spiders.spiders',
'SPIDER_MODULES': ['loknow_spiders.spiders'],
'TELNETCONSOLE_ENABLED': False,
'USER_AGENT': 'AdRetriever (https://adretriever.com)'}
2026-05-29 21:58:19 [py.warnings] WARNING: /home/scrapyd/venv/lib/python3.10/site-packages/scrapy/utils/request.py:248: ScrapyDeprecationWarning: '2.6' is a deprecated value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting.
It is also the default value. In other words, it is normal to get this warning if you have not defined a value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting. This is so for backward compatibility reasons, but it will change in a future version of Scrapy.
See the documentation of the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting for information on how to handle this deprecation.
return cls(crawler)
2026-05-29 21:58:19 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
'scrapy.extensions.memusage.MemoryUsage',
'scrapy.extensions.feedexport.FeedExporter',
'scrapy.extensions.logstats.LogStats',
'scrapy.extensions.throttle.AutoThrottle']
2026-05-29 21:58:19 [ImageSpider] INFO: Current Spider Environment set to: prod
2026-05-29 21:58:19 [ImageSpider] INFO: Retrieving AWS secret: spider_prod_secrets
2026-05-29 21:58:20 [botocore.credentials] INFO: Found credentials in shared credentials file: ~/.aws/credentials
2026-05-29 21:58:20 [ImageSpider] INFO: In base spider ml_kwargs is None and scrape_type is inventory
2026-05-29 21:58:20 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
'loknow_spiders.middlewares.ThrottlingRetryMiddleware',
'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
'scrapy.downloadermiddlewares.stats.DownloaderStats']
2026-05-29 21:58:20 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
'loknow_spiders.middlewares.RetrieverSpiderMiddleware',
'scrapy.spidermiddlewares.referer.RefererMiddleware',
'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
'scrapy.spidermiddlewares.depth.DepthMiddleware']
2026-05-29 21:58:23 [botocore.credentials] INFO: Found credentials in shared credentials file: ~/.aws/credentials
2026-05-29 21:58:23 [scrapy.middleware] INFO: Enabled item pipelines:
['loknow_spiders.pipelines.ImageUrlsValidationPipeline']
2026-05-29 21:58:23 [scrapy.core.engine] INFO: Spider opened
2026-05-29 21:58:23 [loknow_spiders] INFO: Scrape ID: 30e73a75
2026-05-29 21:58:23 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-05-29 21:58:31 [loknow_spiders] INFO: Completed https://cdn.shopify.com/s/files/1/0471/6701/3029/products/womens-red-t-shirt_925x_b1df6e1b-03dd-454f-bc1f-7ff96edf9329.jpg?v=1642460574
2026-05-29 21:58:31 [loknow_spiders] INFO: Completed https://cdn.shopify.com/s/files/1/0471/6701/3029/products/urban-fashion_925x_c73cfbb5-b4c7-4338-bac1-0319c8e3bbc0.jpg?v=1642460571
2026-05-29 21:58:33 [loknow_spiders] INFO: Completed https://cdn.shopify.com/s/files/1/0471/6701/3029/products/blacktee.webp?v=1657030149
2026-05-29 21:58:35 [scrapy.core.engine] INFO: Closing spider (finished)
2026-05-29 21:58:35 [ImageSpider] INFO: RetrieverAPI is making requests to https://api.adretriever.com with headers {'Authorization': 'Token 96dfb640234f1d676d6fa726f8eae6e7aab44cda'}
2026-05-29 21:58:35 [root] INFO: Making post request to https://api.adretriever.com/api/validated_urls/30e73a75
2026-05-29 21:58:35 [loknow_spiders] INFO: Cache hit rate: 88.0%
2026-05-29 21:58:35 [loknow_spiders] INFO: Cached images: 109218f5bb4ac3a3558_1777702805: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/menswear-blue-zip-up-jacket_925x_80e5306a-81a6-4e3a-9685-0e63dd748bf5.jpg?v=1642460564
7607d14ecbeed64708e_1778615434: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/autumn-photographer-taking-picture_925x_2e5bcb1e-aa52-4574-aa2b-a554109a1a00.jpg?v=1642460553
75b9fc34c38d2ae8186_1777455966: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/smiling-woman-poses_925x_d067181f-323e-4d66-a933-cb6a2157ce47.jpg?v=1642460570
2bb03e69e78e767581f_1778441939: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/striped-blouse-fashion_925x_93fb0dcf-2a54-4985-9577-dc50838a9ce5.jpg?v=1642460556
8d160aded99d03da5a6_1778879627: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/smiling-woman-on-snowy-afternoon_925x_4a0cd500-5929-42bf-a523-a5e18a435989.jpg?v=1642460562
1c39602a44e4d77c5a3_1778879628: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/young-hip-woman-at-carnival_925x_1e9b3587-daef-43d0-9ccc-5bf71b3b01c4.jpg?v=1642460566
d221008508702387789_1777529402: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/young-man-in-bright-fashion_925x_8776c187-6747-4815-a0cc-8b03b6dcfe13.jpg?v=1642460550
fb082d5c8abdcf30526_1779308373: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/mens-fall-fashion-jacket_925x_9f898bdc-bcc0-4c03-842a-d2cb6f4bd96c.jpg?v=1642460560
53cc3653ded7dc10570_1779135485: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/woman-outside-brownstone_925x_d62fa07b-cf39-4726-ace5-ceebe9f277c3.jpg?v=1642460567
445afd289ba4fd99dc7_1779832687: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/putting-on-your-shoes_925x_04e71ada-dace-44d3-8b24-0c3cdda0bdf1.jpg?v=1642460577
a633701beabfefdcf48_1778182554: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/city-woman-fashion_925x_2x_952e16c3-72ba-4183-a17d-46cc36f0da09.jpg?v=1642460555
b2688f01c7b24a32b07_1779832687: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/young-female-models-denim_925x_8ae19ff2-79de-446f-9adb-8642e8ef8bcd.jpg?v=1642460559
451d3a4e7d87ba27cd3_1779569227: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/whitetee.jpg?v=1657030158
d2c9ccd17d8d4d7c7c3_1779395250: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/green-t-shirt.jpg?v=1628281944
5b9acaebc32d2ce1fbd_1777455971: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/casual-fashion-woman_925x_e9a705d8-4a1a-473b-aa51-c3b60486c58d.jpg?v=1642460552
9bf408e9cc49aa77391_1779135487: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/leather-jacket-and-tea_925x_265bb656-b6dd-4560-869a-6e78c57a6644.jpg?v=1642460558
0ee06d2e1f14cf348d3_1779569228: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/red-plaid-shirt_925x_721b17ae-a883-4cf6-a1f5-9d9ab0ec78a7.jpg?v=1642460568
fc5966e0c73295686e3_1778359296: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/man-adjusts-blue-tuxedo-bowtie_925x_3c42051f-20aa-42c2-bf71-8bb9d6e4982e.jpg?v=1642460572
a9fb8094aa657520912_1778789176: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/black-bag-over-the-shoulder_925x_cafc7f9c-15db-4c00-8fd5-0ff9a3d4f6fb.jpg?v=1642460563
7e4c66c6cc03ca36539_1778528575: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/woman-in-the-city_925x_1ccce2ce-f11f-4b77-8608-a1e7660b8c10.jpg?v=1642460575
2026-05-29 21:58:35 [loknow_spiders] INFO: Uncached images: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/urban-fashion_925x_c73cfbb5-b4c7-4338-bac1-0319c8e3bbc0.jpg?v=1642460571
https://cdn.shopify.com/s/files/1/0471/6701/3029/products/womens-red-t-shirt_925x_b1df6e1b-03dd-454f-bc1f-7ff96edf9329.jpg?v=1642460574
https://cdn.shopify.com/s/files/1/0471/6701/3029/products/blacktee.webp?v=1657030149
2026-05-29 21:58:35 [scrapy.extensions.feedexport] INFO: Stored jsonlines feed (25 items) in: file:///home/scrapyd/items/loknow_spiders_prod/ImageSpider/7e187f885ba911f1bdc0c3bbae71ecef.jl
2026-05-29 21:58:35 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 7545,
'downloader/request_count': 23,
'downloader/request_method_count/HEAD': 23,
'downloader/response_bytes': 29802,
'downloader/response_count': 23,
'downloader/response_status_count/200': 23,
'dupefilter/filtered': 2,
'elapsed_time_seconds': 12.150264,
'feedexport/success_count/FileFeedStorage': 1,
'file_count': 3,
'file_status_count/uptodate': 3,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2026, 5, 29, 21, 58, 35, 605855),
Tail
2026-05-29 21:58:19 [scrapy.utils.log] INFO: Scrapy 2.10.0 started (bot: retriever)
2026-05-29 21:58:19 [scrapy.utils.log] INFO: Versions: lxml 5.1.0.0, libxml2 2.12.3, cssselect 1.2.0, parsel 1.9.0, w3lib 2.0.0, Twisted 22.4.0, Python 3.10.12 (main, Mar 3 2026, 11:56:32) [GCC 11.4.0], pyOpenSSL 24.1.0 (OpenSSL 3.2.1 30 Jan 2024), cryptography 42.0.5, Platform Linux-6.8.0-1030-aws-x86_64-with-glibc2.35
2026-05-29 21:58:19 [scrapy.addons] INFO: Enabled addons:
[]
2026-05-29 21:58:19 [scrapy.crawler] INFO: Overridden settings:
{'AUTOTHROTTLE_ENABLED': True,
'BOT_NAME': 'retriever',
'HTTPCACHE_EXPIRATION_SECS': 86400,
'IMAGES_STORE_S3_ACL': 'public-read',
'LOG_FILE': '/home/scrapyd/logs/loknow_spiders_prod/ImageSpider/7e187f885ba911f1bdc0c3bbae71ecef.log',
'LOG_FORMATTER': 'loknow_spiders.logging.PoliteLogFormatter',
'LOG_LEVEL': 'INFO',
'NEWSPIDER_MODULE': 'loknow_spiders.spiders',
'SPIDER_MODULES': ['loknow_spiders.spiders'],
'TELNETCONSOLE_ENABLED': False,
'USER_AGENT': 'AdRetriever (https://adretriever.com)'}
2026-05-29 21:58:19 [py.warnings] WARNING: /home/scrapyd/venv/lib/python3.10/site-packages/scrapy/utils/request.py:248: ScrapyDeprecationWarning: '2.6' is a deprecated value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting.
It is also the default value. In other words, it is normal to get this warning if you have not defined a value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting. This is so for backward compatibility reasons, but it will change in a future version of Scrapy.
See the documentation of the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting for information on how to handle this deprecation.
return cls(crawler)
2026-05-29 21:58:19 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
'scrapy.extensions.memusage.MemoryUsage',
'scrapy.extensions.feedexport.FeedExporter',
'scrapy.extensions.logstats.LogStats',
'scrapy.extensions.throttle.AutoThrottle']
2026-05-29 21:58:19 [ImageSpider] INFO: Current Spider Environment set to: prod
2026-05-29 21:58:19 [ImageSpider] INFO: Retrieving AWS secret: spider_prod_secrets
2026-05-29 21:58:20 [botocore.credentials] INFO: Found credentials in shared credentials file: ~/.aws/credentials
2026-05-29 21:58:20 [ImageSpider] INFO: In base spider ml_kwargs is None and scrape_type is inventory
2026-05-29 21:58:20 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
'loknow_spiders.middlewares.ThrottlingRetryMiddleware',
'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
'scrapy.downloadermiddlewares.stats.DownloaderStats']
2026-05-29 21:58:20 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
'loknow_spiders.middlewares.RetrieverSpiderMiddleware',
'scrapy.spidermiddlewares.referer.RefererMiddleware',
'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
'scrapy.spidermiddlewares.depth.DepthMiddleware']
2026-05-29 21:58:23 [botocore.credentials] INFO: Found credentials in shared credentials file: ~/.aws/credentials
2026-05-29 21:58:23 [scrapy.middleware] INFO: Enabled item pipelines:
['loknow_spiders.pipelines.ImageUrlsValidationPipeline']
2026-05-29 21:58:23 [scrapy.core.engine] INFO: Spider opened
2026-05-29 21:58:23 [loknow_spiders] INFO: Scrape ID: 30e73a75
2026-05-29 21:58:23 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-05-29 21:58:31 [loknow_spiders] INFO: Completed https://cdn.shopify.com/s/files/1/0471/6701/3029/products/womens-red-t-shirt_925x_b1df6e1b-03dd-454f-bc1f-7ff96edf9329.jpg?v=1642460574
2026-05-29 21:58:31 [loknow_spiders] INFO: Completed https://cdn.shopify.com/s/files/1/0471/6701/3029/products/urban-fashion_925x_c73cfbb5-b4c7-4338-bac1-0319c8e3bbc0.jpg?v=1642460571
2026-05-29 21:58:33 [loknow_spiders] INFO: Completed https://cdn.shopify.com/s/files/1/0471/6701/3029/products/blacktee.webp?v=1657030149
2026-05-29 21:58:35 [scrapy.core.engine] INFO: Closing spider (finished)
2026-05-29 21:58:35 [ImageSpider] INFO: RetrieverAPI is making requests to https://api.adretriever.com with headers {'Authorization': 'Token 96dfb640234f1d676d6fa726f8eae6e7aab44cda'}
2026-05-29 21:58:35 [root] INFO: Making post request to https://api.adretriever.com/api/validated_urls/30e73a75
2026-05-29 21:58:35 [loknow_spiders] INFO: Cache hit rate: 88.0%
2026-05-29 21:58:35 [loknow_spiders] INFO: Cached images: 109218f5bb4ac3a3558_1777702805: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/menswear-blue-zip-up-jacket_925x_80e5306a-81a6-4e3a-9685-0e63dd748bf5.jpg?v=1642460564
7607d14ecbeed64708e_1778615434: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/autumn-photographer-taking-picture_925x_2e5bcb1e-aa52-4574-aa2b-a554109a1a00.jpg?v=1642460553
75b9fc34c38d2ae8186_1777455966: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/smiling-woman-poses_925x_d067181f-323e-4d66-a933-cb6a2157ce47.jpg?v=1642460570
2bb03e69e78e767581f_1778441939: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/striped-blouse-fashion_925x_93fb0dcf-2a54-4985-9577-dc50838a9ce5.jpg?v=1642460556
8d160aded99d03da5a6_1778879627: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/smiling-woman-on-snowy-afternoon_925x_4a0cd500-5929-42bf-a523-a5e18a435989.jpg?v=1642460562
1c39602a44e4d77c5a3_1778879628: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/young-hip-woman-at-carnival_925x_1e9b3587-daef-43d0-9ccc-5bf71b3b01c4.jpg?v=1642460566
d221008508702387789_1777529402: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/young-man-in-bright-fashion_925x_8776c187-6747-4815-a0cc-8b03b6dcfe13.jpg?v=1642460550
fb082d5c8abdcf30526_1779308373: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/mens-fall-fashion-jacket_925x_9f898bdc-bcc0-4c03-842a-d2cb6f4bd96c.jpg?v=1642460560
53cc3653ded7dc10570_1779135485: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/woman-outside-brownstone_925x_d62fa07b-cf39-4726-ace5-ceebe9f277c3.jpg?v=1642460567
445afd289ba4fd99dc7_1779832687: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/putting-on-your-shoes_925x_04e71ada-dace-44d3-8b24-0c3cdda0bdf1.jpg?v=1642460577
a633701beabfefdcf48_1778182554: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/city-woman-fashion_925x_2x_952e16c3-72ba-4183-a17d-46cc36f0da09.jpg?v=1642460555
b2688f01c7b24a32b07_1779832687: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/young-female-models-denim_925x_8ae19ff2-79de-446f-9adb-8642e8ef8bcd.jpg?v=1642460559
451d3a4e7d87ba27cd3_1779569227: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/whitetee.jpg?v=1657030158
d2c9ccd17d8d4d7c7c3_1779395250: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/green-t-shirt.jpg?v=1628281944
5b9acaebc32d2ce1fbd_1777455971: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/casual-fashion-woman_925x_e9a705d8-4a1a-473b-aa51-c3b60486c58d.jpg?v=1642460552
9bf408e9cc49aa77391_1779135487: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/leather-jacket-and-tea_925x_265bb656-b6dd-4560-869a-6e78c57a6644.jpg?v=1642460558
0ee06d2e1f14cf348d3_1779569228: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/red-plaid-shirt_925x_721b17ae-a883-4cf6-a1f5-9d9ab0ec78a7.jpg?v=1642460568
fc5966e0c73295686e3_1778359296: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/man-adjusts-blue-tuxedo-bowtie_925x_3c42051f-20aa-42c2-bf71-8bb9d6e4982e.jpg?v=1642460572
a9fb8094aa657520912_1778789176: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/black-bag-over-the-shoulder_925x_cafc7f9c-15db-4c00-8fd5-0ff9a3d4f6fb.jpg?v=1642460563
7e4c66c6cc03ca36539_1778528575: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/woman-in-the-city_925x_1ccce2ce-f11f-4b77-8608-a1e7660b8c10.jpg?v=1642460575
2026-05-29 21:58:35 [loknow_spiders] INFO: Uncached images: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/urban-fashion_925x_c73cfbb5-b4c7-4338-bac1-0319c8e3bbc0.jpg?v=1642460571
https://cdn.shopify.com/s/files/1/0471/6701/3029/products/womens-red-t-shirt_925x_b1df6e1b-03dd-454f-bc1f-7ff96edf9329.jpg?v=1642460574
https://cdn.shopify.com/s/files/1/0471/6701/3029/products/blacktee.webp?v=1657030149
2026-05-29 21:58:35 [scrapy.extensions.feedexport] INFO: Stored jsonlines feed (25 items) in: file:///home/scrapyd/items/loknow_spiders_prod/ImageSpider/7e187f885ba911f1bdc0c3bbae71ecef.jl
2026-05-29 21:58:35 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 7545,
'downloader/request_count': 23,
'downloader/request_method_count/HEAD': 23,
'downloader/response_bytes': 29802,
'downloader/response_count': 23,
'downloader/response_status_count/200': 23,
'dupefilter/filtered': 2,
'elapsed_time_seconds': 12.150264,
'feedexport/success_count/FileFeedStorage': 1,
'file_count': 3,
'file_status_count/uptodate': 3,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2026, 5, 29, 21, 58, 35, 605855),
'item_scraped_count': 25,
'log_count/INFO': 23,
'log_count/WARNING': 1,
'memusage/max': 133038080,
'memusage/startup': 133038080,
'response_received_count': 23,
'scheduler/dequeued': 23,
'scheduler/dequeued/memory': 23,
'scheduler/enqueued': 23,
'scheduler/enqueued/memory': 23,
'start_time': datetime.datetime(2026, 5, 29, 21, 58, 23, 455591)}
2026-05-29 21:58:35 [scrapy.core.engine] INFO: Spider closed (finished)