Head
2026-02-28 22:48:17 [scrapy.utils.log] INFO: Scrapy 2.10.0 started (bot: retriever)
2026-02-28 22:48:17 [scrapy.utils.log] INFO: Versions: lxml 5.1.0.0, libxml2 2.12.3, cssselect 1.2.0, parsel 1.9.0, w3lib 2.0.0, Twisted 22.4.0, Python 3.10.12 (main, Jan 26 2026, 14:55:28) [GCC 11.4.0], pyOpenSSL 24.1.0 (OpenSSL 3.2.1 30 Jan 2024), cryptography 42.0.5, Platform Linux-6.8.0-1030-aws-x86_64-with-glibc2.35
2026-02-28 22:48:17 [scrapy.addons] INFO: Enabled addons:
[]
2026-02-28 22:48:17 [scrapy.crawler] INFO: Overridden settings:
{'AUTOTHROTTLE_ENABLED': True,
'BOT_NAME': 'retriever',
'HTTPCACHE_EXPIRATION_SECS': 86400,
'IMAGES_STORE_S3_ACL': 'public-read',
'LOG_FILE': '/home/scrapyd/logs/loknow_spiders_prod/ImageSpider/8dce860014f711f19b12fbebc751a6c4.log',
'LOG_FORMATTER': 'loknow_spiders.logging.PoliteLogFormatter',
'LOG_LEVEL': 'INFO',
'NEWSPIDER_MODULE': 'loknow_spiders.spiders',
'SPIDER_MODULES': ['loknow_spiders.spiders'],
'TELNETCONSOLE_ENABLED': False,
'USER_AGENT': 'AdRetriever (https://adretriever.com)'}
2026-02-28 22:48:17 [py.warnings] WARNING: /home/scrapyd/venv/lib/python3.10/site-packages/scrapy/utils/request.py:248: ScrapyDeprecationWarning: '2.6' is a deprecated value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting.
It is also the default value. In other words, it is normal to get this warning if you have not defined a value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting. This is so for backward compatibility reasons, but it will change in a future version of Scrapy.
See the documentation of the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting for information on how to handle this deprecation.
return cls(crawler)
2026-02-28 22:48:17 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
'scrapy.extensions.memusage.MemoryUsage',
'scrapy.extensions.feedexport.FeedExporter',
'scrapy.extensions.logstats.LogStats',
'scrapy.extensions.throttle.AutoThrottle']
2026-02-28 22:48:17 [ImageSpider] INFO: Current Spider Environment set to: prod
2026-02-28 22:48:17 [ImageSpider] INFO: Retrieving AWS secret: spider_prod_secrets
2026-02-28 22:48:17 [botocore.credentials] INFO: Found credentials in shared credentials file: ~/.aws/credentials
2026-02-28 22:48:17 [ImageSpider] INFO: In base spider ml_kwargs is None and scrape_type is inventory
2026-02-28 22:48:17 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
'loknow_spiders.middlewares.ThrottlingRetryMiddleware',
'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
'scrapy.downloadermiddlewares.stats.DownloaderStats']
2026-02-28 22:48:17 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
'loknow_spiders.middlewares.RetrieverSpiderMiddleware',
'scrapy.spidermiddlewares.referer.RefererMiddleware',
'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
'scrapy.spidermiddlewares.depth.DepthMiddleware']
2026-02-28 22:48:18 [botocore.credentials] INFO: Found credentials in shared credentials file: ~/.aws/credentials
2026-02-28 22:48:18 [scrapy.middleware] INFO: Enabled item pipelines:
['loknow_spiders.pipelines.ImageUrlsValidationPipeline']
2026-02-28 22:48:18 [scrapy.core.engine] INFO: Spider opened
2026-02-28 22:48:18 [loknow_spiders] INFO: Scrape ID: a6b427ef
2026-02-28 22:48:18 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-02-28 22:48:33 [scrapy.core.engine] INFO: Closing spider (finished)
2026-02-28 22:48:33 [ImageSpider] INFO: RetrieverAPI is making requests to https://app.adretriever.com with headers {'Authorization': 'Token 96dfb640234f1d676d6fa726f8eae6e7aab44cda'}
2026-02-28 22:48:33 [root] INFO: Making post request to https://app.adretriever.com/api/validated_urls/a6b427ef
2026-02-28 22:48:33 [loknow_spiders] INFO: Cache hit rate: 100.0%
2026-02-28 22:48:33 [loknow_spiders] INFO: Cached images: 109218f5bb4ac3a3558_1771624204: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/menswear-blue-zip-up-jacket_925x_80e5306a-81a6-4e3a-9685-0e63dd748bf5.jpg?v=1642460564
7607d14ecbeed64708e_1762715482: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/autumn-photographer-taking-picture_925x_2e5bcb1e-aa52-4574-aa2b-a554109a1a00.jpg?v=1642460553
75b9fc34c38d2ae8186_1762715485: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/smiling-woman-poses_925x_d067181f-323e-4d66-a933-cb6a2157ce47.jpg?v=1642460570
2bb03e69e78e767581f_1770152675: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/striped-blouse-fashion_925x_93fb0dcf-2a54-4985-9577-dc50838a9ce5.jpg?v=1642460556
8d160aded99d03da5a6_1770152676: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/smiling-woman-on-snowy-afternoon_925x_4a0cd500-5929-42bf-a523-a5e18a435989.jpg?v=1642460562
1c39602a44e4d77c5a3_1772231982: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/young-hip-woman-at-carnival_925x_1e9b3587-daef-43d0-9ccc-5bf71b3b01c4.jpg?v=1642460566
1aff6a91980e83cb2c8_1762715489: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/womens-red-t-shirt_925x_b1df6e1b-03dd-454f-bc1f-7ff96edf9329.jpg?v=1642460574
f7451c9e878da4a8b1b_1762715489: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/urban-fashion_925x_c73cfbb5-b4c7-4338-bac1-0319c8e3bbc0.jpg?v=1642460571
d221008508702387789_1771624214: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/young-man-in-bright-fashion_925x_8776c187-6747-4815-a0cc-8b03b6dcfe13.jpg?v=1642460550
fb082d5c8abdcf30526_1770152679: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/mens-fall-fashion-jacket_925x_9f898bdc-bcc0-4c03-842a-d2cb6f4bd96c.jpg?v=1642460560
53cc3653ded7dc10570_1762639392: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/woman-outside-brownstone_925x_d62fa07b-cf39-4726-ace5-ceebe9f277c3.jpg?v=1642460567
445afd289ba4fd99dc7_1771624215: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/putting-on-your-shoes_925x_04e71ada-dace-44d3-8b24-0c3cdda0bdf1.jpg?v=1642460577
a633701beabfefdcf48_1762639393: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/city-woman-fashion_925x_2x_952e16c3-72ba-4183-a17d-46cc36f0da09.jpg?v=1642460555
b2688f01c7b24a32b07_1757623722: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/young-female-models-denim_925x_8ae19ff2-79de-446f-9adb-8642e8ef8bcd.jpg?v=1642460559
451d3a4e7d87ba27cd3_1752606951: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/whitetee.jpg?v=1657030158
d2c9ccd17d8d4d7c7c3_1759180763: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/green-t-shirt.jpg?v=1628281944
52c3213321dcf667042_1771624218: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/blacktee.webp?v=1657030149
5b9acaebc32d2ce1fbd_1752606959: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/casual-fashion-woman_925x_e9a705d8-4a1a-473b-aa51-c3b60486c58d.jpg?v=1642460552
9bf408e9cc49aa77391_1752606959: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/leather-jacket-and-tea_925x_265bb656-b6dd-4560-869a-6e78c57a6644.jpg?v=1642460558
0ee06d2e1f14cf348d3_1762715491: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/red-plaid-shirt_925x_721b17ae-a883-4cf6-a1f5-9d9ab0ec78a7.jpg?v=1642460568
fc5966e0c73295686e3_1762715491: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/man-adjusts-blue-tuxedo-bowtie_925x_3c42051f-20aa-42c2-bf71-8bb9d6e4982e.jpg?v=1642460572
a9fb8094aa657520912_1770152688: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/black-bag-over-the-shoulder_925x_cafc7f9c-15db-4c00-8fd5-0ff9a3d4f6fb.jpg?v=1642460563
7e4c66c6cc03ca36539_1771624219: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/woman-in-the-city_925x_1ccce2ce-f11f-4b77-8608-a1e7660b8c10.jpg?v=1642460575
2026-02-28 22:48:33 [loknow_spiders] INFO: Uncached images: None
2026-02-28 22:48:33 [scrapy.extensions.feedexport] INFO: Stored jsonlines feed (25 items) in: file:///home/scrapyd/items/loknow_spiders_prod/ImageSpider/8dce860014f711f19b12fbebc751a6c4.jl
2026-02-28 22:48:33 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 7545,
'downloader/request_count': 23,
'downloader/request_method_count/HEAD': 23,
'downloader/response_bytes': 33275,
'downloader/response_count': 23,
'downloader/response_status_count/200': 23,
'dupefilter/filtered': 2,
'elapsed_time_seconds': 15.289374,
'feedexport/success_count/FileFeedStorage': 1,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2026, 2, 28, 22, 48, 33, 672913),
'item_scraped_count': 25,
'log_count/INFO': 20,
'log_count/WARNING': 1,
'memusage/max': 133296128,
Tail
2026-02-28 22:48:17 [scrapy.utils.log] INFO: Scrapy 2.10.0 started (bot: retriever)
2026-02-28 22:48:17 [scrapy.utils.log] INFO: Versions: lxml 5.1.0.0, libxml2 2.12.3, cssselect 1.2.0, parsel 1.9.0, w3lib 2.0.0, Twisted 22.4.0, Python 3.10.12 (main, Jan 26 2026, 14:55:28) [GCC 11.4.0], pyOpenSSL 24.1.0 (OpenSSL 3.2.1 30 Jan 2024), cryptography 42.0.5, Platform Linux-6.8.0-1030-aws-x86_64-with-glibc2.35
2026-02-28 22:48:17 [scrapy.addons] INFO: Enabled addons:
[]
2026-02-28 22:48:17 [scrapy.crawler] INFO: Overridden settings:
{'AUTOTHROTTLE_ENABLED': True,
'BOT_NAME': 'retriever',
'HTTPCACHE_EXPIRATION_SECS': 86400,
'IMAGES_STORE_S3_ACL': 'public-read',
'LOG_FILE': '/home/scrapyd/logs/loknow_spiders_prod/ImageSpider/8dce860014f711f19b12fbebc751a6c4.log',
'LOG_FORMATTER': 'loknow_spiders.logging.PoliteLogFormatter',
'LOG_LEVEL': 'INFO',
'NEWSPIDER_MODULE': 'loknow_spiders.spiders',
'SPIDER_MODULES': ['loknow_spiders.spiders'],
'TELNETCONSOLE_ENABLED': False,
'USER_AGENT': 'AdRetriever (https://adretriever.com)'}
2026-02-28 22:48:17 [py.warnings] WARNING: /home/scrapyd/venv/lib/python3.10/site-packages/scrapy/utils/request.py:248: ScrapyDeprecationWarning: '2.6' is a deprecated value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting.
It is also the default value. In other words, it is normal to get this warning if you have not defined a value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting. This is so for backward compatibility reasons, but it will change in a future version of Scrapy.
See the documentation of the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting for information on how to handle this deprecation.
return cls(crawler)
2026-02-28 22:48:17 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
'scrapy.extensions.memusage.MemoryUsage',
'scrapy.extensions.feedexport.FeedExporter',
'scrapy.extensions.logstats.LogStats',
'scrapy.extensions.throttle.AutoThrottle']
2026-02-28 22:48:17 [ImageSpider] INFO: Current Spider Environment set to: prod
2026-02-28 22:48:17 [ImageSpider] INFO: Retrieving AWS secret: spider_prod_secrets
2026-02-28 22:48:17 [botocore.credentials] INFO: Found credentials in shared credentials file: ~/.aws/credentials
2026-02-28 22:48:17 [ImageSpider] INFO: In base spider ml_kwargs is None and scrape_type is inventory
2026-02-28 22:48:17 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
'loknow_spiders.middlewares.ThrottlingRetryMiddleware',
'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
'scrapy.downloadermiddlewares.stats.DownloaderStats']
2026-02-28 22:48:17 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
'loknow_spiders.middlewares.RetrieverSpiderMiddleware',
'scrapy.spidermiddlewares.referer.RefererMiddleware',
'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
'scrapy.spidermiddlewares.depth.DepthMiddleware']
2026-02-28 22:48:18 [botocore.credentials] INFO: Found credentials in shared credentials file: ~/.aws/credentials
2026-02-28 22:48:18 [scrapy.middleware] INFO: Enabled item pipelines:
['loknow_spiders.pipelines.ImageUrlsValidationPipeline']
2026-02-28 22:48:18 [scrapy.core.engine] INFO: Spider opened
2026-02-28 22:48:18 [loknow_spiders] INFO: Scrape ID: a6b427ef
2026-02-28 22:48:18 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-02-28 22:48:33 [scrapy.core.engine] INFO: Closing spider (finished)
2026-02-28 22:48:33 [ImageSpider] INFO: RetrieverAPI is making requests to https://app.adretriever.com with headers {'Authorization': 'Token 96dfb640234f1d676d6fa726f8eae6e7aab44cda'}
2026-02-28 22:48:33 [root] INFO: Making post request to https://app.adretriever.com/api/validated_urls/a6b427ef
2026-02-28 22:48:33 [loknow_spiders] INFO: Cache hit rate: 100.0%
2026-02-28 22:48:33 [loknow_spiders] INFO: Cached images: 109218f5bb4ac3a3558_1771624204: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/menswear-blue-zip-up-jacket_925x_80e5306a-81a6-4e3a-9685-0e63dd748bf5.jpg?v=1642460564
7607d14ecbeed64708e_1762715482: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/autumn-photographer-taking-picture_925x_2e5bcb1e-aa52-4574-aa2b-a554109a1a00.jpg?v=1642460553
75b9fc34c38d2ae8186_1762715485: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/smiling-woman-poses_925x_d067181f-323e-4d66-a933-cb6a2157ce47.jpg?v=1642460570
2bb03e69e78e767581f_1770152675: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/striped-blouse-fashion_925x_93fb0dcf-2a54-4985-9577-dc50838a9ce5.jpg?v=1642460556
8d160aded99d03da5a6_1770152676: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/smiling-woman-on-snowy-afternoon_925x_4a0cd500-5929-42bf-a523-a5e18a435989.jpg?v=1642460562
1c39602a44e4d77c5a3_1772231982: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/young-hip-woman-at-carnival_925x_1e9b3587-daef-43d0-9ccc-5bf71b3b01c4.jpg?v=1642460566
1aff6a91980e83cb2c8_1762715489: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/womens-red-t-shirt_925x_b1df6e1b-03dd-454f-bc1f-7ff96edf9329.jpg?v=1642460574
f7451c9e878da4a8b1b_1762715489: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/urban-fashion_925x_c73cfbb5-b4c7-4338-bac1-0319c8e3bbc0.jpg?v=1642460571
d221008508702387789_1771624214: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/young-man-in-bright-fashion_925x_8776c187-6747-4815-a0cc-8b03b6dcfe13.jpg?v=1642460550
fb082d5c8abdcf30526_1770152679: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/mens-fall-fashion-jacket_925x_9f898bdc-bcc0-4c03-842a-d2cb6f4bd96c.jpg?v=1642460560
53cc3653ded7dc10570_1762639392: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/woman-outside-brownstone_925x_d62fa07b-cf39-4726-ace5-ceebe9f277c3.jpg?v=1642460567
445afd289ba4fd99dc7_1771624215: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/putting-on-your-shoes_925x_04e71ada-dace-44d3-8b24-0c3cdda0bdf1.jpg?v=1642460577
a633701beabfefdcf48_1762639393: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/city-woman-fashion_925x_2x_952e16c3-72ba-4183-a17d-46cc36f0da09.jpg?v=1642460555
b2688f01c7b24a32b07_1757623722: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/young-female-models-denim_925x_8ae19ff2-79de-446f-9adb-8642e8ef8bcd.jpg?v=1642460559
451d3a4e7d87ba27cd3_1752606951: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/whitetee.jpg?v=1657030158
d2c9ccd17d8d4d7c7c3_1759180763: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/green-t-shirt.jpg?v=1628281944
52c3213321dcf667042_1771624218: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/blacktee.webp?v=1657030149
5b9acaebc32d2ce1fbd_1752606959: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/casual-fashion-woman_925x_e9a705d8-4a1a-473b-aa51-c3b60486c58d.jpg?v=1642460552
9bf408e9cc49aa77391_1752606959: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/leather-jacket-and-tea_925x_265bb656-b6dd-4560-869a-6e78c57a6644.jpg?v=1642460558
0ee06d2e1f14cf348d3_1762715491: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/red-plaid-shirt_925x_721b17ae-a883-4cf6-a1f5-9d9ab0ec78a7.jpg?v=1642460568
fc5966e0c73295686e3_1762715491: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/man-adjusts-blue-tuxedo-bowtie_925x_3c42051f-20aa-42c2-bf71-8bb9d6e4982e.jpg?v=1642460572
a9fb8094aa657520912_1770152688: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/black-bag-over-the-shoulder_925x_cafc7f9c-15db-4c00-8fd5-0ff9a3d4f6fb.jpg?v=1642460563
7e4c66c6cc03ca36539_1771624219: https://cdn.shopify.com/s/files/1/0471/6701/3029/products/woman-in-the-city_925x_1ccce2ce-f11f-4b77-8608-a1e7660b8c10.jpg?v=1642460575
2026-02-28 22:48:33 [loknow_spiders] INFO: Uncached images: None
2026-02-28 22:48:33 [scrapy.extensions.feedexport] INFO: Stored jsonlines feed (25 items) in: file:///home/scrapyd/items/loknow_spiders_prod/ImageSpider/8dce860014f711f19b12fbebc751a6c4.jl
2026-02-28 22:48:33 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 7545,
'downloader/request_count': 23,
'downloader/request_method_count/HEAD': 23,
'downloader/response_bytes': 33275,
'downloader/response_count': 23,
'downloader/response_status_count/200': 23,
'dupefilter/filtered': 2,
'elapsed_time_seconds': 15.289374,
'feedexport/success_count/FileFeedStorage': 1,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2026, 2, 28, 22, 48, 33, 672913),
'item_scraped_count': 25,
'log_count/INFO': 20,
'log_count/WARNING': 1,
'memusage/max': 133296128,
'memusage/startup': 133296128,
'response_received_count': 23,
'scheduler/dequeued': 23,
'scheduler/dequeued/memory': 23,
'scheduler/enqueued': 23,
'scheduler/enqueued/memory': 23,
'start_time': datetime.datetime(2026, 2, 28, 22, 48, 18, 383539)}
2026-02-28 22:48:33 [scrapy.core.engine] INFO: Spider closed (finished)