2026-05-29 21:58:14 [scrapy.utils.log] INFO: Scrapy 2.10.0 started (bot: retriever)
2026-05-29 21:58:14 [scrapy.utils.log] INFO: Versions: lxml 5.1.0.0, libxml2 2.12.3, cssselect 1.2.0, parsel 1.9.0, w3lib 2.0.0, Twisted 22.4.0, Python 3.10.12 (main, Mar 3 2026, 11:56:32) [GCC 11.4.0], pyOpenSSL 24.1.0 (OpenSSL 3.2.1 30 Jan 2024), cryptography 42.0.5, Platform Linux-6.8.0-1030-aws-x86_64-with-glibc2.35
2026-05-29 21:58:14 [scrapy.addons] INFO: Enabled addons:
[]
2026-05-29 21:58:14 [scrapy.crawler] INFO: Overridden settings:
{'AUTOTHROTTLE_ENABLED': True,
'BOT_NAME': 'retriever',
'HTTPCACHE_EXPIRATION_SECS': 86400,
'IMAGES_STORE_S3_ACL': 'public-read',
'LOG_FILE': '/home/scrapyd/logs/loknow_spiders_prod/ImageSpider/7a899cee5ba911f1bdc0c3bbae71ecef.log',
'LOG_FORMATTER': 'loknow_spiders.logging.PoliteLogFormatter',
'LOG_LEVEL': 'INFO',
'NEWSPIDER_MODULE': 'loknow_spiders.spiders',
'SPIDER_MODULES': ['loknow_spiders.spiders'],
'TELNETCONSOLE_ENABLED': False,
'USER_AGENT': 'AdRetriever (https://adretriever.com)'}
2026-05-29 21:58:14 [py.warnings] WARNING: /home/scrapyd/venv/lib/python3.10/site-packages/scrapy/utils/request.py:248: ScrapyDeprecationWarning: '2.6' is a deprecated value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting.
It is also the default value. In other words, it is normal to get this warning if you have not defined a value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting. This is so for backward compatibility reasons, but it will change in a future version of Scrapy.
See the documentation of the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting for information on how to handle this deprecation.
return cls(crawler)
2026-05-29 21:58:14 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
'scrapy.extensions.memusage.MemoryUsage',
'scrapy.extensions.feedexport.FeedExporter',
'scrapy.extensions.logstats.LogStats',
'scrapy.extensions.throttle.AutoThrottle']
2026-05-29 21:58:14 [ImageSpider] INFO: Current Spider Environment set to: prod
2026-05-29 21:58:14 [ImageSpider] INFO: Retrieving AWS secret: spider_prod_secrets
2026-05-29 21:58:14 [botocore.credentials] INFO: Found credentials in shared credentials file: ~/.aws/credentials
2026-05-29 21:58:15 [ImageSpider] INFO: In base spider ml_kwargs is None and scrape_type is inventory
2026-05-29 21:58:15 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
'loknow_spiders.middlewares.ThrottlingRetryMiddleware',
'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
'scrapy.downloadermiddlewares.stats.DownloaderStats']
2026-05-29 21:58:15 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
'loknow_spiders.middlewares.RetrieverSpiderMiddleware',
'scrapy.spidermiddlewares.referer.RefererMiddleware',
'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
'scrapy.spidermiddlewares.depth.DepthMiddleware']
2026-05-29 21:58:16 [botocore.credentials] INFO: Found credentials in shared credentials file: ~/.aws/credentials
2026-05-29 21:58:16 [scrapy.middleware] INFO: Enabled item pipelines:
['loknow_spiders.pipelines.ImageUrlsValidationPipeline']
2026-05-29 21:58:16 [scrapy.core.engine] INFO: Spider opened
2026-05-29 21:58:16 [loknow_spiders] INFO: Scrape ID: baa9f915
2026-05-29 21:58:16 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-05-29 21:59:16 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-05-29 22:00:16 [scrapy.extensions.logstats] INFO: Crawled 10 pages (at 10 pages/min), scraped 10 items (at 10 items/min)
2026-05-29 22:01:16 [scrapy.extensions.logstats] INFO: Crawled 32 pages (at 22 pages/min), scraped 33 items (at 23 items/min)
2026-05-29 22:02:16 [scrapy.extensions.logstats] INFO: Crawled 55 pages (at 23 pages/min), scraped 56 items (at 23 items/min)
2026-05-29 22:03:16 [scrapy.extensions.logstats] INFO: Crawled 80 pages (at 25 pages/min), scraped 81 items (at 25 items/min)
2026-05-29 22:04:16 [scrapy.extensions.logstats] INFO: Crawled 100 pages (at 20 pages/min), scraped 101 items (at 20 items/min)
2026-05-29 22:04:46 [scrapy.core.engine] INFO: Closing spider (finished)
2026-05-29 22:04:46 [ImageSpider] INFO: RetrieverAPI is making requests to https://api.adretriever.com with headers {'Authorization': 'Token 96dfb640234f1d676d6fa726f8eae6e7aab44cda'}
2026-05-29 22:04:46 [root] INFO: Making post request to https://api.adretriever.com/api/validated_urls/baa9f915
2026-05-29 22:04:47 [loknow_spiders] INFO: Cache hit rate: 100.0%
2026-05-29 22:04:47 [loknow_spiders] INFO: Cached images: a50f080260b67756ee4_1713563580: https://drive.google.com/uc?export=download&id=1kS3nFT4hK_JHaz3PkQdWRxbV30WN4hFe
0f738f7b5f296455b2f_1749746116: https://drive.google.com/uc?export=download&id=1VImUSQWXWfLtQX45N3Cld62nZ3PA6Soy
1fabd73e26456b7882f_1776442298: https://drive.google.com/uc?export=download&id=1foL7K2XqvHc561P_XrxQyQV9MmTO1I-e
e871bdbbcbe7881fc47_1710882028: https://drive.google.com/uc?export=download&id=1q_xFrp9498wjEdhQvOkbKZGwTiDDrVzF
4b48fb5b3f89ba0eb79_1717624548: https://drive.google.com/uc?export=download&id=1nxA-T0G6fTIbFOOhH8x6JWQIitgcEFJC
64dd7ba64f088fdacd2_1710881940: https://drive.google.com/uc?export=download&id=1dqSOI0wiOuqrb-LUNdNoCHuTfK4kHS64
743f9caf819b86eedd4_1759759567: https://drive.google.com/uc?export=download&id=1m6boOoSXYUN-mfVnYe0-c329Ik19Pm6b
c9b19b89995b4aeec48_1732030052: https://drive.google.com/uc?export=download&id=1yAIHjq_WU-8vhqIDDfmhpRHV24Y9RoUn
c8b8f59717dde7a566e_1759759597: https://drive.google.com/uc?export=download&id=1vI8aDLpGeIL1SJK3eF-R6ZaM0W9WnLNP
31166f9a5e362d3fd13_1747155762: https://drive.google.com/uc?export=download&id=1OgEan4R05njOY6gsvchaRxZYQKerikSy
8cabf30b81df6e275d8_1723747289: https://drive.google.com/uc?export=download&id=1MsMV-NuwEZtH2vnQ4rdyoF69yBCT2ekO
a2ed501ab0a17ed5c29_1777389676: https://drive.google.com/uc?export=download&id=1jzNqsOHyAk83kGPhfRVxcVtYNHiHeynQ
ae2d5e23dd3e9e9467b_1759759566: https://drive.google.com/uc?export=download&id=11lz8NYcitdIY_lYq18tV0ro8D7QbzobQ
bc11069b3e7abf11e20_1776971524: https://drive.google.com/uc?export=download&id=1TVtjTia6rHfVTvK01WGQog7vT6lQ-bJA
d01751208d88ff494fd_1777389672: https://drive.google.com/uc?export=download&id=1eDUK5LA8uJqFEw0PMfCWnrJuVpoERF42
cf25d78d712815df6b2_1778604559: https://drive.google.com/uc?export=download&id=1mCutF_R75fnq_skocEJQSYWm41B1Xn26
014bb413854eef212ae_1759759499: https://drive.google.com/uc?export=download&id=1PdhHgaMB-HNVisNatV5TTW9EM_OqQih5
0072e4dbc4b2aaeadc7_1759759493: https://drive.google.com/uc?export=download&id=1N0Ev5hweINmVqi0FN9uOA_d9QtGraIev
8961e93f6226e4b11e0_1713563634: https://drive.google.com/uc?export=download&id=1m_oK88Di1maSZ13zAmu2cCPd42jS9D76
c5aedb7ce3455efde5f_1775747337: https://drive.google.com/uc?export=download&id=1pnLJuFm7G-v2gy06KabjFCzwN6yNJ49S
e2dba3e927c2196fb48_1713565774: https://drive.google.com/uc?export=download&id=1e6PKjW2aoc-9-yl2DG2FB3aB-unNf7ke
942412a839309ca3155_1731082662: https://drive.google.com/uc?export=download&id=1l11K9uf8WPUIXTvKMwYv3a9JLaO1tOxI
8eda8e0d6f7ced412db_1754523281: https://drive.google.com/uc?export=download&id=1kEeo9atuAdR0fHHF-yEx3xzGC6wzroDx
f2670f755cbbad961c8_1751920687: https://drive.google.com/uc?export=download&id=1PtKX9JkGPPOxLTNr1T66SUUUbaM1iMbD
8cdd579497ec1fe0219_1710882086: https://drive.google.com/uc?export=download&id=1T7KPRYZssREg1RwTNEQP5fA5OeYFPFv2
cf4a954d660d91318f3_1750702242: https://drive.google.com/uc?export=download&id=1QEa_uvddBWtajI8T1ia8Pilsns2aoulX
3e7fca43163a05374b3_1751920690: https://drive.google.com/uc?export=download&id=1VJx12JOk94IZYda4Slj6uew26dVhY26D
da9a183a1c17d05c7f9_1749746150: https://drive.google.com/uc?export=download&id=1lTL3ntCYbM8AtyMncC4_EeVUAwVQJ0Cy
f074e419bf4cba5c2fd_1711059105: https://drive.google.com/uc?export=download&id=1cE3h249UY2K0m8O5ElsrdJXj0TSHP1pJ
ec1da4b49e4115cac64_1761074350: https://drive.google.com/uc?export=download&id=14aMJ2ebhO4NFQy5gmsJawuloHkZNmmtG
00f3ddcd505d029918c_1732117741: https://drive.google.com/uc?export=download&id=15Ci-7cCMmWIuuTl3ch7R7V72ig_Y9KI3
749f44101252b423a8f_1713563461: https://drive.google.com/uc?export=download&id=1BSsIilwqYjhNMC0pMo-k3Rqq4P3EQljM
b2b628da416da7a9baf_1730998271: https://drive.google.com/uc?export=download&id=1rMmE8nxGkc-1kKRRbU9nkuem_nCtAu9w
8151927db0699230d16_1746200936: https://drive.google.com/uc?export=download&id=1NydmM6mvyU50d1ugoRsgFsrzaSNr9gFE
c6f7fb28add0e01e89a_1717624699: https://drive.google.com/uc?export=download&id=12_kY8QgK03zuiFBDvEgiV9_wshWf6rDu
5eb0055708dd35a505d_1747771829: https://drive.google.com/uc?export=download&id=1Zw5DbNajiJmT30D8VGj8gAAxP71BV2zA
e761194c41b9d656155_1748543432: https://drive.google.com/uc?export=download&id=1-O2psfGffWPJxZdUxyEGwzfQN2pfcx34
7afc209c37451b981da_1710882031: https://drive.google.com/uc?export=download&id=1VKe1YMr27CnaQKtMPHm_wvtzf4AHhp85
eaf1ca0ed423e89c0fe_1777323671: https://drive.google.com/uc?export=download&id=1zRl0umOH4Xr72wM6SL1K8eH1Qu4-iebf
07089a034c847b37b5f_1748454929: https://drive.google.com/uc?export=download&id=1Wa9BF6yg-1cH7OhkA_0Owuih0bPpmbLH
9b2da42a2e9828ee08d_1747771827: https://drive.google.com/uc?export=download&id=1uxs9E3pwX7hFyh5Ikn2Ah9kB_XVrgUR7
05c3f5115743b5196d0_1759759597: https://drive.google.com/uc?export=download&id=1LX1AxVMV7Sta77EU2GwPHeorc9Clq3Zs
0ad69683dea9780d1db_1747779822: https://drive.google.com/uc?export=download&id=1TMtPrMbfE1Rbk4iaYXYnt35Cb38Et3a9
6e17eb43a987e06fd5a_1779907057: https://drive.google.com/uc?export=download&id=1qbDGEixwoaqT-ETHXB_1A6pnEWXtKTiV
a936856eea11d09ea2f_1778083137: https://drive.google.com/uc?export=download&id=19RapYW9bX6DAr4urdfCa7wK5SJNrsHLG
687193971769fe4b421_1745537700: https://drive.google.com/uc?export=download&id=1KhVhRkwDgMjILHr7qJqCTAIeFkKJ7geR
ebe67eb1129ffd92b57_1724344991: https://drive.google.com/uc?export=download&id=1-wD53bKEt5IsbeW7YZl2mNsrUY9FsdT8
712c6fa5a9615dbf048_1749746188: https://drive.google.com/uc?export=download&id=1-tCl0MJ-v7im4WTG7GUSDqjPxXUdB9Q4
c3e66156e9005a9b998_1715614498: https://drive.google.com/uc?export=download&id=1QrLozyNUWyLgMrQ6z3CsL5E2L5bXkNUV
7f7b964deeb1b98d252_1710882178: https://drive.google.com/uc?export=download&id=1pGciFExiIxt7YfvDDzVUi7-A-cPPSYM6
17338968ca3536546ad_1779290372: https://drive.google.com/uc?export=download&id=1qWGUNog6cHmDSteEjbOJuRhw1W8EYZm1
bfb1a7d0cc16ee66fae_1751920714: https://drive.google.com/uc?export=download&id=1ipHPaVqKPyD2QcsIRjLm78yruIMFkcuG
e32f11411879fc0f750_1720541718: https://drive.google.com/uc?export=download&id=1QGBMFAbAJfjw6CJVUEtP78bvWAGN6SvX
f9f20233540e3f00480_1749746121: https://drive.google.com/uc?export=download&id=1wOdQ4C6OIJ-ScJ2hKWTsa_B-KNDFOdRa
666086b9ee8c4d9471c_1719611281: https://drive.google.com/uc?export=download&id=1xEkFpOgmkYq1A9qzB0-X9w0DtVbMxlG7
1e4b76b26c3b73a978a_1711059213: https://drive.google.com/uc?export=download&id=1BZptopXsa4uF4ZbAwm5i_kTmR3EoDfOb
76988f08055dcd11b82_1717624632: https://drive.google.com/uc?export=download&id=1obsIgflNuooIS9vbvaVRL77rwl358_Xg
ed5cfe7ad5d70ee273c_1723744101: https://drive.google.com/uc?export=download&id=1c4tsuVheYYDRt5w7vbdw9ZMP44DM4OS_
b7b125282e4eb750639_1748546888: https://drive.google.com/uc?export=download&id=1m-MXHCzWPv_pL7ysCcMxuVXsULo5PWaz
02b17089a4c48425760_1776971520: https://drive.google.com/uc?export=download&id=1CE0a72UR7CciE3hTGf2PparjzelvBSGZ
6584228adb478d1c7a4_1759759457: https://drive.google.com/uc?export=download&id=1oOubBRL_MeIOpZzGfLhp8GnfZx3fNg-T
f957d4163e786cc9458_1720537030: https://drive.google.com/uc?export=download&id=14yCokvVVFTn_DyI_yUazzmF8UVdSC1Na
c30d3c4346199def1a5_1713563695: https://drive.google.com/uc?export=download&id=1MT8X-3jP9xrMZa_-hdUHGpmicSXDq0SN
2d4fd0f23dc83de449c_1713563371: https://drive.google.com/uc?export=download&id=1SuoCnMGl8SMx5jG9w9fL-XtQbAhtcamh
538a676bd00a15f625b_1747852529: https://drive.google.com/uc?export=download&id=13enMjOLLOkwP53fHKdGOKQBK2IRlS8l4
9358835f82476a5bb67_1753124328: https://drive.google.com/uc?export=download&id=1XmSox2eeL4J5d5QjrASEDsjiS4T4G3rf
02a1feb9ec4f6696e5c_1717624601: https://drive.google.com/uc?export=download&id=1z2ETFU_wYRuSyJzKzm9yfQ1W-0e2_ZE3
70a241e4a7dda8369ec_1749746151: https://drive.google.com/uc?export=download&id=1Ae2fRwQ9XCurb_9VTuHn4Wnh4sMgzRV2
712d261ad74a7759bdb_1723744156: https://drive.google.com/uc?export=download&id=18WP6AL1m2_fS3RVJdzrPgKvfL9Yasrcn
192052c805c7a25abed_1723744205: https://drive.google.com/uc?export=download&id=1sHPd2-Yo70ulUCRhP4o8D1bRdsKH0YDz
a0d4905cd811c1e56f5_1713563689: https://drive.google.com/uc?export=download&id=1-k9CUBD7YhnGUyH4aahyugYuQr9MKnxJ
dc729d398d2e799f608_1732544607: https://drive.google.com/uc?export=download&id=182MPXvAooRA5GObUhrYBv71oTzNuhH38
f8e2c7703a83f0ded33_1713563511: https://drive.google.com/uc?export=download&id=1Ck9MJgIBIny28hNql38GCdhlMKbVyK0m
a4befee265aebe86a33_1710886290: https://drive.google.com/uc?export=download&id=1PqIE68J7hgs1VrCyQ3yL3iqI-3JiulOG
f58b3fce63799af583b_1717624735: https://drive.google.com/uc?export=download&id=1BXSP96X9xP6O6UWwtAp0gkIOq5Ilzz-G
9942914216f3d02396c_1751920659: https://drive.google.com/uc?export=download&id=14hinhEIIETGdJO8wRQJhVcEvya0aSXC3
d6e65a78c1e60c44b86_1745537697: https://drive.google.com/uc?export=download&id=1-FcgGClB9dIbVGbzTMjUmYtoYAFGJY4C
0cec1a1cb642511394b_1748546919: https://drive.google.com/uc?export=download&id=1RODyv7zdnRCmm57aaRjKrxG0Y0bn8DEE
d32a39cb5449a726151_1746200934: https://drive.google.com/uc?export=download&id=1b3utlOkHiUucCxPO4drvJ8G1czHa8MZD
7bc5a3484f0f1d04dea_1759759454: https://drive.google.com/uc?export=download&id=1ScG5-NenQPx4gJiCDxMFHRiT0DpynhTU
07dfc8541cbc5ef6f8b_1732648081: https://drive.google.com/uc?export=download&id=1PuFlaJVMI8EPllhEQvm3I2TIsWQSXtBP
79cc5fb4c1a950c8972_1713563420: https://drive.google.com/uc?export=download&id=1rByFwHh9o5jDtTnXCRE_cCi_vmZ5ZZ_D
8730cd412241c89379c_1717624729: https://drive.google.com/uc?export=download&id=1ZUOh6kunjfP0mSEtVF08LaCD2YowLZyC
5f3269aa06e1559af2e_1710881945: https://drive.google.com/uc?export=download&id=13RRTKoB5jSJtu8cNRYLoDW1pULSJywP-
7f170e13212380a7106_1779217341: https://drive.google.com/uc?export=download&id=1AB-WOHDowYLRpaqosVgO7kmZWieYW0NJ
e9bb5e7d547875ddd85_1778083139: https://drive.google.com/uc?export=download&id=1bx42LTpbV74CsReOxvzUU-Ckm-vGE2pi
cc4b2629b70383c0eba_1776435283: https://drive.google.com/uc?export=download&id=1jWBfYst5ZMwPYzkcfHifuo4NydpQRjup
717e7a538c7a19e681b_1713563473: https://drive.google.com/uc?export=download&id=1DP5cGc-bU_J6KRvfxkGumlbRnEepNQLF
fc852df96c7d68e234f_1748546920: https://drive.google.com/uc?export=download&id=1PvOhEUMb972arJSP_DVNNONAxDn5ygii
73945b421732d8de797_1717624668: https://drive.google.com/uc?export=download&id=1Z7xl-jnPzQxD3c80mbj72XZxPHdfYTXG
8ecffcc4b1db8f2b146_1774540716: https://drive.google.com/uc?export=download&id=1tMGC5OmL34m97MTVzzBbRtfSb5pDjzIJ
3c46801173e71c81594_1777323667: https://drive.google.com/uc?export=download&id=17EUh03CZy2B_lkSsO189_YEHAJE7lIJk
773d64161fc4d9504ae_1710884666: https://drive.google.com/uc?export=download&id=14rgp9LAoBWPRQytDYQsR3f1yx_-PAKLU
4a900d4883fda86fcea_1713565695: https://drive.google.com/uc?export=download&id=1lGpz_T74mS66gVjoeqerDVlMtwLqTOzB
03ea42ebe8d88885fb3_1713563345: https://drive.google.com/uc?export=download&id=1TrTy0c-ktvTfDbiNw-VgnYkoxhZHo_Ib
fdf29206079d28cbfa4_1711059164: https://drive.google.com/uc?export=download&id=1BKtYHziVbVUlTcKgX_vynydYdZBjxpGB
a536d39a4913de2aec5_1713563518: https://drive.google.com/uc?export=download&id=1kosx01m1xwOLBsXhCbi0lzjWxVYXminD
34e45c30730e0689f62_1760631858: https://drive.google.com/uc?export=download&id=1gq3YLUUSoRAZnZzl3qd5nNJYQsdf03bb
b021cfc470cae08d68b_1748030401: https://drive.google.com/uc?export=download&id=1UlSn6KdqqiGrN2oF9LxPgKren9kCHBAR
3d17cf12c69c4149e03_1710882664: https://drive.google.com/uc?export=download&id=1SADidkisbkOCG_F0Jcjz78GEem71JohX
a3810f4b26670d59b14_1717624607: https://drive.google.com/uc?export=download&id=1xrD6KffzsCEh_InskGLd6bzELzaAhDuc
a3d4af6f6be3b74a2e7_1759759600: https://drive.google.com/uc?export=download&id=19G33-CAdhtjJ_ERfVVc-QWXZy3UfLlJq
0adf3e83abe5ea43367_1775166142: https://drive.google.com/uc?export=download&id=1gDHeiZdQMRyg1Ruzq3DISdsde-dah9tN
fb294cc77cead6aa96d_1759759460: https://drive.google.com/uc?export=download&id=1OKALJQkiUTq0Rgy00TIEyAMCbWhl-ue4
eed8fc5d88dc39f5186_1717624570: https://drive.google.com/uc?export=download&id=1VQNUZ_fPzAFF1s_DpoGtaeP9NNKi62ir
83bc418740cfea269af_1723744091: https://drive.google.com/uc?export=download&id=1wTVZ70uf2IC0FyVJDqGUK5yPY9iQdKTn
e9b3d2db968a23d5a75_1777065344: https://drive.google.com/uc?export=download&id=1qtPUL7rEndWRj_Bwqm4ulKq0JjbexU8L
74cad9ec72775731fb5_1747771685: https://drive.google.com/uc?export=download&id=15pxSE1t9cSt-hcM5t73_9I0WmfqL_A9J
2c0d98224adc1ec4214_1713563489: https://drive.google.com/uc?export=download&id=1n75codOWIYQFyYzomWNChmfr7a03CZHN
2e7962236ac618c0e64_1749746184: https://drive.google.com/uc?export=download&id=1ssjbr_1-AwgkaQCaUo09HheEcReAJiEa
fa5ce0d10b2ed81e8d3_1759759605: https://drive.google.com/uc?export=download&id=1WNy5MS0uj2pb7wid0Z8hPTsfQ95w228g
dbc784fe227a59f5ac7_1747159311: https://drive.google.com/uc?export=download&id=1A3Qb_L3N6czsroZZmK9LVcYzApWjO3Pk
0d2f63b8abb2bbc38a5_1759759453: https://drive.google.com/uc?export=download&id=18ZedJOwKZTXmdQDmfaZ3W80PFdWU77gs
a8d361d51d9689e0b71_1717624640: https://drive.google.com/uc?export=download&id=1jA-cXjXC8YhH_ROKCQsD2twZJgGn9KXV
03d0c74cfcd807147eb_1723744251: https://drive.google.com/uc?export=download&id=1pNPUxwrHC0ZEG4xdVm7DCqXb5-SXFdZf
751f43f9a24e77143dd_1723744179: https://drive.google.com/uc?export=download&id=1n09gM7coJMvsVdcb77SZjuVWdWzMiDPz
2026-05-29 22:04:47 [loknow_spiders] INFO: Uncached images: None
2026-05-29 22:04:47 [scrapy.extensions.feedexport] INFO: Stored jsonlines feed (119 items) in: file:///home/scrapyd/items/loknow_spiders_prod/ImageSpider/7a899cee5ba911f1bdc0c3bbae71ecef.jl
2026-05-29 22:04:47 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 65888,
'downloader/request_count': 232,
'downloader/request_method_count/HEAD': 232,
'downloader/response_bytes': 786841,
'downloader/response_count': 232,
'downloader/response_status_count/200': 116,
'downloader/response_status_count/303': 116,
'dupefilter/filtered': 3,
'elapsed_time_seconds': 390.623557,
'feedexport/success_count/FileFeedStorage': 1,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2026, 5, 29, 22, 4, 47, 64202),
'item_scraped_count': 119,
'log_count/INFO': 26,
'log_count/WARNING': 1,
'memusage/max': 135458816,
'memusage/startup': 133115904,
'response_received_count': 116,
'scheduler/dequeued': 232,
'scheduler/dequeued/memory': 232,
'scheduler/enqueued': 232,
'scheduler/enqueued/memory': 232,
'start_time': datetime.datetime(2026, 5, 29, 21, 58, 16, 440645)}
2026-05-29 22:04:47 [scrapy.core.engine] INFO: Spider closed (finished)