Desktop version Jobs
  • Analysis
  • Categories
  • Charts
  • Logs
  • Crawler.stats
  • projectloknow_spiders_prod
    spiderImageSpider
    job482514b65bab11f1bdc0c3bbae71ecef
    first_log_time2026-05-29 22:11:11
    latest_log_time2026-05-29 22:13:13
    runtime0:02:02
    crawled_pages 162
    scraped_items 181
    shutdown_reasonN/A
    finish_reasonfinished
    log_critical_count0
    log_error_count0
    log_warning_count32
    log_redirect_count0
    log_retry_count0
    log_ignore_count0
    latest_crawl
    latest_scrape
    latest_log
    current_time
    latest_itemN/A
    • WARNING+

    • warning_logs
      last 10 of 32

      2026-05-29 22:12:34 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21710-1.JPG
      2026-05-29 22:12:34 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21702-2.JPG
      2026-05-29 22:12:34 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21702-1.JPG
      2026-05-29 22:12:34 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21794-1.JPG
      2026-05-29 22:12:38 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21710-3.JPG
      2026-05-29 22:13:10 [scrapy.pipelines.files] WARNING: File (empty-content): Empty file from <GET https://www.wsnorth.com/wp-content/uploads/truck_inventory/21787-1.JPG> referred in <None>: no-content
      2026-05-29 22:13:12 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21863-1.JPG
      2026-05-29 22:13:12 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21861-2.JPG
      2026-05-29 22:13:12 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21863-3.JPG
      2026-05-29 22:13:13 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21861-1.JPG

      INFO

      DEBUG

    • scrapy_version

      2.10.0
    • latest_stat

      2026-05-29 22:12:15 [scrapy.extensions.logstats] INFO: Crawled 99 pages (at 99 pages/min), scraped 120 items (at 120 items/min)
    • Head

      2026-05-29 22:11:11 [scrapy.utils.log] INFO: Scrapy 2.10.0 started (bot: retriever)
      2026-05-29 22:11:11 [scrapy.utils.log] INFO: Versions: lxml 5.1.0.0, libxml2 2.12.3, cssselect 1.2.0, parsel 1.9.0, w3lib 2.0.0, Twisted 22.4.0, Python 3.10.12 (main, Mar  3 2026, 11:56:32) [GCC 11.4.0], pyOpenSSL 24.1.0 (OpenSSL 3.2.1 30 Jan 2024), cryptography 42.0.5, Platform Linux-6.8.0-1030-aws-x86_64-with-glibc2.35
      2026-05-29 22:11:11 [scrapy.addons] INFO: Enabled addons:
      []
      2026-05-29 22:11:11 [scrapy.crawler] INFO: Overridden settings:
      {'AUTOTHROTTLE_ENABLED': True,
       'BOT_NAME': 'retriever',
       'HTTPCACHE_EXPIRATION_SECS': 86400,
       'IMAGES_STORE_S3_ACL': 'public-read',
       'LOG_FILE': '/home/scrapyd/logs/loknow_spiders_prod/ImageSpider/482514b65bab11f1bdc0c3bbae71ecef.log',
       'LOG_FORMATTER': 'loknow_spiders.logging.PoliteLogFormatter',
       'LOG_LEVEL': 'INFO',
       'NEWSPIDER_MODULE': 'loknow_spiders.spiders',
       'SPIDER_MODULES': ['loknow_spiders.spiders'],
       'TELNETCONSOLE_ENABLED': False,
       'USER_AGENT': 'AdRetriever (https://adretriever.com)'}
      2026-05-29 22:11:11 [py.warnings] WARNING: /home/scrapyd/venv/lib/python3.10/site-packages/scrapy/utils/request.py:248: ScrapyDeprecationWarning: '2.6' is a deprecated value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting.
      
      It is also the default value. In other words, it is normal to get this warning if you have not defined a value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting. This is so for backward compatibility reasons, but it will change in a future version of Scrapy.
      
      See the documentation of the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting for information on how to handle this deprecation.
        return cls(crawler)
      
      2026-05-29 22:11:12 [scrapy.middleware] INFO: Enabled extensions:
      ['scrapy.extensions.corestats.CoreStats',
       'scrapy.extensions.memusage.MemoryUsage',
       'scrapy.extensions.feedexport.FeedExporter',
       'scrapy.extensions.logstats.LogStats',
       'scrapy.extensions.throttle.AutoThrottle']
      2026-05-29 22:11:12 [ImageSpider] INFO: Current Spider Environment set to: prod
      2026-05-29 22:11:12 [ImageSpider] INFO: Retrieving AWS secret: spider_prod_secrets
      2026-05-29 22:11:12 [botocore.credentials] INFO: Found credentials in shared credentials file: ~/.aws/credentials
      2026-05-29 22:11:12 [ImageSpider] INFO: In base spider ml_kwargs is None and scrape_type is inventory
      2026-05-29 22:11:12 [scrapy.middleware] INFO: Enabled downloader middlewares:
      ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
       'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
       'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
       'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
       'loknow_spiders.middlewares.ThrottlingRetryMiddleware',
       'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
       'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
       'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
       'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
       'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
       'scrapy.downloadermiddlewares.stats.DownloaderStats']
      2026-05-29 22:11:12 [scrapy.middleware] INFO: Enabled spider middlewares:
      ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
       'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
       'loknow_spiders.middlewares.RetrieverSpiderMiddleware',
       'scrapy.spidermiddlewares.referer.RefererMiddleware',
       'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
       'scrapy.spidermiddlewares.depth.DepthMiddleware']
      2026-05-29 22:11:14 [botocore.credentials] INFO: Found credentials in shared credentials file: ~/.aws/credentials
      2026-05-29 22:11:14 [scrapy.middleware] INFO: Enabled item pipelines:
      ['loknow_spiders.pipelines.ImageUrlsValidationPipeline']
      2026-05-29 22:11:14 [scrapy.core.engine] INFO: Spider opened
      2026-05-29 22:11:14 [loknow_spiders] INFO: Scrape ID: 5eff230b
      
      2026-05-29 22:11:14 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
      2026-05-29 22:11:27 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21616-2.JPG
      
      2026-05-29 22:11:27 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21616-1.JPG
      2026-05-29 22:11:33 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21685-1.JPG
      2026-05-29 22:11:35 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21685-2.JPG
      
      2026-05-29 22:11:38 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21685-3.JPG
      2026-05-29 22:11:40 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21684-1.JPG
      2026-05-29 22:11:43 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21684-2.JPG
      
      2026-05-29 22:11:46 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21684-3.JPG
      2026-05-29 22:11:58 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21742-1.JPG
      
      2026-05-29 22:11:58 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21742-2.JPG
      2026-05-29 22:12:01 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21415-1.JPG
      2026-05-29 22:12:01 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21816-2.JPG
      2026-05-29 22:12:01 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21415-3.JPG
      
      2026-05-29 22:12:02 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21816-1.JPG
      2026-05-29 22:12:15 [scrapy.extensions.logstats] INFO: Crawled 99 pages (at 99 pages/min), scraped 120 items (at 120 items/min)
      2026-05-29 22:12:18 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21681-1.JPG
      
      2026-05-29 22:12:19 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21681-3.JPG
      2026-05-29 22:12:27 [scrapy.pipelines.files] WARNING: File (empty-content): Empty file from <GET https://www.wsnorth.com/wp-content/uploads/truck_inventory/21783-1.JPG> referred in <None>: no-content
      
      2026-05-29 22:12:27 [loknow_spiders] INFO: Failed to download image: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21783-1.JPG -- [Failure instance: Traceback (failure with no frames): <class 'scrapy.pipelines.files.FileException'>: empty-content
      ]
      2026-05-29 22:12:31 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21703-2.JPG
      2026-05-29 22:12:31 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21703-1.JPG
      2026-05-29 22:12:33 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21710-2.JPG
      2026-05-29 22:12:34 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21794-2.JPG
      2026-05-29 22:12:34 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21710-1.JPG
      2026-05-29 22:12:34 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21702-2.JPG
      2026-05-29 22:12:34 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21702-1.JPG
      2026-05-29 22:12:34 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21794-1.JPG
      
      2026-05-29 22:12:38 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21710-3.JPG
      2026-05-29 22:13:10 [scrapy.pipelines.files] WARNING: File (empty-content): Empty file from <GET https://www.wsnorth.com/wp-content/uploads/truck_inventory/21787-1.JPG> referred in <None>: no-content
      
      2026-05-29 22:13:10 [loknow_spiders] INFO: Failed to download image: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21787-1.JPG -- [Failure instance: Traceback (failure with no frames): <class 'scrapy.pipelines.files.FileException'>: empty-content
      ]
      2026-05-29 22:13:12 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21863-1.JPG
      2026-05-29 22:13:12 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21861-2.JPG
      2026-05-29 22:13:12 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21863-3.JPG
      2026-05-29 22:13:13 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21861-1.JPG
      2026-05-29 22:13:13 [scrapy.core.engine] INFO: Closing spider (finished)
      2026-05-29 22:13:13 [ImageSpider] INFO: RetrieverAPI is making requests to https://api.adretriever.com with headers {'Authorization': 'Token 96dfb640234f1d676d6fa726f8eae6e7aab44cda'}
      2026-05-29 22:13:13 [root] INFO: Making post request to https://api.adretriever.com/api/validated_urls/5eff230b
      2026-05-29 22:13:13 [loknow_spiders] INFO: Cache hit rate: 82.87%
      2026-05-29 22:13:13 [loknow_spiders] INFO: Cached images: 55647ea2f5180e9851d_1744474575: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21458-1.JPG
      a07f66c6ba8811dc79b_1744474576: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21458-2.JPG
      3e565d7c3844c84cd3e_1744474576: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21458-3.JPG
      fdf6d391f6aa850d3b7_1744474534: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21454-1.JPG
      42a18067152b545b293_1744474535: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21454-2.JPG
    • Tail

      2026-05-29 22:11:14 [loknow_spiders] INFO: Scrape ID: 5eff230b
      
      2026-05-29 22:11:14 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
      2026-05-29 22:11:27 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21616-2.JPG
      
      2026-05-29 22:11:27 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21616-1.JPG
      2026-05-29 22:11:33 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21685-1.JPG
      2026-05-29 22:11:35 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21685-2.JPG
      
      2026-05-29 22:11:38 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21685-3.JPG
      2026-05-29 22:11:40 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21684-1.JPG
      2026-05-29 22:11:43 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21684-2.JPG
      
      2026-05-29 22:11:46 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21684-3.JPG
      2026-05-29 22:11:58 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21742-1.JPG
      
      2026-05-29 22:11:58 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21742-2.JPG
      2026-05-29 22:12:01 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21415-1.JPG
      2026-05-29 22:12:01 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21816-2.JPG
      2026-05-29 22:12:01 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21415-3.JPG
      
      2026-05-29 22:12:02 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21816-1.JPG
      2026-05-29 22:12:15 [scrapy.extensions.logstats] INFO: Crawled 99 pages (at 99 pages/min), scraped 120 items (at 120 items/min)
      2026-05-29 22:12:18 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21681-1.JPG
      
      2026-05-29 22:12:19 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21681-3.JPG
      2026-05-29 22:12:27 [scrapy.pipelines.files] WARNING: File (empty-content): Empty file from <GET https://www.wsnorth.com/wp-content/uploads/truck_inventory/21783-1.JPG> referred in <None>: no-content
      
      2026-05-29 22:12:27 [loknow_spiders] INFO: Failed to download image: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21783-1.JPG -- [Failure instance: Traceback (failure with no frames): <class 'scrapy.pipelines.files.FileException'>: empty-content
      ]
      2026-05-29 22:12:31 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21703-2.JPG
      2026-05-29 22:12:31 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21703-1.JPG
      2026-05-29 22:12:33 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21710-2.JPG
      2026-05-29 22:12:34 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21794-2.JPG
      2026-05-29 22:12:34 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21710-1.JPG
      2026-05-29 22:12:34 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21702-2.JPG
      2026-05-29 22:12:34 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21702-1.JPG
      2026-05-29 22:12:34 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21794-1.JPG
      
      2026-05-29 22:12:38 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21710-3.JPG
      2026-05-29 22:13:10 [scrapy.pipelines.files] WARNING: File (empty-content): Empty file from <GET https://www.wsnorth.com/wp-content/uploads/truck_inventory/21787-1.JPG> referred in <None>: no-content
      
      2026-05-29 22:13:10 [loknow_spiders] INFO: Failed to download image: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21787-1.JPG -- [Failure instance: Traceback (failure with no frames): <class 'scrapy.pipelines.files.FileException'>: empty-content
      ]
      2026-05-29 22:13:12 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21863-1.JPG
      2026-05-29 22:13:12 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21861-2.JPG
      2026-05-29 22:13:12 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21863-3.JPG
      2026-05-29 22:13:13 [ImageSpider] WARNING: Failed to validate image: Image not accessible - https://www.wsnorth.com/wp-content/uploads/truck_inventory/21861-1.JPG
      2026-05-29 22:13:13 [scrapy.core.engine] INFO: Closing spider (finished)
      2026-05-29 22:13:13 [ImageSpider] INFO: RetrieverAPI is making requests to https://api.adretriever.com with headers {'Authorization': 'Token 96dfb640234f1d676d6fa726f8eae6e7aab44cda'}
      2026-05-29 22:13:13 [root] INFO: Making post request to https://api.adretriever.com/api/validated_urls/5eff230b
      2026-05-29 22:13:13 [loknow_spiders] INFO: Cache hit rate: 82.87%
      2026-05-29 22:13:13 [loknow_spiders] INFO: Cached images: 55647ea2f5180e9851d_1744474575: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21458-1.JPG
      a07f66c6ba8811dc79b_1744474576: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21458-2.JPG
      3e565d7c3844c84cd3e_1744474576: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21458-3.JPG
      fdf6d391f6aa850d3b7_1744474534: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21454-1.JPG
      42a18067152b545b293_1744474535: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21454-2.JPG
      5b7d8b3e56705f3f73d_1744474536: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21454-3.JPG
      29e8c02562640606beb_1744474569: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21457-1.JPG
      3bef9bb99d2cc6e8750_1744474570: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21457-2.JPG
      5e365b010207c9e668c_1744474570: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21457-3.JPG
      c882ce28fc8951839c0_1753119850: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21781-1.JPG
      492d6fca85df1f7c5e4_1753119850: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21781-2.JPG
      d03fdf4c3485ba7d51a_1753119851: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21781-3.JPG
      cb640b8bc2cda101931_1755179369: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21791-1.JPG
      0edafae667506b09c85_1755179370: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21791-2.JPG
      7995f5631d609b4770d_1755179371: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21791-3.JPG
      014fc04eef702507009_1741754267: https://www.wsnorth.com/wp-content/uploads/2025/03/Western-Star-Trucks-North-Logo-80h.webp
      0a06595919c622c6c68_1763184824: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21845-1.JPG
      49184d143c05a7b5166_1763184824: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21845-2.JPG
      0bd4a7ce22f1e950bd8_1763184825: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21845-3.JPG
      32c9feab9bdeae4be8c_1742919107: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21604-1.JPG
      d6a84c89bafaaac0429_1742919107: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21604-2.JPG
      a0098d0d6719ace92cb_1742919108: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21604-3.JPG
      d2bfb680f6eada30e16_1753796605: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21412-1.JPG
      0f088182744f247c831_1753796606: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21412-2.JPG
      386e362ed1f3bcff26b_1753796606: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21412-3.JPG
      9db1693bad33805d27d_1744474747: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21678-1.JPG
      5443033ccefb0f054b3_1744474748: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21678-2.JPG
      0e8f10f87f86c304f40_1744474748: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21678-3.JPG
      40f5bffa5bf70a64b95_1775577615: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21680-1.JPG
      45e34002587e10c99b8_1775577615: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21680-2.JPG
      e86d29c50c732888b7b_1775577616: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21680-3.JPG
      e3c03d5422ed55038ef_1744474522: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21536-1.JPG
      927d8d83c641c0bf0eb_1744474523: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21536-2.JPG
      1056ed8b1559bd5bab2_1744474524: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21536-3.JPG
      b1c934d39b762fd5a98_1744474884: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21588-1.JPG
      6bf027bb832667547da_1744474884: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21588-2.JPG
      11433a5743978481461_1744474884: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21588-3.JPG
      e888f7cfb8c2cd0a9c0_1744474864: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21590-1.JPG
      7092e31d76993d211b3_1744474865: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21590-2.JPG
      5ff1589e0114ba22b63_1744474866: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21590-3.JPG
      789a78b4ebca2679008_1744474621: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21526-1.JPG
      f42139fa429d0ade2ad_1744474622: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21526-2.JPG
      6ede9ef4f7cf95c7199_1744474622: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21526-3.JPG
      630cc3b529b87089dea_1744474511: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21763-1.JPG
      75e231b6cb62bb5c2f5_1744474511: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21763-2.JPG
      219c8669e88e629871f_1744474512: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21763-3.JPG
      3c911a0e3471ba5ca64_1744474636: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21535-1.JPG
      39ffaa456d5a95654bc_1744474637: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21535-2.JPG
      f776f1d57ab3c8a2df5_1744474637: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21535-3.JPG
      478e0b46378d927a56d_1744474828: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21446-1.JPG
      0d1ea58bb63cef5e5ed_1744474828: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21446-2.JPG
      d4d2ef8a8b0ca9f23d8_1744474829: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21446-3.JPG
      d8b499e14ce61803917_1771897524: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21686-1.JPG
      3ba01f1941730719d9a_1771897525: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21686-3.JPG
      5cd5783117c93107d0c_1753969488: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21668-1.JPG
      d63ebfdfe8ef94c1171_1753969489: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21668-2.JPG
      953fd826c3566138871_1753969490: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21668-3.JPG
      84f78767bfcb309608c_1773345621: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21682-1.JPG
      f4d573d8247fe879e20_1773345621: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21682-2.JPG
      ee7f3af57e6ca24c740_1753119499: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21398-1.JPG
      9a52fa74516b29b2374_1753119499: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21398-2.JPG
      ae58386502f3648930d_1746651614: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21394-1.JPG
      8d9f6262985f969bc50_1746651615: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21394-2.JPG
      0cfac532c52b8918a95_1746651615: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21394-3.JPG
      16dfe22a6aa399d7538_1766610014: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21404-1.JPG
      efde03e7adf0d88ee0c_1766610014: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21404-2.JPG
      0822c64d25d29d56644_1766610014: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21404-3.JPG
      14840dbe02500cdb66f_1744474563: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21541-1.JPG
      32a2755e69e12a4ca0e_1744474564: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21541-2.JPG
      110aa98e63e7f6845e5_1744474565: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21541-3.JPG
      65bb3abe23f799f632c_1753451019: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21413-1.JPG
      62c783e380a02aa5b6a_1753451020: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21413-2.JPG
      8656ec9fd1f79a6d8ac_1753451021: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21413-3.JPG
      5eb0e3251c53484bc05_1744474529: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21539-1.JPG
      a88efd16cf0c63f5f80_1744474529: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21539-2.JPG
      2d4de712e8e6189e466_1744474530: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21539-3.JPG
      f537fb57ecd1fdad86c_1744474660: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21431-1.JPG
      c8fdbf6b290743f4b39_1744474661: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21431-2.JPG
      028af9c84b9c5b5a16e_1744474662: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21431-3.JPG
      19f448273f5ca3cf354_1744474666: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21540-1.JPG
      392f4455d59a8275e3d_1744474667: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21540-2.JPG
      5728f2fd9fb57947d52_1744474667: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21540-3.JPG
      1269a0fe2b3a907c8bd_1764826396: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21852-1.JPG
      a74901d8a718ac3cca5_1764826397: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21852-3.JPG
      f3c54ebf66b0c1d9f61_1771897611: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21884-1.JPG
      493279b421ed14d197d_1771897612: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21884-3.JPG
      2bf8cce1d2805fb286e_1744474672: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21564-1.JPG
      e520fe1851bf2512432_1744474672: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21564-2.JPG
      c5d743b1eff14d54d3a_1744474673: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21564-3.JPG
      960edb9de4fa94e796b_1744474654: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21429-1.JPG
      4e7d47f2010e9095955_1744474655: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21429-2.JPG
      2b6c21df0475e77b9fd_1744474656: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21429-3.JPG
      c6b2a6eb50b03c00010_1744474938: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21784-1.JPG
      a05298e92902c3571c2_1744474938: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21784-2.JPG
      c1b757f022ae5d51315_1744474939: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21784-3.JPG
      0dcc58273cc313bb5a8_1778169624: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21711-1.JPG
      ac4672bfa82f77f48d2_1778169624: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21711-2.JPG
      29ee5cb21d63bc48612_1753119600: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21651-1.JPG
      557a1b20c990c09a991_1753119601: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21651-2.JPG
      6f351fe074283fbdf43_1753119602: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21651-3.JPG
      fbcc164ee3e0e887a41_1771897541: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21704-1.JPG
      f0dfa9246599f8c54d0_1771897542: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21704-2.JPG
      ac1f78f28ffe913fb39_1758980811: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21723-1.JPG
      0856f5208d7bbf9d950_1758980811: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21723-2.JPG
      9e2616451687d8aaf32_1758980812: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21723-3.JPG
      de69f3e751d0d7ce012_1753119535: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21599-1.JPG
      40438e489159a8e1270_1753119536: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21599-2.JPG
      3a9dba815522bb08bd4_1753119537: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21599-3.JPG
      d7d89109100560ccb61_1744474799: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21694-1.JPG
      32c56ca8962d32ed93c_1744474800: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21694-2.JPG
      c2312980cac8cae0d88_1744474801: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21694-3.JPG
      92dc562ea26d375803f_1753119876: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21807-1.JPG
      14e4378dc89cb8a3804_1753119876: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21807-2.JPG
      ffafb373f934e9a7d78_1753119877: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21807-3.JPG
      14eb77c19e075215961_1765919124: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21853-1.JPG
      5892598dc6a115e575f_1765919124: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21853-2.JPG
      3e01571bd0cc988b9b8_1765919124: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21853-3.JPG
      4a749b60f162dbf6fe2_1754488144: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21814-1.JPG
      3b5f2985681061314bc_1754488145: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21814-2.JPG
      0fd3c1be9f3718a8669_1754488146: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21814-3.JPG
      3511cbaadf26b8fd4e9_1744474701: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21575-1.JPG
      60bbb41053b49703e4d_1744474702: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21575-2.JPG
      48c533335532f93fd7a_1765573543: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21872-1.JPG
      b15e4b90dd7e951c840_1765573544: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21872-2.JPG
      c4ea48498005f742455_1765919154: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21862-1.JPG
      365c242d496c9161103_1765919155: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21862-3.JPG
      8a74354ebfa72b2c9da_1645639114: https://www.wsnorth.com/wp-content/uploads/2018/11/site-logo.png
      b3e48dd0ee892a36895_1753364634: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21611-1.JPG
      104b633bae4c35f14b7_1753364634: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21611-2.JPG
      2026-05-29 22:13:13 [loknow_spiders] INFO: Uncached images: https://www.wsnorth.com/wp-content/uploads/truck_inventory/21787-1.JPG
      https://www.wsnorth.com/wp-content/uploads/truck_inventory/21783-1.JPG
      2026-05-29 22:13:13 [scrapy.extensions.feedexport] INFO: Stored jsonlines feed (181 items) in: file:///home/scrapyd/items/loknow_spiders_prod/ImageSpider/482514b65bab11f1bdc0c3bbae71ecef.jl
      2026-05-29 22:13:13 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
      {'downloader/request_bytes': 42942,
       'downloader/request_count': 162,
       'downloader/request_method_count/GET': 2,
       'downloader/request_method_count/HEAD': 160,
       'downloader/response_bytes': 47566,
       'downloader/response_count': 162,
       'downloader/response_status_count/200': 133,
       'downloader/response_status_count/404': 29,
       'dupefilter/filtered': 21,
       'elapsed_time_seconds': 118.820748,
       'feedexport/success_count/FileFeedStorage': 1,
       'finish_reason': 'finished',
       'finish_time': datetime.datetime(2026, 5, 29, 22, 13, 13, 773984),
       'item_scraped_count': 181,
       'log_count/INFO': 23,
       'log_count/WARNING': 32,
       'memusage/max': 135892992,
       'memusage/startup': 133091328,
       'response_received_count': 162,
       'scheduler/dequeued': 160,
       'scheduler/dequeued/memory': 160,
       'scheduler/enqueued': 160,
       'scheduler/enqueued/memory': 160,
       'start_time': datetime.datetime(2026, 5, 29, 22, 11, 14, 953236)}
      2026-05-29 22:13:13 [scrapy.core.engine] INFO: Spider closed (finished)
      
    • Log

      /1/log/utf8/loknow_spiders_prod/ImageSpider/482514b65bab11f1bdc0c3bbae71ecef/?job_finished=True&ui=mobile

    • Source

      https://scrapyd.adretriever.com/logs/loknow_spiders_prod/ImageSpider/482514b65bab11f1bdc0c3bbae71ecef.log

  • sourcelog
    last update time2026-05-29 22:13:13
    last update timestamp1780092793
    downloader / request bytes42942
    downloader / request count162
    downloader / request method count / GET2
    downloader / request method count / HEAD160
    downloader / response bytes47566
    downloader / response count162
    downloader / response status count / 200133
    downloader / response status count / 40429
    dupefilter / filtered21
    elapsed time seconds118.820748
    feedexport / success count / FileFeedStorage1
    finish reasonfinished
    finish timedatetime.datetime(2026, 5, 29, 22, 13, 13, 773984)
    item scraped count181
    log count / INFO23
    log count / WARNING32
    memusage / max135892992
    memusage / startup133091328
    response received count162
    scheduler / dequeued160
    scheduler / dequeued / memory160
    scheduler / enqueued160
    scheduler / enqueued / memory160
    start timedatetime.datetime(2026, 5, 29, 22, 11, 14, 953236)