-
Head
2026-05-29 22:47:50 [scrapy.utils.log] INFO: Scrapy 2.10.0 started (bot: retriever)
2026-05-29 22:47:50 [scrapy.utils.log] INFO: Versions: lxml 5.1.0.0, libxml2 2.12.3, cssselect 1.2.0, parsel 1.9.0, w3lib 2.0.0, Twisted 22.4.0, Python 3.10.12 (main, Mar 3 2026, 11:56:32) [GCC 11.4.0], pyOpenSSL 24.1.0 (OpenSSL 3.2.1 30 Jan 2024), cryptography 42.0.5, Platform Linux-6.8.0-1030-aws-x86_64-with-glibc2.35
2026-05-29 22:47:50 [scrapy.addons] INFO: Enabled addons:
[]
2026-05-29 22:47:50 [scrapy.crawler] INFO: Overridden settings:
{'AUTOTHROTTLE_ENABLED': True,
'BOT_NAME': 'retriever',
'HTTPCACHE_ENABLED': True,
'HTTPCACHE_EXPIRATION_SECS': 86400,
'IMAGES_STORE_S3_ACL': 'public-read',
'LOG_FILE': '/home/scrapyd/logs/loknow_spiders_prod/MainstSpider/69cb400e5bb011f1bdc0c3bbae71ecef.log',
'LOG_FORMATTER': 'loknow_spiders.logging.PoliteLogFormatter',
'LOG_LEVEL': 'INFO',
'NEWSPIDER_MODULE': 'loknow_spiders.spiders',
'ROBOTSTXT_OBEY': True,
'SPIDER_MODULES': ['loknow_spiders.spiders'],
'TELNETCONSOLE_ENABLED': False,
'USER_AGENT': 'AdRetriever (https://adretriever.com)'}
2026-05-29 22:47:50 [py.warnings] WARNING: /home/scrapyd/venv/lib/python3.10/site-packages/scrapy/utils/request.py:248: ScrapyDeprecationWarning: '2.6' is a deprecated value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting.
It is also the default value. In other words, it is normal to get this warning if you have not defined a value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting. This is so for backward compatibility reasons, but it will change in a future version of Scrapy.
See the documentation of the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting for information on how to handle this deprecation.
return cls(crawler)
2026-05-29 22:47:50 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
'scrapy.extensions.memusage.MemoryUsage',
'scrapy.extensions.feedexport.FeedExporter',
'scrapy.extensions.logstats.LogStats',
'scrapy.extensions.throttle.AutoThrottle']
2026-05-29 22:47:50 [MainstSpider] INFO: Current Spider Environment set to: prod
2026-05-29 22:47:50 [MainstSpider] INFO: Retrieving AWS secret: spider_prod_secrets
2026-05-29 22:47:50 [botocore.credentials] INFO: Found credentials in shared credentials file: ~/.aws/credentials
2026-05-29 22:47:50 [MainstSpider] INFO: In base spider ml_kwargs is None and scrape_type is inventory
2026-05-29 22:47:50 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.robotstxt.RobotsTxtMiddleware',
'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
'loknow_spiders.middlewares.ThrottlingRetryMiddleware',
'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
'scrapy.downloadermiddlewares.stats.DownloaderStats',
'scrapy.downloadermiddlewares.httpcache.HttpCacheMiddleware']
2026-05-29 22:47:50 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
'loknow_spiders.middlewares.RetrieverSpiderMiddleware',
'scrapy.spidermiddlewares.referer.RefererMiddleware',
'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
'scrapy.spidermiddlewares.depth.DepthMiddleware']
2026-05-29 22:47:52 [scrapy.middleware] INFO: Enabled item pipelines:
['loknow_spiders.pipelines.MainstPipeline',
'loknow_spiders.pipelines.RetrieverPipeline']
2026-05-29 22:47:52 [scrapy.core.engine] INFO: Spider opened
2026-05-29 22:47:52 [MainstSpider] INFO: RetrieverAPI is making requests to https://api.adretriever.com with headers {'Authorization': 'Token 96dfb640234f1d676d6fa726f8eae6e7aab44cda'}
2026-05-29 22:47:52 [loknow_spiders] INFO: Scrape 94e27e89 has been opened.
2026-05-29 22:47:52 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-05-29 22:48:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/allyn-arms-apartments
2026-05-29 22:48:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/aquarius-place
2026-05-29 22:48:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/amantea-tower
2026-05-29 22:48:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/amera-manor
2026-05-29 22:48:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/alexander-apartments
2026-05-29 22:48:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/donna-place
2026-05-29 22:48:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/dickens-apartments
2026-05-29 22:48:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/diane-apartments-2
2026-05-29 22:48:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/diane-apartments-1
2026-05-29 22:48:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/dhillon-tower
2026-05-29 22:48:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/deluca-manor
2026-05-29 22:48:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/delburn-house
2026-05-29 22:48:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/dartmouth-apartments
2026-05-29 22:48:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/dahlstrom-manor
2026-05-29 22:48:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/cypress-manor
2026-05-29 22:48:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/crescent-heights-view
2026-05-29 22:48:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/crescent-heights-manor
2026-05-29 22:48:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/crescent-heights-apartments
2026-05-29 22:48:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/courtenay-riverway-apartments
2026-05-29 22:48:10 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/coronation-place
2026-05-29 22:48:10 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/cornerstone-apartments
2026-05-29 22:48:10 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/confederation-place
2026-05-29 22:48:15 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/grapes-manor
2026-05-29 22:48:15 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/granville-place
2026-05-29 22:48:15 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/grand-apartments
2026-05-29 22:48:15 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/grace-manor
2026-05-29 22:48:15 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/glenbow-manor
2026-05-29 22:48:15 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/gilford-apartments
2026-05-29 22:48:15 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/gemini-apartments
2026-05-29 22:48:15 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/gateway-place
2026-05-29 22:48:15 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/gateway-manor-apartments
2026-05-29 22:48:15 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/gary-manor
2026-05-29 22:48:15 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/froom-apartments
2026-05-29 22:48:15 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/forestview-heights-apartments
2026-05-29 22:48:15 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/floyd-manor
2026-05-29 22:48:15 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/flores-manor
2026-05-29 22:48:15 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/falconcrest-village
2026-05-29 22:48:15 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/fairhaven-apartments
2026-05-29 22:48:15 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/evergreen-manor-apartments
2026-05-29 22:48:15 [MainstSpider] INFO: Processed https://www.mainst.biz/apartments/edmonton/evansdale-court-apartments
-
Tail
2026-05-29 22:49:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/mainstreet-on-fort-saskatchewan
2026-05-29 22:49:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/mainstreet-on-elbow
2026-05-29 22:49:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/mainstreet-on-33rd
2026-05-29 22:49:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/mainstreet-on-124th
2026-05-29 22:49:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/mainstreet-mission-hills
2026-05-29 22:49:09 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/mainstreet-mews
2026-05-29 22:49:10 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/mainstreet-marda-loop-apartments
2026-05-29 22:49:10 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/mainstreet-maple-ridge-apartments
2026-05-29 22:49:10 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/mainstreet-hilltop
2026-05-29 22:49:10 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/mainstreet-hart-apartments
2026-05-29 22:49:13 [MainstSpider] INFO: Processed https://www.mainst.biz/apartments/surrey/mainstreet-guildford-townhomes
2026-05-29 22:49:13 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/mack-apartments
2026-05-29 22:49:13 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/lynden-court
2026-05-29 22:49:13 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/lorraine-manor
2026-05-29 22:49:13 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/abbey-tower
2026-05-29 22:49:14 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/loftus-manor
2026-05-29 22:49:14 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/linde-apartments
2026-05-29 22:49:14 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/lincoln-terrace
2026-05-29 22:49:14 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/libra-apartments
2026-05-29 22:49:14 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/leo-manor-apartments
2026-05-29 22:49:14 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/legislative-estates-apartments
2026-05-29 22:49:14 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/lawson-place
2026-05-29 22:49:14 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/laurentian-apartments
2026-05-29 22:49:14 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/lauderdale-manor
2026-05-29 22:49:14 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/lancelot-apartments
2026-05-29 22:49:14 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/kleisinger-45
2026-05-29 22:49:14 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/kleisinger-3
2026-05-29 22:49:14 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/kleisinger-27
2026-05-29 22:49:15 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/kirkness-park-apartments
2026-05-29 22:49:15 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/kingsway-lofts-and-apartments
2026-05-29 22:49:17 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/kings-court
2026-05-29 22:49:17 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/king-edward
2026-05-29 22:49:17 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/key-west-apartments
2026-05-29 22:49:17 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/kewanee-manor
2026-05-29 22:49:17 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/kerpel-apartments
2026-05-29 22:49:17 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/kensington-apartments
2026-05-29 22:49:17 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/karen-hall
2026-05-29 22:49:18 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/kane-apartments
2026-05-29 22:49:18 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/kamloops-tranquille-apartments
2026-05-29 22:49:18 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/julliard-south
2026-05-29 22:49:18 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/julliard-north
2026-05-29 22:49:18 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/julia-place
2026-05-29 22:49:18 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/joy-manor-apartments
2026-05-29 22:49:18 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/john-manor
2026-05-29 22:49:18 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/jerry-place
2026-05-29 22:49:18 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/jazz-manor
2026-05-29 22:49:18 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/jays-place
2026-05-29 22:49:18 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/jasper-ave-lookout-2
2026-05-29 22:49:18 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/inglewood-124-apartments
2026-05-29 22:49:18 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/ingle-apartments
2026-05-29 22:49:18 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/imperial-parkside
2026-05-29 22:49:18 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/huntsville-apartments
2026-05-29 22:49:19 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/greystone-court
2026-05-29 22:49:19 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/grey-apartments
2026-05-29 22:49:21 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/youngstown-townhomes-and-apartments
2026-05-29 22:49:21 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/grey-909
2026-05-29 22:49:21 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/greenwood-gardens
2026-05-29 22:49:21 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/elbow-mission-apartments
2026-05-29 22:49:21 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/elbow-court
2026-05-29 22:49:21 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/eileen-manor
2026-05-29 22:49:21 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/edmonton-trail-apartments
2026-05-29 22:49:22 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/edgar-estates
2026-05-29 22:49:22 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/eastwood-manor
2026-05-29 22:49:22 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/duchess-manor
2026-05-29 22:49:22 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/downtown-edmonton-apartments
2026-05-29 22:49:22 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/doverglen-estates
2026-05-29 22:49:22 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/dorset-arms
2026-05-29 22:49:22 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/confederation-park-apartments
2026-05-29 22:49:22 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/compton-court
2026-05-29 22:49:22 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/college-manor-mainstreet-apartments
2026-05-29 22:49:22 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/college-grove
2026-05-29 22:49:22 [root] INFO: Making post request to https://api.adretriever.com/api/import/94e27e89
2026-05-29 22:49:23 [loknow_spiders] INFO: Created: 98 items out of 100
2026-05-29 22:49:23 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/college-1127
2026-05-29 22:49:23 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/cochrane-house-apartments
2026-05-29 22:49:23 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/cochrane-104
2026-05-29 22:49:23 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/clareview-court
2026-05-29 22:49:23 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/chinook-woods
2026-05-29 22:49:24 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/chinook-place
2026-05-29 22:49:24 [MainstSpider] INFO: Processed https://www.mainst.biz/apartments/calgary/chinook-meadows-apartments
2026-05-29 22:49:25 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/chilliwack-landing-apartments
2026-05-29 22:49:25 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/chilliwack-district-mainstreet-apartments
2026-05-29 22:49:25 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/chilliwack-core-mainstreet-apartments
2026-05-29 22:49:25 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/chilliwack-central-apartments
2026-05-29 22:49:25 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/chelsey-apartments
2026-05-29 22:49:25 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/chelsea-park-gardens
2026-05-29 22:49:25 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/cheam-view-apartments-2
2026-05-29 22:49:26 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/cheam-view-apartments-1
2026-05-29 22:49:26 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/charlotte-manor
2026-05-29 22:49:26 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/charles-rutherford-apartments
2026-05-29 22:49:26 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/cesar-place
2026-05-29 22:49:26 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/centre-crescent-apartments-2
2026-05-29 22:49:26 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/centre-crescent-apartments-1
2026-05-29 22:49:26 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/centre-110
2026-05-29 22:49:26 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/centre-102
2026-05-29 22:49:26 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/central-lrt-apartments
2026-05-29 22:49:26 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/central-beltline-apartments
2026-05-29 22:49:27 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/cedarwood-arms
2026-05-29 22:49:27 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/cedartree-village
2026-05-29 22:49:27 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/cedarbrae-manor
2026-05-29 22:49:27 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/cedar-grove
2026-05-29 22:49:27 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/cathy-manor
2026-05-29 22:49:27 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/cascade-manor
2026-05-29 22:49:27 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/carlson-apartments
2026-05-29 22:49:28 [scrapy.spidermiddlewares.httperror] INFO: Ignoring response <404 https://www.mainst.biz/bridgeland-hillside-apartments>: HTTP status code is not handled or not allowed
2026-05-29 22:49:29 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/carina-court
2026-05-29 22:49:29 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/cardona-place
2026-05-29 22:49:29 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/capricorn-apartments
2026-05-29 22:49:29 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/capital-manor
2026-05-29 22:49:29 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/camino-manor
2026-05-29 22:49:29 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/cameron-manor-1
2026-05-29 22:49:29 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/camelia-court
2026-05-29 22:49:29 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/cambridge-manor
2026-05-29 22:49:29 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/byron-apartments
2026-05-29 22:49:29 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/broadview-legislature-apartments
2026-05-29 22:49:29 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/bridgeport-manor
2026-05-29 22:49:29 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/bridgeland-apartments
2026-05-29 22:49:29 [MainstSpider] INFO: Processed https://www.mainst.biz/apartments/calgary/bowness-village-apartments
2026-05-29 22:49:29 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/bowness-4347-apartments
2026-05-29 22:49:29 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/bowness-4315-apartments
2026-05-29 22:49:29 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/bowness-4311
2026-05-29 22:49:29 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/bow-river-townhomes
2026-05-29 22:49:29 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/booth-apartments
2026-05-29 22:49:48 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/2501-apartments
2026-05-29 22:49:48 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/bankview-place
2026-05-29 22:49:48 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/bankview-hill-apartments
2026-05-29 22:49:48 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/bankview-1737-apartments
2026-05-29 22:49:48 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/bankview-1711
2026-05-29 22:49:48 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/avonhurst-3525
2026-05-29 22:49:48 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/avila-apartments
2026-05-29 22:49:48 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/avenue-tower
2026-05-29 22:49:48 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/avenue-houses
2026-05-29 22:49:48 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/aurora-place-apartments
2026-05-29 22:49:48 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/athlone-park-townhomes
2026-05-29 22:49:48 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/astoria-manor
2026-05-29 22:49:48 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/assiniboine
2026-05-29 22:49:48 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/aspen-apartments
2026-05-29 22:49:48 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/ashley-court
2026-05-29 22:49:48 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/ariel-apartments
2026-05-29 22:49:48 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/argyll-apartments
2026-05-29 22:49:48 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/arcade-heritage-apartments
2026-05-29 22:49:48 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/anna-court-apartments
2026-05-29 22:49:48 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/amherst-apartments
2026-05-29 22:49:50 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/airdrie-meadows
2026-05-29 22:49:52 [scrapy.extensions.logstats] INFO: Crawled 474 pages (at 197 pages/min), scraped 470 items (at 208 items/min)
2026-05-29 22:49:59 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/beltline-garden-apartments
2026-05-29 22:50:00 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/beltline-apartments-off-17th
2026-05-29 22:50:10 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/beaumaris-lake-apartments
2026-05-29 22:50:14 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/alex-44
2026-05-29 22:50:17 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/alex-38
2026-05-29 22:50:18 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/alex-20
2026-05-29 22:50:19 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/yarmouth-apartments
2026-05-29 22:50:21 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/bonaventure-apartments
2026-05-29 22:50:21 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/berkley-manor
2026-05-29 22:50:21 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/beltline-on-14th
2026-05-29 22:50:21 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/beltline-apartments
2026-05-29 22:50:22 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/belmae-manor
2026-05-29 22:50:22 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/beauview-apartments
2026-05-29 22:50:22 [MainstSpider] INFO: Processed https://www.mainst.biz/find-an-apartment/bannerman-apartments
2026-05-29 22:50:22 [scrapy.core.engine] INFO: Closing spider (finished)
2026-05-29 22:50:22 [root] INFO: Making post request to https://api.adretriever.com/api/import/94e27e89
2026-05-29 22:50:22 [loknow_spiders] INFO: Created: 76 items out of 84
2026-05-29 22:50:22 [scrapy.extensions.feedexport] INFO: Stored jsonlines feed (484 items) in: file:///home/scrapyd/items/loknow_spiders_prod/MainstSpider/69cb400e5bb011f1bdc0c3bbae71ecef.jl
2026-05-29 22:50:22 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 334188,
'downloader/request_count': 1246,
'downloader/request_method_count/GET': 1246,
'downloader/response_bytes': 24502588,
'downloader/response_count': 1246,
'downloader/response_status_count/200': 487,
'downloader/response_status_count/301': 758,
'downloader/response_status_count/404': 1,
'elapsed_time_seconds': 150.141889,
'feedexport/success_count/FileFeedStorage': 1,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2026, 5, 29, 22, 50, 22, 844787),
'httpcache/firsthand': 51,
'httpcache/hit': 1195,
'httpcache/miss': 51,
'httpcache/store': 51,
'httpcompression/response_bytes': 103937490,
'httpcompression/response_count': 488,
'httperror/response_ignored_count': 1,
'httperror/response_ignored_status_count/404': 1,
'item_scraped_count': 484,
'log_count/INFO': 512,
'log_count/WARNING': 1,
'memusage/max': 179142656,
'memusage/startup': 125198336,
'request_depth_max': 1,
'response_received_count': 488,
'robotstxt/request_count': 2,
'robotstxt/response_count': 2,
'robotstxt/response_status_count/200': 2,
'scheduler/dequeued': 1244,
'scheduler/dequeued/memory': 1244,
'scheduler/enqueued': 1244,
'scheduler/enqueued/memory': 1244,
'start_time': datetime.datetime(2026, 5, 29, 22, 47, 52, 702898)}
2026-05-29 22:50:22 [scrapy.core.engine] INFO: Spider closed (finished)
-
Log
/1/log/utf8/loknow_spiders_prod/MainstSpider/69cb400e5bb011f1bdc0c3bbae71ecef/?job_finished=True&ui=mobile
-
Source
https://scrapyd.adretriever.com/logs/loknow_spiders_prod/MainstSpider/69cb400e5bb011f1bdc0c3bbae71ecef.log