Python 치료 - 로그인 인증 문제

Question

방금 ​​치료를 시작했습니다. 나는 치료에서 로그인에 관한 몇 가지 문제에 직면하고있다. 나는 웹 사이트 www.instacart.com에서 긁힌 자국 물건을 시험하고있다. 하지만 로깅 문제에 직면하고있다.

다음은 다음 코드

import scrapy 

from scrapy.loader import ItemLoader 
from project.items import ProjectItem 
from scrapy.http import Request 
from scrapy import optional_features 
optional_features.remove('boto') 

class FirstSpider(scrapy.Spider): 
    name = "first" 
    allowed_domains = ["https://instacart.com"] 
    start_urls = [ 
     "https://www.instacart.com" 
    ] 

    def init_request(self): 
     return Request(url=self.login_page, callback=self.login) 

    def login(self, response): 
     return scrapy.FormRequest('https://www.instacart.com/#login', 
            formdata={'email': 'xxx@xxx.com', 'password': 'xxxxxx', 
               }, 
            callback=self.parse) 

    def check_login_response(self, response): 
     return scrapy.Request('https://www.instacart.com/', self.parse) 

    def parse(self, response): 
     if "Goutam" in response.body: 
      print "Successfully logged in. Let's start crawling!" 
     else: 
      print "Login unsuccessful" 

에게 오류 메시지를

C:\Users\gouta\PycharmProjects\CSG_Scraping\csg_wholefoods>scrapy crawl first 
2016-06-15 10:44:50 [scrapy] INFO: Scrapy 1.0.3 started (bot: project) 
2016-06-15 10:44:50 [scrapy] INFO: Optional features available: ssl, http11 
2016-06-15 10:44:50 [scrapy] INFO: Overridden settings: {'NEWSPIDER_MODULE': 'project.spiders', 'SPIDER_MODULES': ['project.spiders'], 'ROBOTSTXT_OBEY': True, 'BOT_NAME': 'project'} 
2016-06-15 10:44:50 [scrapy] INFO: Enabled extensions: CloseSpider, TelnetConsole, LogStats, CoreStats, SpiderState 
2016-06-15 10:44:50 [scrapy] INFO: Enabled downloader middlewares: RobotsTxtMiddleware, HttpAuthMiddleware, DownloadTimeoutMiddleware, UserAgentMiddleware, RetryMiddleware, DefaultHeadersMiddleware, MetaRefreshMiddleware, HttpCompressionMiddleware, RedirectMiddleware, CookiesMiddleware, ChunkedTransferMiddleware, DownloaderStats 
2016-06-15 10:44:51 [scrapy] INFO: Enabled spider middlewares: HttpErrorMiddleware, OffsiteMiddleware, RefererMiddleware, UrlLengthMiddleware, DepthMiddleware 
2016-06-15 10:44:51 [scrapy] INFO: Enabled item pipelines: 
2016-06-15 10:44:51 [scrapy] INFO: Spider opened 
2016-06-15 10:44:51 [scrapy] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min) 
2016-06-15 10:44:51 [scrapy] DEBUG: Telnet console listening on 127.0.0.1:6023 
2016-06-15 10:44:51 [scrapy] DEBUG: Crawled (200) <GET https://www.instacart.com/robots.txt> (referer: None) 
2016-06-15 10:44:51 [scrapy] DEBUG: Crawled (200) <GET https://www.instacart.com> (referer: None) 
Login unsuccessful 
2016-06-15 10:44:51 [scrapy] INFO: Closing spider (finished) 
2016-06-15 10:44:51 [scrapy] INFO: Dumping Scrapy stats: 
{'downloader/request_bytes': 440, 
'downloader/request_count': 2, 
'downloader/request_method_count/GET': 2, 
'downloader/response_bytes': 17182, 
'downloader/response_count': 2, 
'downloader/response_status_count/200': 2, 
'finish_reason': 'finished', 
'finish_time': datetime.datetime(2016, 6, 15, 15, 44, 51, 767000), 
'log_count/DEBUG': 3, 
'log_count/INFO': 7, 
'response_received_count': 2, 
'scheduler/dequeued': 1, 
'scheduler/dequeued/memory': 1, 
'scheduler/enqueued': 1, 
'scheduler/enqueued/memory': 1, 
'start_time': datetime.datetime(2016, 6, 15, 15, 44, 51, 31000)} 
2016-06-15 10:44:51 [scrapy] INFO: Spider closed (finished) 

어떤 조언을 높게 평가되어 내가 무엇을 잘못하고 있는가에. 당신이 게시 데이터를 보면

Answer 1

, 당신이 게시물에 대한 필드를 볼 수 있습니다

from scrapy.crawler import CrawlerProcess 
import scrapy 

from scrapy.http import Request 

class FirstSpider(scrapy.Spider): 
    name = "first" 
    allowed_domains = ["instacart.com"] 
    start_urls = [ 
     "https://www.instacart.com" 
    ] 

    def start_requests(self): 
     return [Request(url="https://www.instacart.com", callback=self.login)] 

    def login(self, response): 
     return scrapy.FormRequest('https://www.instacart.com/accounts/login', 
            headers={"X-Requested-With": "XMLHttpRequest"}, 
            formdata={'user[email]': 'xxxxxxx@gmail.com', 'user[password]': 'xxxxx', 
              "authenticity_token": response.xpath(
               "//meta[@name='csrf-token']/@content").extract_first()}, 
            callback=self.parse,dont_filter=True) 


    def parse(self, response): 
     print(response.body) 
     if "Goutam" in response.body: 
      print "Successfully logged in. Let's start crawling!" 
     else: 
      print "Login unsuccessful" 

:

headers = {"X-Requested-With":"XMLHttpRequest"} 
formdata = {'user[email]': 'xxx@xxx.com', 'user[password]': 'xxxx', 
      "authenticity_token":response.xpath("//meta[@csrf-token]/@content").extract_first()} 

가 실제로 몇 문제, 전체 작업 거미입니다

유일한 문제는 반응을 사용하여 많은 콘텐츠가로드되었습니다.