CONNECT 헤더 읽기

Question

CONNECT 요청에 대한 응답으로 보낸 헤더에 유용한 정보를 넣는 프록시 서비스 (proxymesh)를 사용하고 있습니다. 어떤 이유로, Python's httplib doesn't parse them의 경우 :

> CONNECT example.com:443 HTTP/1.1 
> Host: example.com:443 
> 
< HTTP/1.1 200 Connection established 
< X-Useful-Header: value # completely ignored 
< 

는 requests 모듈은 내부적으로 httplib을 사용하기 때문에 그것은뿐만 아니라 그들을 무시합니다. CONNECT 요청에서 헤더를 추출하려면 어떻게해야합니까?

Answer 1

파이썬의 httplib은 실제로 터널을 만들 때 이러한 헤더를 무시합니다. 그것은 해키,하지만 당신은 그들을 차단하고 실제 HTTP 응답의 헤더와 함께 "헤더"라인을 병합 할 수 있습니다 : 당신은 다음 세션에 어댑터를 설치할 수 있습니다

import socket 
import httplib 
import requests 

from requests.packages.urllib3.connection import HTTPSConnection 
from requests.packages.urllib3.connectionpool import HTTPSConnectionPool 
from requests.packages.urllib3.poolmanager import ProxyManager 

from requests.adapters import HTTPAdapter 


class ProxyHeaderHTTPSConnection(HTTPSConnection): 
    def __init__(self, *args, **kwargs): 
     super(ProxyHeaderHTTPSConnection, self).__init__(*args, **kwargs) 
     self._proxy_headers = [] 

    def _tunnel(self): 
     self.send("CONNECT %s:%d HTTP/1.0\r\n" % (self._tunnel_host, self._tunnel_port)) 

     for header, value in self._tunnel_headers.iteritems(): 
      self.send("%s: %s\r\n" % (header, value)) 

     self.send("\r\n") 

     response = self.response_class(self.sock, strict=self.strict, method=self._method) 
     version, code, message = response._read_status() 

     if version == "HTTP/0.9": 
      # HTTP/0.9 doesn't support the CONNECT verb, so if httplib has 
      # concluded HTTP/0.9 is being used something has gone wrong. 
      self.close() 
      raise socket.error("Invalid response from tunnel request") 

     if code != 200: 
      self.close() 
      raise socket.error("Tunnel connection failed: %d %s" % (code, message.strip())) 

     self._proxy_headers = [] 

     while True: 
      line = response.fp.readline(httplib._MAXLINE + 1) 

      if len(line) > httplib._MAXLINE: 
       raise LineTooLong("header line") 

      if not line or line == '\r\n': 
       break 

      # The line is a header, save it 
      if ':' in line: 
       self._proxy_headers.append(line) 

    def getresponse(self, buffering=False): 
     response = super(ProxyHeaderHTTPSConnection, self).getresponse(buffering) 
     response.msg.headers.extend(self._proxy_headers) 

     return response 


class ProxyHeaderHTTPSConnectionPool(HTTPSConnectionPool): 
    ConnectionCls = ProxyHeaderHTTPSConnection 


class ProxyHeaderProxyManager(ProxyManager): 
    def _new_pool(self, scheme, host, port): 
     assert scheme == 'https' 

     return ProxyHeaderHTTPSConnectionPool(host, port, **self.connection_pool_kw) 


class ProxyHeaderHTTPAdapter(HTTPAdapter): 
    def proxy_manager_for(self, proxy, **proxy_kwargs): 
     if proxy in self.proxy_manager: 
      manager = self.proxy_manager[proxy] 
     else: 
      proxy_headers = self.proxy_headers(proxy) 
      manager = self.proxy_manager[proxy] = ProxyHeaderProxyManager(
       proxy_url=proxy, 
       proxy_headers=proxy_headers, 
       num_pools=self._pool_connections, 
       maxsize=self._pool_maxsize, 
       block=self._pool_block, 
       **proxy_kwargs) 

     return manager 

:

session = requests.Session() 
session.mount('https://', ProxyHeaderHTTPAdapter()) 

response = session.get('https://example.com', proxies={...}) 

프록시의 헤더 것이다 응답 헤더와 병합되므로 프록시가 응답 헤더를 직접 수정 한 것처럼 동작해야합니다.