파이썬 3.5 urllib.request 403 금지 된 파일

Question

import urllib.request 
import urllib 
from bs4 import BeautifulSoup 


url = "https://www.brightscope.com/ratings" 
page = urllib.request.urlopen(url) 
soup = BeautifulSoup(page, "html.parser") 

print(soup.title) 

위의 사이트로 이동하려고했는데 코드에서 403 Forbidden Error가 튀어 나오고 있습니다.

어떤 아이디어?

C:\Users\jerem\AppData\Local\Programs\Python\Python35-32\python.exe "C:/Users/jerem/PycharmProjects/webscraper/url scraper.py" Traceback (most recent call last): File "C:/Users/jerem/PycharmProjects/webscraper/url scraper.py", line 7, in page = urllib.request.urlopen(url) File "C:\Users\jerem\AppData\Local\Programs\Python\Python35-32\lib\urllib\request.py", line 163, in urlopen return opener.open(url, data, timeout) File "C:\Users\jerem\AppData\Local\Programs\Python\Python35-32\lib\urllib\request.py", line 472, in open response = meth(req, response) File "C:\Users\jerem\AppData\Local\Programs\Python\Python35-32\lib\urllib\request.py", line 582, in http_response 'http', request, response, code, msg, hdrs) File "C:\Users\jerem\AppData\Local\Programs\Python\Python35-32\lib\urllib\request.py", line 510, in error return self._call_chain(*args) File "C:\Users\jerem\AppData\Local\Programs\Python\Python35-32\lib\urllib\request.py", line 444, in _call_chain result = func(*args) File "C:\Users\jerem\AppData\Local\Programs\Python\Python35-32\lib\urllib\request.py", line 590, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden

Answer 1

import requests 
from bs4 import BeautifulSoup 


url = "https://www.brightscope.com/ratings" 
headers = {'User-Agent':'Mozilla/5.0'} 
page = requests.get(url) 
soup = BeautifulSoup(page.text, "html.parser") 

print(soup.title) 

에서 첫째

<title>BrightScope Ratings</title> 

, reuqests보다는 urllib 사용합니다.

이상 headers ~ requests을 추가하십시오. 기본값이 User-Agent 인 사이트가 마음에 들지 않으므로 사이트에서 금지합니다.