R 스크랩 아톰 피드를 데이터 프레임에 넣기

Question

나는 아톰 피드와 각 기사 링크를 가져 오는 데 문제가있는 R 스크레이퍼를 만들고 있습니다. 여기에 내 코드가있다 :

url <- "http://www.stwnewspress.com/search/?mode=article&q=&nsa=eedition&t=article&l=1000&s=&sd=desc&f=atom&d=&d1=&d2=" 
pageSource <- getURL(url, encoding = "UTF-8") 
parsed <- htmlParse(pageSource) 
titles <- xpathSApply(parsed, '//entry/title', xmlValue) 
authors <- xpathSApply(parsed, '//entry/author', xmlValue) 
links <- xpathSApply(parsed, '//entry/link/@href') 
dataFrame <- data.frame(pubDates, titles, authors) 

내 문제는 18 권의 저자, ​​18 명의 저자, ​​20 개의 링크가있다. 피드 페이지에서 처음 두 개의 링크를 가져 오는 것 같지만, 그만 두는 방법을 모르겠습니다.

도움 주셔서 감사합니다.

Answer 1

개별 노드가 아닌 "// entry"에서 작업 할 수 있습니다.

out <- xpathApply(parsed, "//entry", function(x){ 
    children <- xmlChildren(x) 
    title <- xmlValue(children$title) 
    author <- xmlValue(children$author) 
    links <- children[names(children)%in%"link"] 
    links <- sapply(links, function(y){xmlGetAttr(y, "href")}) 
    data.frame(title, author, links, stringsAsFactors = FALSE) 
}) 

> out[[1]] 
              title   author 
1 Soap opera star in serious injury crash in Ohio CNHI News Service 
2 Soap opera star in serious injury crash in Ohio CNHI News Service 
                                               links 
1                      http://www.stwnewspress.com/cnhi_network/article_71fb99db-0d47-5ead-9276-cae9c947babc.html 
2 http://bloximages.chicago2.vip.townnews.com/stwnewspress.com/content/tncms/assets/v3/editorial/d/97/d97a9815-29c8-5b90-be11-41a3a8b12e9f/54354a7b66bd9.image.jpg?resize=300%2C450 
> out[[2]] 
            title         author 
link Q5: Voter registration deadline nears By Michelle Charles/Stillwater News Press 
                          links 
link http://www.stwnewspress.com/news/local_news/article_ba35bd60-4ea4-11e4-8da8-93d495865336.html 

그런 다음 함께 rbind 개별 항목 수 :

res <- do.call(rbind.data.frame, out) 
> str(res) 
'data.frame': 147 obs. of 3 variables: 
$ title : chr "Soap opera star in serious injury crash in Ohio" "Soap opera star in serious injury crash in Ohio" "Q5: Voter registration deadline nears" "Oklahoma State assault under investigation" ... 
$ author: chr "CNHI News Service" "CNHI News Service" "By Michelle Charles/Stillwater News Press" "By Megan Sando/Stillwater News Press" ... 
$ links : chr "http://www.stwnewspress.com/cnhi_network/article_71fb99db-0d47-5ead-9276-cae9c947babc.html" "http://bloximages.chicago2.vip.townnews.com/stwnewspress.com/content/tncms/assets/v3/editorial/d/97/d97a9815-29c8-5b90-be11-41a"| __truncated__ "http://www.stwnewspress.com/news/local_news/article_ba35bd60-4ea4-11e4-8da8-93d495865336.html" "http://www.stwnewspress.com/news/local_news/article_7023a110-4ea4-11e4-82dd-f735d5c5ed44.html" ... 

이 그것을 x를 호출하는 기능이 작동하는 첫 번째 항목을보고하는 방법을 이해하려면 :

일부 항목 노드는 예를 들어, 여러 링크가

url <- "http://www.stwnewspress.com/search/?mode=article&q=&nsa=eedition&t=article&l=1000&s=&sd=desc&f=atom&d=&d1=&d2=" 
pageSource <- getURL(url, encoding = "UTF-8") 
parsed <- htmlParse(pageSource) 
x <- parsed["//entry"][[1]] 
children <- xmlChildren(x) 

> names(children) 
[1] "title" "author" "link"  "id"  "content" "category" 
[7] "updated" 

> children$title 
<title>BYRON YORK: Jindal a GOP darkhorse in 2016 race</title> 

> xmlValue(children$title) 
[1] "BYRON YORK: Jindal a GOP darkhorse in 2016 race"