如何循环一个 div 并仅使用美丽汤和蟒蛇获取段落标记中的文本？

Question

我正在使用美丽汤和python来抓取网页，并仅从网站中提取段落标签中的文本。这是我想抓取的页面我想要所有段落标签中的所有文本。

Answer 1

始终使用硒作为节省资源的最后手段。

from selenium import webdriver

url = 'https://www.who.int/csr/disease/coronavirus_infections/faq_dec12/en/'

driver = webdriver.Chrome()

try:

  driver.get(url)

  div_text = driver.find_element_by_id('primary').text

  with open('website_content.txt','w') as f:

    f.write(div_text)

except Exception as e:

  print(e)

finally:

  if driver is not None:

    driver.close()

你可以通过要求和美丽的汤来实现同样的事情，如下所示：

import requests as rq

from bs4 import BeautifulSoup

response  = rq.get(url)

if response.status_code == 200:

  soup = BeautifulSoup(response.text,'html.parser')

  div_text = soup.find('div',{'id':'primary'}).text

  with open('website_content.txt','w') as f:

    f.write(div_text)