无法使用 Selenium 检索类值

Question

构建一个基本的网络爬虫，我无法检索包含产品价格的类值。谷歌搜索中列出的所有价格的类名称都是qptdjc。这是价格的 HTML 标签 <div class="qptdjc">$179.99</div>import seleniumfrom selenium import webdriver as wbimport pandas as pdimport timebrowser = wb.Chrome(executable_path='C:/Users/ethan/Downloads/chromedriver_win32(1)/chromedriver')browser.get('https://www.google.com/search?source=hp&ei=nnZ6X8SDO4WE9PwPj5KC4AQ&q=144hz+monitor&oq=144hz+monitor&gs_lcp=CgZwc3ktYWIQAzIFCAAQsQMyBQgAELEDMgIIADICCAAyAggAMgIIADICCAAyAggAMgIIADICCAA6DggAEOoCELQCEJoBEOUCOgsILhDHARCvARCTAjoFCC4QsQM6CAguEMcBEK8BOgsILhCxAxDHARCjAjoICAAQsQMQgwE6CAguELEDEIMBOg4ILhCxAxCDARDHARCvAVDyDligHmDsH2gBcAB4AIABiwGIAZMHkgEEMTIuMZgBAKABAaoBB2d3cy13aXqwAQY&sclient=psy-ab&ved=0ahUKEwjEyoykppzsAhUFAp0JHQ-JAEwQ4dUDCAk&uact=5')productInfoList = browser.find_elements_by_class_name('qptdjc')prices = browser.find_elements_by_xpath('//td[@class="qptdjc"]')prices_list = []for p in range(len(prices)):    prices_list.append(prices[p].text)print(len(productInfoList))print(*prices_list, sep = ", ")print(*prices, sep = ", ")

Answer 1

所以我等待所有的元素出现。抓取所有内容，循环并获取其内部 HTML。

productInfoList=WebDriverWait(driver, 20).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "div.r4awE > span")))

prices=WebDriverWait(driver, 20).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "div.qptdjc")))

prices_list = []

for price in prices:

    prices_list.append(price.get_attribute('innerHTML').split('<')[0].strip())

pprint(prices_list)

print(len(productInfoList))

输出

['$229.99',

 '$187.52',

 '$249.99']

3

进口

from selenium.webdriver.common.by import By

from selenium.webdriver.support.ui import WebDriverWait 

from selenium.webdriver.support import expected_conditions as EC

from pprint import pprint