如何在Python中获取嵌套的href？

Question

目标（我需要重复搜索数百次）：1.在“ https://www.ncbi.nlm.nih.gov/ipg/ ”中搜索（例如“WP_000177210.1”）（即https://www.ncbi.nlm.nih.gov/ipg/?term=WP_000177210.1）2.选择表格第二列“CDS Region in Nucleotide”中的第一条记录（即“NC_011415.1 1997353-1998831 (-)”，https: //www.ncbi.nlm.nih.gov/nuccore/NC_011415.1 ?from=1997353&to=1998831&strand=2 ）3.选择该序列名称下的“FASTA”4.获取fasta序列（即“NC_011415.1：c1998831-1997353大肠杆菌SE11，完整序列ATGACTTTATGGATTAACGGTGACTGGATAACGGGCCAGGGCGCATCGCGTGTGAAGCGTAATCCGGTAT CGGGCGAG......”）。代码1.在“ https://www.ncbi.nlm.nih.gov/ipg/ ”中搜索（例如“WP_000177210.1”）import requestsfrom bs4 import BeautifulSoupurl = "https://www.ncbi.nlm.nih.gov/ipg/"r = requests.get(url, params = "WP_000177210.1")if r.status_code == requests.codes.ok:    soup = BeautifulSoup(r.text,"lxml")2.选择表第二列“核苷酸中的 CDS 区域”中的第一条记录（在本例中为“NC_011415.1 1997353-1998831 (-)”）（即https://www.ncbi.nlm.nih.gov /nuccore/NC_011415.1?from=1997353&to=1998831&strand=2 )# try 1 (wrong)## I tried this first, but it seemed like it only accessed to the first level of the href?!for a in soup.find_all('a', href=True):    if (a['href'][:8]) =="/nuccore":        print("Found the URL:", a['href'])# try 2 (not sure how to access nested href)## According to the label I saw in the Develop Tools, I think I need to get the href in the following nested structure. However, it didn't work.soup.select("html div #maincontent div div div #ph-ipg div table tbody tr td a")我现在就卡在这一步了......聚苯乙烯这是我第一次处理html格式。我也是第一次在这里提问。我可能不太清楚地表达这个问题。如果有任何问题，请告诉我。

Answer 1

不使用 NCBI 的 REST API，

import time

from bs4 import BeautifulSoup

from selenium import webdriver

# Opens a firefox webbrowser for scrapping purposes

browser = webdriver.Firefox(executable_path=r'your\path\geckodriver.exe') # Put your own path here

# Allows you to load a page completely (with all of the JS)

browser.get('https://www.ncbi.nlm.nih.gov/ipg/?term=WP_000177210.1')

# Delay turning the page into a soup in order to collect the newly fetched data

time.sleep(3)

# Creates the soup

soup = BeautifulSoup(browser.page_source, "html")

# Gets all the links by filtering out ones with just '/nuccore' and keeping ones that include '/nuccore'

links = [a['href'] for a in soup.find_all('a', href=True) if '/nuccore' in a['href'] and not a['href'] == '/nuccore']

笔记：

你需要这个包selenium

您需要安装GeckoDriver