我正在尝试为黄页中的每个企业抓取 href。我对使用scrapy和第二天很陌生。我正在使用请求来获取实际 url 以使用蜘蛛进行搜索。我的代码做错了什么?我希望最终让scrapy 去每个企业并抓取它的地址和其他信息。# -*- coding: utf-8 -*-import scrapyimport requestssearch = "Plumbers"location = "Hammond, LA"url = "https://www.yellowpages.com/search"q = {'search_terms': search, 'geo_location_terms': location}page = requests.get(url, params=q)page = page.urlclass YellowpagesSpider(scrapy.Spider): name = 'quotes' allowed_domains = ['yellowpages.com'] start_urls = [page] def parse(self, response): self.log("I just visited: " + response.url) items = response.css('span.text::text') for items in items: print(items)
添加回答
举报
0/150
提交
取消