class EastSpider(scrapy.Spider):
name = 'East'
allowed_domains = ['****.com']
start_urls = ['http://finance.***.com/news.html'] def parse(self, response):
nextUrl = response.xpath('//*[contains(@class,"page-btn")]/@href') for url in nextUrl.extract():
time.sleep(1) yield Request(urljoin(response.url,url))
contentUrl = response.xpath('//p[@class="title"]/a/@href') for urls in contentUrl.extract():
time.sleep(1) yield Request(urls,callback = self.parse)
pass代码如上,但是在命令行运行scrapy crawl East -o East.csv的结果,East.csv是个空文件,什么都没写进去。我看人家说要yield,但是自己搞了搞也没行...尝试了在for循环之外添加yield url、yield urls报错,说是在定义之前引用了,然后在for循环之内添加又没效果,还是空文件...
添加回答
举报
0/150
提交
取消