qq动态头像女生,哈斯根金融集团,常平教育网
本文为大家分享了python爬取酷狗音乐排行榜的具体代码,供大家参考,具体内容如下
#coding=utf-8 from pymongo import mongoclient import time import requests from lxml import etree client = mongoclient() #连接mongo hello = client.hello #连接数据库 user = hello.song #连接表 headers = { 'user-agent': 'mozilla/5.0 (android 6.0; nexus 5 build/mra58n)\ applewebkit/537.36 (khtml, like gecko) chrome/65.0.3325.181 mobile safari/537.36'} def get_info(url): ''' get源码,encode,解析,xpath,保存 ''' response = requests.get(url, headers=headers) response = response.text.encode('utf-8') selector = etree.html(response) soup = selector.xpath('//*[@class="pc_temp_songlist "]/ul//li/a/text()') #保存到本地 # with open('aa.txt','a') as f: # for i in soup: # f.write(i.encode('utf-8') + '\n') #存入数据库 for i in soup: user.insert({'song': i}) if __name__ == '__main__': urls = ['http://www.kugou.com/yy/rank/home/{}-8888.html?from=rank'.format(str(i)) for i in range(1, 24)] for url in urls: print(url) get_info(url)
以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持移动技术网。
如对本文有疑问,请在下面进行留言讨论,广大热心网友会与你互动!! 点击进行留言回复
Python 实现将numpy中的nan和inf,nan替换成对应的均值
python爬虫把url链接编码成gbk2312格式过程解析
网友评论