推荐学习书目
› Learn Python the Hard Way
Python Sites
› PyPI - Python Package Index
› http://diveintopython.org/toc/index.html
› Pocoo
值得关注的项目
› PyPy
› Celery
› Jinja2
› Read the Docs
› gevent
› pyenv
› virtualenv
› Stackless Python
› Beautiful Soup
› 结巴中文分词
› Green Unicorn
› Sentry
› Shovel
› Pyflakes
› pytest
Python 编程
› pep8 Checker
Styles
› PEP 8
› Google Python Style Guide
› Code Style from The Hitchhiker's Guide
这是一个创建于 2373 天前的主题，其中的信息可能已经有所发展或是发生改变。
import ...
def get_links():
urls = []
for page in range(1, 71):
list_view = 'http://sz.58.com/tech/pn{}'.format(page)
url_1 = 'http://sz.58.com/tech/'
url_2 = 'x.shtml'
wb_data = requests.get(list_view, headers=headers)
soup = BeautifulSoup(wb_data.text, 'html.parser')
for link in soup.select('div.job_name a'):
urls.append(url_1+str(link.get('urlparams').split('=')[-1].strip('_q'))+url_2)
return urls
def get_info():
urls = get_links()
try:
for url in urls:
wb_data = requests.get(url, headers=headers)
soup = BeautifulSoup(wb_data.text, 'html.parser')
time.sleep(2)
data = {
'job': soup.select('.pos_title')[0].text,
'salary': soup.select('.pos_salary')[0].text,
'condition': soup.select('.item_condition')[1].text,
'exprience': soup.select('.item_condition')[2].text
}
print(data)
except IndexError:
pass
except requests.exceptions.ConnectionError:
pass
get_info()
def get_links():
urls = []
for page in range(1, 71):
list_view = 'http://sz.58.com/tech/pn{}'.format(page)
url_1 = 'http://sz.58.com/tech/'
url_2 = 'x.shtml'
wb_data = requests.get(list_view, headers=headers)
soup = BeautifulSoup(wb_data.text, 'html.parser')
for link in soup.select('div.job_name a'):
urls.append(url_1+str(link.get('urlparams').split('=')[-1].strip('_q'))+url_2)
return urls
def get_info():
urls = get_links()
try:
for url in urls:
wb_data = requests.get(url, headers=headers)
soup = BeautifulSoup(wb_data.text, 'html.parser')
time.sleep(2)
data = {
'job': soup.select('.pos_title')[0].text,
'salary': soup.select('.pos_salary')[0].text,
'condition': soup.select('.item_condition')[1].text,
'exprience': soup.select('.item_condition')[2].text
}
print(data)
except IndexError:
pass
except requests.exceptions.ConnectionError:
pass
get_info()
 |
1
cuzfinal 2018-06-14 17:00:19 +08:00
肯定是 58 更新了反爬策略呗。还有并不想看你的代码。
|
 |
2
beforeuwait 2018-06-14 17:05:42 +08:00
楼上+1
|
 |
3
JCZ2MkKb5S8ZX9pq 2018-06-14 18:20:49 +08:00
是有一点反爬，但不算严格。正在跑，稳得一比。
|
 |
4
jmjiafan 2018-07-16 11:44:02 +08:00 via Android
关键是电话号码的信息采集不到了
|
Select Language