在Python的网络爬虫中,网页解析是一项重要的技术。而在泛滥的网页解析库中,BeautifulSoup库凭借其简略易用而广受欢迎。在本篇文章中,咱们将学习BeautifulSoup库的根本用法。
一、BeautifulSoup的装置与根本应用
首先,咱们须要应用pip命令来装置BeautifulSoup库,命令如下:
pip install beautifulsoup4
装置实现后,咱们就能够开始应用BeautifulSoup来解析网页了。首先,咱们须要导入BeautifulSoup类,而后应用BeautifulSoup类的构造方法创立一个BeautifulSoup对象,代码如下:
from bs4 import BeautifulSouphtml_doc = """<html><head><title>The Dormouse's story</title></head><body><p class="title"><b>The Dormouse's story</b></p>"""soup = BeautifulSoup(html_doc, 'html.parser')print(soup.prettify())
二、网页元素的提取
BeautifulSoup提供了一系列办法,让咱们能够轻松的提取出网页中的元素。例如,咱们能够应用tag.name
属性获取标签的名字,tag.string
属性获取标签内的字符串,应用tag['attr']
获取标签的属性,代码如下:
from bs4 import BeautifulSouphtml_doc = """<html><head><title>The Dormouse's story</title></head><body><p class="title"><b>The Dormouse's story</b></p>"""soup = BeautifulSoup(html_doc, 'html.parser')title_tag = soup.titleprint(title_tag.name) # 输入:titleprint(title_tag.string) # 输入:The Dormouse's story
三、网页元素的查找
BeautifulSoup提供了find
和find_all
办法,让咱们能够轻松的查找到网页中的元素。例如,咱们能够查找到所有的p
标签,代码如下:
from bs4 import BeautifulSouphtml_doc = """<html><head><title>The Dormouse's story</title></head><body><p class="title"><b>The Dormouse's story</b></p><p class="story">Once upon a time there were three little sisters; and their names were</p>"""soup = BeautifulSoup(html_doc, 'html.parser')p_tags = soup.find_all('p')for p in p_tags: print(p.string)
四、CSS选择器的应用
BeautifulSoup还反对CSS选择器,咱们能够应用select
办法来应用CSS选择器抉择元素,例如:
from bs4 import BeautifulSouphtml_doc = """<html><head><title>The Dormouse's story</title></head><body><p class="title"><b>The Dormouse's story</b></p><p class="story">Once upon a time there were three little sisters; and their names were</p>"""soup = BeautifulSoup(html_doc, 'html.parser')title_tag = soup.select('p.title')for title in title_tag: print(title.string)
以上就是BeautifulSoup库的根本用法,通过BeautifulSoup,咱们能够轻松地解析出网页中的元素,为网络爬虫提供弱小的反对。