Day:006（2 ） | Python爬虫：高效数据抓取的编程技术（爬虫工具）

selenium 层级定位

窗口的定位：

对于一个现代的web应用，经常会出现框架（frame）或窗口（window）的应用，这也就给我们的定位带来了一个难题。

有时候我们定位一个元素，定位器没有问题，但一直定位不了，这时候就要检查这个元素是否在一个frame中，seelnium webdriver提供了一个switch_to_frame方法，可以很轻松的来解决这个问题。

多层框架或窗口的定位：

driver.switch_to.frame()

frame.html

<html>
<head><meta http-equiv="content-type"
content="text/html;charset=utf-8" /><title>frame</title><script type="text/javascript"
src="https://cdn.jsdelivr.net/npm/jquery@1.1
2.4/dist/jquery.min.js"></script><link href="http://netdna.bootstrapcdn.com/twitterbootstrap/2.3.2/css/bootstrapcombined.min.css"
rel="stylesheet" />
</head><body><div class="row-fluid"><div class="span10 well"><h3>frame</h3><iframe id="f1"
src="inner.html" width="800" , height="600">
</iframe></div></div>
</body>
<script
src="https://cdn.jsdelivr.net/npm/@bootcss/v
3.bootcss.com@1.0.8/dist/js/bootstrap.min.js
"></script></html>
</html>

inner.html

<html>
<head><meta http-equiv="content-type"
content="text/html;charset=utf-8" /><title>inner</title>
</head>
<body><div class="row-fluid"><div class="span6 well"><h3>inner</h3><iframe id="f2"
src="https://cn.bing.com/" width="700"
height="500"></iframe></div></div>
</body>
</html>

switch_to_frame()

from selenium.webdriver.chrome.service
import Service
from selenium import webdriver
from selenium.webdriver.common.by import By
import os
from time import sleepdef select_frame():# 创建驱动s =
Service(executable_path='./chromedriver.exe')# 创建浏览器driver = webdriver.Chrome(service=s)file_path = 'file:///' +os.path.abspath('./html/outer.html')# 打开网页driver.get(file_path)# 切换framedriver.switch_to.frame('f1')driver.switch_to.frame('f2')# 定位元素，输入要搜索的内容driver.find_element(By.ID,'sb_form_q').send_keys('百战')# 定位按钮，点击搜索driver.find_element(By.ID,'search_icon').click()sleep(3)driver.quit()
if __name__ =='__main__':select_frame()

selenium 处理下拉框

在爬取数据时，有时数据太多，而官网提供了筛选功能select标签，像这样的数据，我们只需要定位元素，点击即可。

<html><body><select id="ShippingMethod"
onchange="updateShipping(options[selectedIndex]);" name="ShippingMethod"><option value="12.51">UPS NextDay Air ==> $12.51</option><option value="11.61">UPS NextDay Air Saver ==> $11.61</option><option value="10.69">UPS 3 DaySelect ==> $10.69</option><option value="9.03">UPS 2nd DayAir ==> $9.03</option><option value="8.34">UPS Ground==> $8.34</option><option value="9.25">USPSPriority Mail Insured ==> $9.25</option><option value="7.45">USPSPriority Mail ==> $7.45</option><option value="3.20"selected="">USPS First Class ==>$3.20</option></select></body>
</html>

from selenium.webdriver.common.by import By
driver= webdriver.Chrome()
file_path =  'file:///' +
os.path.abspath('drop_down.html')
driver.get(file_path)
time.sleep(2)
m=driver.find_element(By.ID,"ShippingMethod"
)
m.find_element(By.XPATH,"//option[@value='10
.69']").click()
time.sleep(3)
driver.quit()

selenium处理弹窗

有时，页面可能要弹窗口。只需要去定位弹窗上的“确定”按钮即可

switch_to

焦点集中到页面上的一个警告（提示）

accept()

接受警告提示

切换至弹窗

chrome.switch_to.alert.accept()
chrome.switch_to_alert().accept()  # 过期

代码

html

<!DOCTYPE html>
<html lang="en">
<head><meta charset="UTF-8"><title>This is a page</title>
</head>
<body><div id = "container"><div style="font: size30px;">Hello,Python Spider</div></div>
</body>
<script>alert('这个是测试弹窗')
</script>
</html>

python

from lib2to3.pgen2 import driver
from selenium.webdriver.chrome.service
import Service
from selenium import webdriver
from time import sleep
import osdef test_windows():# 创建驱动对象s = Service('./chromedriver.exe')# 创建浏览器driver = webdriver.Chrome(service=s)# 访问页面# 设置页面地址file_path = 'file:///' +os.path.abspath('./html/test03.html')driver.get(file_path)sleep(3)# 定位弹出窗口，并点击driver.switch_to.alert.accept()sleep(3)driver.quit()if __name__ =='__main__':test_windows()

selenium拖拽元素

要完成元素的拖拽，首先需要指定被拖动的元素和拖动目标元素，然后利用 ActionChains 类来实现，ActionChains用于定制动作。

通过ActionChains对象中的perform()执行动作

代码

html

<!doctype html>
<html lang="en">
<head><meta charset="utf-8"><meta name="viewport"content="width=device-width, initialscale=1"><title>jQuery UI Draggable - Autoscroll</title><link rel="stylesheet" href="http://code.jquery.com/ui/1.12.1/themes/base/jqueryui.css"><style>#draggable, #draggable2, #draggable3 {
width: 100px; 
height: 100px; 
padding: 0.5em;
float: left; 
margin: 0 10px 10px 0; 
}body {font-family: Arial, Helvetica,sans-serif;}table {font-size: 1em;}.ui-draggable, .ui-droppable{
background-position: top;
}</style><script src="https://code.jquery.com/jquery1.12.4.js"></script><script src="https://code.jquery.com/ui/1.12.1/jquery-ui.js"></script><script>$( function() {$( "#draggable" ).draggable({scroll: true });$( "#draggable2" ).draggable({scroll: true, scrollSensitivity: 100 });$( "#draggable3" ).draggable({scroll: true, scrollSpeed: 100 });} );</script></head>
<body><div id="draggable" class="ui-widgetcontent"><p>Scroll set to true, defaultsettings</p>
</div><div id="draggable2" class="ui-widgetcontent"><p>scrollSensitivity set to 100</p>
</div><div id="draggable3" class="ui-widgetcontent"><p>scrollSpeed set to 100</p>
</div>
<div style="height: 5000px; width: 1px;">
</div>
</body>
</html>

python

import os
from time import sleep
from selenium import webdriver
from selenium.webdriver import ActionChainschrome =webdriver.Chrome(executable_path='./tools/chromedriver.exe')
file =f'file:///{os.path.abspath("./html/scroll.html")}'
chrome.get(file)div1 =chrome.find_element_by_id('draggable')
div2 =chrome.find_element_by_id('draggable2')
div3 =chrome.find_element_by_id('draggable3')
print(div1)
print(div2)
print(div3)
sleep(2)ac1 =ActionChains(chrome).drag_and_drop(div1,div2)
ac1.perform()
sleep(2)ac2 =ActionChains(chrome).drag_and_drop_by_offset(div3,10,10)
for i in range(5):ac2.perform()sleep(1)sleep(3)
chrome.quit()

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：http://www.mzph.cn/news/808582.shtml

如若内容造成侵权/违法违规/事实不符，请联系多彩编程网进行投诉反馈email:809451989@qq.com，一经查实，立即删除！