diff --git a/DrissionPage/__init__.py b/DrissionPage/__init__.py index cd53866..a8db515 100644 --- a/DrissionPage/__init__.py +++ b/DrissionPage/__init__.py @@ -14,4 +14,4 @@ from ._pages.chromium_page import ChromiumPage from ._pages.mix_page import MixPage from ._pages.mix_page import MixPage as WebPage -__version__ = '4.1.0.0b12' +__version__ = '4.1.0.0b13' diff --git a/DrissionPage/_base/base.py b/DrissionPage/_base/base.py index 8fa9a7c..32fb08e 100644 --- a/DrissionPage/_base/base.py +++ b/DrissionPage/_base/base.py @@ -136,10 +136,11 @@ class DrissionElement(BaseElement): for x in self.eles('xpath:./text() | *')] return [format_html(x.strip(' ').rstrip('\n')) for x in texts if x and sub('[\r\n\t ]', '', x) != ''] - def parent(self, level_or_loc=1, index=1): + def parent(self, level_or_loc=1, index=1, timeout=None): """返回上面某一级父元素,可指定层数或用查询语法定位 :param level_or_loc: 第几级父元素,1开始,或定位符 :param index: 当level_or_loc传入定位符,使用此参数选择第几个结果,1开始 + :param timeout: 时间(秒) :return: 上级元素对象 """ if isinstance(level_or_loc, int): @@ -154,7 +155,7 @@ class DrissionElement(BaseElement): else: raise TypeError('level_or_loc参数只能是tuple、int或str。') - return self._ele(loc, timeout=0, relative=True, raise_err=False, method='parent()') + return self._ele(loc, timeout=timeout, relative=True, raise_err=False, method='parent()') def child(self, locator='', index=1, timeout=None, ele_only=True): """返回直接子元素元素或节点组成的列表,可用查询语法筛选 diff --git a/DrissionPage/_base/base.pyi b/DrissionPage/_base/base.pyi index 39cd123..05eb3e1 100644 --- a/DrissionPage/_base/base.pyi +++ b/DrissionPage/_base/base.pyi @@ -13,9 +13,10 @@ from DownloadKit import DownloadKit from .._elements.none_element import NoneElement from .._elements.session_element import SessionElement from .._functions.elements import SessionElementsList +from .._pages.chromium_frame import ChromiumFrame from .._pages.chromium_page import ChromiumPage -from .._pages.session_page import SessionPage from .._pages.mix_page import MixPage +from .._pages.session_page import SessionPage class BaseParser(object): @@ -82,6 +83,8 @@ class BaseElement(BaseParser): def nexts(self): ... + def get_frame(self, loc_or_ind, timeout=None) -> ChromiumFrame: ... + class DrissionElement(BaseElement): @@ -103,7 +106,8 @@ class DrissionElement(BaseElement): def parent(self, level_or_loc: Union[tuple, str, int] = 1, - index: int = 1) -> Union[DrissionElement, None]: ... + index: int = 1, + timeout: float = None) -> Union[DrissionElement, None]: ... def child(self, locator: Union[Tuple[str, str], str, int] = '', diff --git a/DrissionPage/_elements/chromium_element.py b/DrissionPage/_elements/chromium_element.py index 539a028..810d1df 100644 --- a/DrissionPage/_elements/chromium_element.py +++ b/DrissionPage/_elements/chromium_element.py @@ -232,13 +232,14 @@ class ChromiumElement(DrissionElement): if (is_checked and uncheck) or (not is_checked and not uncheck): self.click() - def parent(self, level_or_loc=1, index=1): + def parent(self, level_or_loc=1, index=1, timeout=0): """返回上面某一级父元素,可指定层数或用查询语法定位 :param level_or_loc: 第几级父元素,1开始,或定位符 :param index: 当level_or_loc传入定位符,使用此参数选择第几个结果,1开始 + :param timeout: 查找超时时间(秒) :return: 上级元素对象 """ - return super().parent(level_or_loc, index) + return super().parent(level_or_loc, index, timeout=timeout) def child(self, locator='', index=1, timeout=None, ele_only=True): """返回当前元素的一个符合条件的直接子元素,可用查询语法筛选,可指定返回筛选结果的第几个 @@ -1041,10 +1042,11 @@ class ShadowRoot(BaseElement): Thread(target=run_js, args=(self, script, as_expr, self.owner.timeouts.script if timeout is None else timeout, args)).start() - def parent(self, level_or_loc=1, index=1): + def parent(self, level_or_loc=1, index=1, timeout=0): """返回上面某一级父元素,可指定层数或用查询语法定位 :param level_or_loc: 第几级父元素,或定位符 :param index: 当level_or_loc传入定位符,使用此参数选择第几个结果 + :param timeout: 查找超时时间(秒) :return: ChromiumElement对象 """ if isinstance(level_or_loc, int): @@ -1061,12 +1063,13 @@ class ShadowRoot(BaseElement): else: raise TypeError('level_or_loc参数只能是tuple、int或str。') - return self.parent_ele._ele(loc, timeout=0, relative=True, raise_err=False, method='parent()') + return self.parent_ele._ele(loc, timeout=timeout, relative=True, raise_err=False, method='parent()') - def child(self, locator='', index=1): + def child(self, locator='', index=1, timeout=None): """返回直接子元素元素或节点组成的列表,可用查询语法筛选 :param locator: 用于筛选的查询语法 :param index: 第几个查询结果,1开始 + :param timeout: 查找超时时间(秒) :return: 直接子元素或节点文本组成的列表 """ if not locator: @@ -1078,14 +1081,16 @@ class ShadowRoot(BaseElement): loc = loc[1].lstrip('./') loc = f'xpath:./{loc}' - ele = self._ele(loc, index=index, relative=True) + ele = self._ele(loc, index=index, relative=True, timeout=timeout) - return ele if ele else NoneElement(self.owner, 'child()', {'locator': locator, 'index': index}) + return ele if ele else NoneElement(self.owner, 'child()', + {'locator': locator, 'index': index, 'timeout': timeout}) - def next(self, locator='', index=1): + def next(self, locator='', index=1, timeout=None): """返回当前元素后面一个符合条件的同级元素,可用查询语法筛选,可指定返回筛选结果的第几个 :param locator: 用于筛选的查询语法 :param index: 第几个查询结果,1开始 + :param timeout: 查找超时时间(秒) :return: ChromiumElement对象 """ loc = get_loc(locator, True) @@ -1094,15 +1099,17 @@ class ShadowRoot(BaseElement): loc = loc[1].lstrip('./') xpath = f'xpath:./{loc}' - ele = self.parent_ele._ele(xpath, index=index, relative=True) + ele = self.parent_ele._ele(xpath, index=index, relative=True, timeout=timeout) - return ele if ele else NoneElement(self.owner, 'next()', {'locator': locator, 'index': index}) + return ele if ele else NoneElement(self.owner, 'next()', + {'locator': locator, 'index': index, 'timeout': timeout}) - def before(self, locator='', index=1): + def before(self, locator='', index=1, timeout=None): """返回文档中当前元素前面符合条件的一个元素,可用查询语法筛选,可指定返回筛选结果的第几个 查找范围不限同级元素,而是整个DOM文档 :param locator: 用于筛选的查询语法 :param index: 前面第几个查询结果,1开始 + :param timeout: 查找超时时间(秒) :return: 本元素前面的某个元素或节点 """ loc = get_loc(locator, True) @@ -1111,23 +1118,27 @@ class ShadowRoot(BaseElement): loc = loc[1].lstrip('./') xpath = f'xpath:./preceding::{loc}' - ele = self.parent_ele._ele(xpath, index=index, relative=True) + ele = self.parent_ele._ele(xpath, index=index, relative=True, timeout=timeout) - return ele if ele else NoneElement(self.owner, 'before()', {'locator': locator, 'index': index}) + return ele if ele else NoneElement(self.owner, 'before()', + {'locator': locator, 'index': index, 'timeout': timeout}) - def after(self, locator='', index=1): + def after(self, locator='', index=1, timeout=None): """返回文档中此当前元素后面符合条件的一个元素,可用查询语法筛选,可指定返回筛选结果的第几个 查找范围不限同级元素,而是整个DOM文档 :param locator: 用于筛选的查询语法 :param index: 后面第几个查询结果,1开始 + :param timeout: 查找超时时间(秒) :return: 本元素后面的某个元素或节点 """ - nodes = self.afters(locator=locator) - return nodes[index - 1] if nodes else NoneElement(self.owner, 'after()', {'locator': locator, 'index': index}) + nodes = self.afters(locator=locator, timeout=timeout) + return nodes[index - 1] if nodes else NoneElement(self.owner, 'after()', + {'locator': locator, 'index': index, 'timeout': timeout}) - def children(self, locator=''): + def children(self, locator='', timeout=None): """返回当前元素符合条件的直接子元素或节点组成的列表,可用查询语法筛选 :param locator: 用于筛选的查询语法 + :param timeout: 查找超时时间(秒) :return: 直接子元素或节点文本组成的列表 """ if not locator: @@ -1139,11 +1150,12 @@ class ShadowRoot(BaseElement): loc = loc[1].lstrip('./') loc = f'xpath:./{loc}' - return self._ele(loc, index=None, relative=True) + return self._ele(loc, index=None, relative=True, timeout=timeout) - def nexts(self, locator=''): + def nexts(self, locator='', timeout=None): """返回当前元素后面符合条件的同级元素或节点组成的列表,可用查询语法筛选 :param locator: 用于筛选的查询语法 + :param timeout: 查找超时时间(秒) :return: ChromiumElement对象组成的列表 """ loc = get_loc(locator, True) @@ -1152,12 +1164,13 @@ class ShadowRoot(BaseElement): loc = loc[1].lstrip('./') xpath = f'xpath:./{loc}' - return self.parent_ele._ele(xpath, index=None, relative=True) + return self.parent_ele._ele(xpath, index=None, relative=True, timeout=timeout) - def befores(self, locator=''): + def befores(self, locator='', timeout=None): """返回文档中当前元素前面符合条件的元素或节点组成的列表,可用查询语法筛选 查找范围不限同级元素,而是整个DOM文档 :param locator: 用于筛选的查询语法 + :param timeout: 查找超时时间(秒) :return: 本元素前面的元素或节点组成的列表 """ loc = get_loc(locator, True) @@ -1166,18 +1179,19 @@ class ShadowRoot(BaseElement): loc = loc[1].lstrip('./') xpath = f'xpath:./preceding::{loc}' - return self.parent_ele._ele(xpath, index=None, relative=True) + return self.parent_ele._ele(xpath, index=None, relative=True, timeout=timeout) - def afters(self, locator=''): + def afters(self, locator='', timeout=None): """返回文档中当前元素后面符合条件的元素或节点组成的列表,可用查询语法筛选 查找范围不限同级元素,而是整个DOM文档 :param locator: 用于筛选的查询语法 + :param timeout: 查找超时时间(秒) :return: 本元素后面的元素或节点组成的列表 """ eles1 = self.nexts(locator) loc = get_loc(locator, True)[1].lstrip('./') xpath = f'xpath:./following::{loc}' - return eles1 + self.parent_ele._ele(xpath, index=None, relative=True) + return eles1 + self.parent_ele._ele(xpath, index=None, relative=True, timeout=timeout) def ele(self, locator, index=1, timeout=None): """返回当前元素下级符合条件的一个元素 @@ -1251,7 +1265,10 @@ class ShadowRoot(BaseElement): css = [] for i in eles: c = i.css_path - if c.startswith('html:nth-child(1)>body:nth-child(1)>shadow_root:nth-child(1)'): + if c in ('html:nth-child(1)', 'html:nth-child(1)>body:nth-child(1)', + 'html:nth-child(1)>body:nth-child(1)>shadow_root:nth-child(1)'): + continue + elif c.startswith('html:nth-child(1)>body:nth-child(1)>shadow_root:nth-child(1)'): c = c[61:] css.append(c) if index is not None: @@ -1265,7 +1282,8 @@ class ShadowRoot(BaseElement): else: node_ids = [self.owner._run_cdp('DOM.querySelector', nodeId=self._node_id, selector=i)['nodeId'] for i in css] - if 0 in node_ids: + node_ids = [i for i in node_ids if i] + if not node_ids: return None r = make_chromium_eles(self.owner, _ids=node_ids, index=index, is_obj_id=False) return None if r is False else r @@ -1476,6 +1494,8 @@ def make_chromium_eles(page, _ids, index=1, is_obj_id=True, ele_only=False): else: # 获取全部 nodes = ChromiumElementsList(page=page) for obj_id in _ids: + # if obj_id == 0: + # continue tmp = get_node_func(page, obj_id, ele_only) if tmp is False: return False diff --git a/DrissionPage/_elements/chromium_element.pyi b/DrissionPage/_elements/chromium_element.pyi index c04e13d..1af2011 100644 --- a/DrissionPage/_elements/chromium_element.pyi +++ b/DrissionPage/_elements/chromium_element.pyi @@ -101,7 +101,8 @@ class ChromiumElement(DrissionElement): def parent(self, level_or_loc: Union[tuple, str, int] = 1, - index: int = 1) -> ChromiumElement: ... + index: int = 1, + timeout: float = 0) -> ChromiumElement: ... def child(self, locator: Union[Tuple[str, str], str, int] = '', @@ -309,31 +310,31 @@ class ShadowRoot(BaseElement): def run_async_js(self, script: str, *args, as_expr: bool = False, timeout: float = None) -> None: ... - def parent(self, level_or_loc: Union[str, int] = 1, index: int = 1) -> ChromiumElement: ... + def parent(self, level_or_loc: Union[str, int] = 1, index: int = 1, timeout: float = 0) -> ChromiumElement: ... def child(self, locator: Union[Tuple[str, str], str] = '', - index: int = 1) -> ChromiumElement: ... + index: int = 1, timeout:float=None) -> ChromiumElement: ... def next(self, locator: Union[Tuple[str, str], str] = '', - index: int = 1) -> ChromiumElement: ... + index: int = 1, timeout:float=None) -> ChromiumElement: ... def before(self, locator: Union[Tuple[str, str], str] = '', - index: int = 1) -> ChromiumElement: ... + index: int = 1, timeout:float=None) -> ChromiumElement: ... def after(self, locator: Union[Tuple[str, str], str] = '', - index: int = 1) -> ChromiumElement: ... + index: int = 1, timeout:float=None) -> ChromiumElement: ... - def children(self, locator: Union[Tuple[str, str], str] = '') -> List[ChromiumElement]: ... + def children(self, locator: Union[Tuple[str, str], str] = '', timeout:float=None) -> List[ChromiumElement]: ... - def nexts(self, locator: Union[Tuple[str, str], str] = '') -> List[ChromiumElement]: ... + def nexts(self, locator: Union[Tuple[str, str], str] = '', timeout:float=None) -> List[ChromiumElement]: ... - def befores(self, locator: Union[Tuple[str, str], str] = '') -> List[ChromiumElement]: ... + def befores(self, locator: Union[Tuple[str, str], str] = '', timeout:float=None) -> List[ChromiumElement]: ... - def afters(self, locator: Union[Tuple[str, str], str] = '') -> List[ChromiumElement]: ... + def afters(self, locator: Union[Tuple[str, str], str] = '', timeout:float=None) -> List[ChromiumElement]: ... def ele(self, locator: Union[Tuple[str, str], str], diff --git a/DrissionPage/_functions/elements.py b/DrissionPage/_functions/elements.py index c91abfe..2a459b4 100644 --- a/DrissionPage/_functions/elements.py +++ b/DrissionPage/_functions/elements.py @@ -170,6 +170,7 @@ class SessionFilter(SessionFilterOne): """ self._list = _text_all(self._list, SessionElementsList(page=self._list._page), text=text, fuzzy=fuzzy, contain=contain) + return self def _get_attr(self, name, value, method, equal=True): """返回通过某个方法可获得某个值的元素 diff --git a/DrissionPage/_functions/tools.py b/DrissionPage/_functions/tools.py index 29ffb9e..ed97953 100644 --- a/DrissionPage/_functions/tools.py +++ b/DrissionPage/_functions/tools.py @@ -226,6 +226,8 @@ def raise_error(result, ignore=None, user=False): r = StorageError() elif error == 'Sanitizing cookie failed': r = CookieFormatError(f'cookie格式不正确:{result["args"]}') + elif error == 'Invalid header name': + r = ValueError(f'header名不正确。\n参数:{result["args"]["headers"]}') elif error == 'Given expression does not evaluate to a function': r = JavaScriptError(f'传入的js无法解析成函数:\n{result["args"]["functionDeclaration"]}') elif error.endswith("' wasn't found"): diff --git a/DrissionPage/_functions/web.py b/DrissionPage/_functions/web.py index ccbc84c..98cd105 100644 --- a/DrissionPage/_functions/web.py +++ b/DrissionPage/_functions/web.py @@ -372,11 +372,15 @@ def format_headers(txt): """ if isinstance(txt, (dict, CaseInsensitiveDict)): for k, v in txt.items(): - txt[k] = str(v) + if k in (':method', ':scheme', ':authority', ':path'): + txt.pop(k) + else: + txt[k] = str(v) return txt headers = {} for header in txt.split('\n'): if header: name, value = header.split(': ', maxsplit=1) - headers[name] = value + if name not in (':method', ':scheme', ':authority', ':path'): + headers[name] = value return headers diff --git a/DrissionPage/_pages/chromium_frame.py b/DrissionPage/_pages/chromium_frame.py index 2f30aff..62264bd 100644 --- a/DrissionPage/_pages/chromium_frame.py +++ b/DrissionPage/_pages/chromium_frame.py @@ -10,6 +10,7 @@ from re import search, findall, DOTALL from time import sleep, perf_counter from .._elements.chromium_element import ChromiumElement +from .._functions.settings import Settings from .._pages.chromium_base import ChromiumBase from .._units.listener import FrameListener from .._units.rect import FrameRect @@ -21,6 +22,24 @@ from ..errors import ContextLostError, ElementLostError, PageDisconnectedError, class ChromiumFrame(ChromiumBase): + _Frames = {} + + def __new__(cls, owner, ele, info=None): + """ + :param owner: frame所在的页面对象 + :param ele: frame所在元素 + :param info: frame所在元素信息 + """ + node = info['node'] if info else owner._run_cdp('DOM.describeNode', backendNodeId=ele._backend_id)['node'] + if Settings.singleton_tab_obj and node['frameId'] in cls._Frames: + r = cls._Frames[node['frameId']] + while not hasattr(r, '_frame_id'): + sleep(.1) + return r + r = object.__new__(cls) + cls._Frames[node['frameId']] = r + return r + def __init__(self, owner, ele, info=None): """ :param owner: frame所在的页面对象 @@ -33,7 +52,7 @@ class ChromiumFrame(ChromiumBase): self._frame_ele = ele self._reloading = False - node = info['node'] if not info else owner._run_cdp('DOM.describeNode', backendNodeId=ele._backend_id)['node'] + node = info['node'] if info else owner._run_cdp('DOM.describeNode', backendNodeId=ele._backend_id)['node'] self._frame_id = node['frameId'] if self._is_inner_frame(): self._is_diff_domain = False @@ -385,10 +404,11 @@ class ChromiumFrame(ChromiumBase): else: return self.doc_ele._run_js(script, *args, as_expr=as_expr, timeout=timeout) - def parent(self, level_or_loc=1, index=1): + def parent(self, level_or_loc=1, index=1, timeout=0): """返回上面某一级父元素,可指定层数或用查询语法定位 :param level_or_loc: 第几级父元素,1开始,或定位符 :param index: 当level_or_loc传入定位符,使用此参数选择第几个结果,1开始 + :param timeout: 查找超时时间(秒) :return: 上级元素对象 """ return self.frame_ele.parent(level_or_loc, index) diff --git a/DrissionPage/_pages/chromium_frame.pyi b/DrissionPage/_pages/chromium_frame.pyi index 54e3594..76b9bce 100644 --- a/DrissionPage/_pages/chromium_frame.pyi +++ b/DrissionPage/_pages/chromium_frame.pyi @@ -21,6 +21,7 @@ from .._units.waiter import FrameWaiter class ChromiumFrame(ChromiumBase): + _Frames: dict = ... def __init__(self, owner: Union[ChromiumTab, ChromiumFrame], @@ -156,7 +157,8 @@ class ChromiumFrame(ChromiumBase): def parent(self, level_or_loc: Union[Tuple[str, str], str, int] = 1, - index: int = 1) -> ChromiumElement: ... + index: int = 1, + timeout: float = 0) -> ChromiumElement: ... def prev(self, locator: Union[Tuple[str, str], str, int] = '', diff --git a/DrissionPage/_units/setter.py b/DrissionPage/_units/setter.py index eea1870..1c46abd 100644 --- a/DrissionPage/_units/setter.py +++ b/DrissionPage/_units/setter.py @@ -770,6 +770,7 @@ class WindowSetter(object): return self._owner._run_cdp('Browser.getWindowForTarget') except: sleep(.1) + raise RuntimeError('获取窗口信息失败。') def _perform(self, bounds): """执行改变窗口大小操作