昺h zddlZddlZddlmZmZddlmZddlmZej e Z GddeZ dS)N)IteratorList)Document) BaseLoadercjeZdZdZdeefdZdedefdZdee fdZ dee fdZ d S) AsyncChromiumLoaderzJScrape HTML pages from URLs using a headless instance of the Chromium.urlscX||_ ddl}dS#t$rtdwxYw)z Initialize the loader with a list of URL paths. Args: urls (List[str]): A list of URLs to scrape content from. Raises: ImportError: If the required 'playwright' package is not installed. rNz`playwright is required for AsyncChromiumLoader. Please install it with `pip install playwright`.)r playwright ImportError)selfr r s u/var/www/html/services/rag-service/venv/lib/python3.11/site-packages/langchain_community/document_loaders/chromium.py__init__zAsyncChromiumLoader.__init__sU           C  s)urlreturnc:Kddlm}tdd}|4d{V}|jdd{V} |d{V}||d{V|d{V}tdn#t$r}d |}Yd}~nd}~wwxYw| d{Vdddd{Vn#1d{VswxYwY|S) a Asynchronously scrape the content of a given URL using Playwright's async API. Args: url (str): The URL to scrape. Returns: str: The scraped HTML content or an error message if an exception occurs. r)async_playwrightzStarting scraping...NT)headlesszContent scrapedzError: ) playwright.async_apirloggerinfochromiumlaunchnew_pagegotocontent Exceptionclose)r rrresultspbrowserpagees rascrape_playwrightz&AsyncChromiumLoader.ascrape_playwright's :99999 *+++##%% " " " " " " "J--t-<<<<<<<'>s'C'CDDL #H xHHH H H H H I IcDt|S)z Load and return all Documents from the provided URLs. Returns: List[Document]: A list of Document objects containing the scraped content from each URL. )listr-)r s rloadzAsyncChromiumLoader.loadRsDNN$$%%%r.N) __name__ __module__ __qualname____doc__rstrrr%rrr-r1r.rrr s**3i.CC6I8H-IIII &d8n & & & & & &r.r) r*loggingtypingrrlangchain_core.documentsr)langchain_community.document_loaders.baser getLoggerr2rrr7r.rr=s!!!!!!!!------@@@@@@  8 $ $O&O&O&O&O&*O&O&O&O&O&r.