matching the glob will be loaded.
exclude – A list of patterns to exclude from the loader.
show_progress – Whether to show a progress bar or not (requires tqdm).
Proxies to the file system loader.
parser – A blob parser which knows how to parse blobs into documents
num_workers – Max number of concurrent workers to use.
lazy_load() → Iterator[Document][source]¶
Load documents lazily with concurrent parsing.
load() → List[Document]¶
Load all documents.
load_and_split(text_splitter: Optional[TextSplitter] = None) → List[Document]¶
Load all documents and split them into sentences.
Examples using ConcurrentLoader¶
Concurrent Loader