sample_size – The maximum number of files you would like to load from the
directory.
randomize_sample – Suffle the files to get a random sample.
sample_seed – set the seed of the random shuffle for reporoducibility.
Methods
__init__(path[, glob, silent_errors, ...])
Initialize with a path to directory and how to glob over it.
lazy_load()
A lazy loader for Documents.
load()
Load documents.
load_and_split([text_splitter])
Load Documents and split into chunks.
load_file(item, path, docs, pbar)
Load a file.
__init__(path: str, glob: str = '**/[!.]*', silent_errors: bool = False, load_hidden: bool = False, loader_cls: ~typing.Union[~typing.Type[~langchain.document_loaders.unstructured.UnstructuredFileLoader], ~typing.Type[~langchain.document_loaders.text.TextLoader], ~typing.Type[~langchain.document_loaders.html_bs.BSHTMLLoader]] = <class 'langchain.document_loaders.unstructured.UnstructuredFileLoader'>, loader_kwargs: ~typing.Optional[dict] = None, recursive: bool = False, show_progress: bool = False, use_multithreading: bool = False, max_concurrency: int = 4, *, sample_size: int = 0, randomize_sample: bool = False, sample_seed: ~typing.Optional[int] = None)[source]¶
Initialize with a path to directory and how to glob over it.
Parameters
path – Path to directory.
glob – Glob pattern to use to find files. Defaults to “**/[!.]*”
(all files except hidden).
silent_errors – Whether to silently ignore errors. Defaults to False.
load_hidden – Whether to load hidden files. Defaults to False.