pyarrow.fs.FileSelector

class pyarrow.fs.FileSelector(base_dir, bool allow_not_found=False, bool recursive=False)

Bases: _Weakrefable

File and directory selector.

It contains a set of options that describes how to search for files and directories.

Parameters:

base_dirstr

The directory in which to select files. Relative paths also work, use ‘.’ for the current directory and ‘..’ for the parent.

allow_not_foundbool, default False

The behavior if base_dir doesn’t exist in the filesystem. If false, an error is returned. If true, an empty selection is returned.

recursivebool, default False

Whether to recurse into subdirectories.

Examples

List the contents of a directory and subdirectories:

>>> selector_1 = fs.FileSelector(local_path, recursive=True)
>>> local.get_file_info(selector_1) 
[<FileInfo for 'tmp/alphabet/example.dat': type=FileType.File, size=4>,
<FileInfo for 'tmp/alphabet/subdir': type=FileType.Directory>,
<FileInfo for 'tmp/alphabet/subdir/example_copy.dat': type=FileType.File, size=4>]

List only the contents of the base directory:

>>> selector_2 = fs.FileSelector(local_path)
>>> local.get_file_info(selector_2) 
[<FileInfo for 'tmp/alphabet/example.dat': type=FileType.File, size=4>,
<FileInfo for 'tmp/alphabet/subdir': type=FileType.Directory>]

Return empty selection if the directory doesn’t exist:

>>> selector_not_found = fs.FileSelector(local_path + '/missing',
...                                      recursive=True,
...                                      allow_not_found=True)
>>> local.get_file_info(selector_not_found)
[]

__init__(*args, **kwargs)

Methods

__init__(*args, **kwargs)

Attributes

allow_not_found
base_dir
recursive

allow_not_found

base_dir

recursive