secretflow.data.io package#

Submodules#

secretflow.data.io.oss module#

Functions:

s3fs()

Return a s3 filesystem instance.

open(path[, mode])

Open a oss object.
secretflow.data.io.oss.s3fs()[source]#

Return a s3 filesystem instance.

secretflow.data.io.oss.open(path, mode='rb')[source]#

Open a oss object.

Parameters

  • path – oss file path.

  • mode – optional; open mode.

Returns

A file-like object.

secretflow.data.io.util module#

Functions:

open(filepath[, mode])

Open a oss/http/https file.

is_local_file(uri)

read_csv_wrapper(filepath, **kwargs)

A wrapper of pandas read_csv and supports oss file.

to_csv_wrapper(df, filepath, **kwargs)

A wrapper of pandas to_csv and supports oss file.
secretflow.data.io.util.open(filepath: Union[str, Path], mode='rb')[source]#

Open a oss/http/https file.

Parameters

  • filepath – The file path, which can be an oss, or pathlib.Path object.

  • mode – optional. open mode.

Returns

the file object.

secretflow.data.io.util.is_local_file(uri: str) → bool[source]#
secretflow.data.io.util.read_csv_wrapper(filepath, **kwargs) → DataFrame[source]#

A wrapper of pandas read_csv and supports oss file.

Parameters

  • filepath – the file path.

  • kwargs – all other arguments are same with pandas.DataFrame.read_csv().

Returns

a pandas DataFrame.

secretflow.data.io.util.to_csv_wrapper(df: DataFrame, filepath, **kwargs)[source]#

A wrapper of pandas to_csv and supports oss file.

Parameters

  • filepath – the file path.

  • kwargs – all other arguments are same with pandas.DataFrame.read_csv().

Returns

a pandas DataFrame.

Module contents#