secretflow.utils package

Subpackages

• secretflow.utils.simulation package
  • Subpackages
  • Submodules
  • secretflow.utils.simulation.datasets module
  • secretflow.utils.simulation.tf_gnn_model module
  • Module contents

Submodules

secretflow.utils.compressor module

Classes:

Compressor()	Abstract base class for cross device data compressor
SparseCompressor(sparse_rate)
RandomSparse(sparse_rate)	Random sparse compressor compress data by randomly set element to zero.
TopkSparse(sparse_rate)	Topk sparse compressor use topK algorithm to transfer dense matrix into sparse matrix.

class secretflow.utils.compressor.Compressor

基类：ABC

Abstract base class for cross device data compressor

Methods:

compress(data)	Compress data before send.
decompress(data)	Decompress data after receive.

abstract compress(data: Union[ndarray, List[ndarray]]) → Union[Any, List[Any]]

Compress data before send.

参数

data (Union[np.ndarray, List[np.ndarray]]) – data need to compress.

返回

compressed data.

返回类型

Union[Any, List[Any]]

abstract decompress(data: Union[Any, List[Any]]) → Union[ndarray, List[ndarray]]

Decompress data after receive.

参数

data (Union[Any, List[Any]]) – data need to decompress.

返回

decompressed data.

返回类型

Union[np.ndarray, List[np.ndarray]]

class secretflow.utils.compressor.SparseCompressor(sparse_rate: float)

基类：Compressor

Methods:

__init__(sparse_rate)	Initialize
compress(data)	Compress data to sparse matrix before send.
decompress(data)	Decompress data from sparse matrix to dense after received.

__init__(sparse_rate: float)

Initialize

参数

sparse_rate – the percentage of cells are zero.

compress(data: Union[ndarray, List[ndarray]]) → Union[spmatrix, List[spmatrix]]

Compress data to sparse matrix before send.

参数

data (Union[np.ndarray, List[np.ndarray]]) – data need to compress.

返回

compressed data.

返回类型

Union[sparse.spmatrix, List[sparse.spmatrix]]

decompress(data: Union[spmatrix, List[spmatrix]]) → Union[ndarray, List[ndarray]]

Decompress data from sparse matrix to dense after received.

参数

data (Union[sparse.spmatrix, List[sparse.spmatrix]]) – data need to decompress.

返回

decompressed data.

返回类型

Union[np.ndarray, List[np.ndarray]]

class secretflow.utils.compressor.RandomSparse(sparse_rate: float)

基类：SparseCompressor

Random sparse compressor compress data by randomly set element to zero.

Methods:

__init__(sparse_rate)

Initialize

__init__(sparse_rate: float)

Initialize

参数

sparse_rate – the percentage of cells are zero.

class secretflow.utils.compressor.TopkSparse(sparse_rate: float)

基类：SparseCompressor

Topk sparse compressor use topK algorithm to transfer dense matrix into sparse matrix.

Methods:

__init__(sparse_rate)

Initialize

__init__(sparse_rate: float)

Initialize

参数

sparse_rate – the percentage of cells are zero.

secretflow.utils.errors module

Exceptions:

AlreadyExistsError	Raise when already exists.
InvalidArgumentError	Raise when invalid argument.
NotFoundError	Raise if not found.
PartyNotFoundError	Raise if party not found.
UnexpectedError	Raise when unexpected.
HttpNotOkError	Raise if http code is not 200

exception secretflow.utils.errors.AlreadyExistsError

基类：Exception

Raise when already exists.

exception secretflow.utils.errors.InvalidArgumentError

基类：Exception

Raise when invalid argument.

exception secretflow.utils.errors.NotFoundError

基类：Exception

Raise if not found.

exception secretflow.utils.errors.PartyNotFoundError

基类：Exception

Raise if party not found.

exception secretflow.utils.errors.UnexpectedError

基类：Exception

Raise when unexpected.

exception secretflow.utils.errors.HttpNotOkError

基类：Exception

Raise if http code is not 200

secretflow.utils.hash module

Functions:

sha256sum(filename)

secretflow.utils.hash.sha256sum(filename: str)

secretflow.utils.io module

Functions:

rows_count(filename)

get rows count from file

secretflow.utils.io.rows_count(filename)

get rows count from file

secretflow.utils.ndarray_bigint module

Functions:

randbits(shape, bits)
randint(shape, min, max)
arange(max)
zeros(shape)

Classes:

BigintNdArray(data, shape)

secretflow.utils.ndarray_bigint.randbits(shape: tuple, bits)

secretflow.utils.ndarray_bigint.randint(shape: tuple, min, max)

secretflow.utils.ndarray_bigint.arange(max)

secretflow.utils.ndarray_bigint.zeros(shape)

class secretflow.utils.ndarray_bigint.BigintNdArray(data, shape)

基类：object

Methods:

__init__(data, shape)
resize(shape)
to_list()
to_numpy()
to_hnp(encoder)
to_bytes(bytes_per_int[, byteorder])

__init__(data, shape)

resize(shape)

to_list()

to_numpy()

to_hnp(encoder)

to_bytes(bytes_per_int, byteorder='little')

secretflow.utils.ndarray_encoding module

Functions:

encode(m, fxp_bits)	Encode float ndarray to uint64 finite field.
decode(m, fxp_bits)	Decode ndarray from uint64 finite field to the float.

secretflow.utils.ndarray_encoding.encode(m: ndarray, fxp_bits: int) → ndarray

Encode float ndarray to uint64 finite field. Float will times 2**fxp_bits firstly.

参数

• m (np.ndarray) – the ndarray to encode.

• fraction_precision (int) – keep how many decimal digits after the dot. Must provide if ndarray dtype is float.

返回

the encoded ndarray.

返回类型

np.ndarray

secretflow.utils.ndarray_encoding.decode(m: ndarray, fxp_bits: int) → ndarray

Decode ndarray from uint64 finite field to the float. Fraction precision shall be corresponding to encoding fraction precision.

参数

• m (np.ndarray) – the ndarray to decode.

• fxp_bits (int) – the decimal digits to keep when encoding float. Must provide if the original dtype is float.

返回

the decoded float ndarray.

返回类型

np.ndarray

secretflow.utils.sigmoid module

Functions:

t1_sig(x[, limit])	taylor series referenced from: https://mortendahl.github.io/2017/04/17/private-deep-learning-with-mpc/
t3_sig(x[, limit])	taylor series referenced from: https://mortendahl.github.io/2017/04/17/private-deep-learning-with-mpc/
t5_sig(x[, limit])	taylor series referenced from: https://mortendahl.github.io/2017/04/17/private-deep-learning-with-mpc/
seg3_sig(x)	f(x) = 0.5 + 0.125x if -4 <= x <= 4
df_sig(x)	https://dergipark.org.tr/en/download/article-file/54559 Dataflow implementation of sigmoid function: F(x) = 0.5 * ( x / ( 1 + |x| ) ) + 0.5 df_sig has higher precision than sr_sig if x in [-2, 2]
sr_sig(x)	https://en.wikipedia.org/wiki/Sigmoid_function#Examples Square Root approximation functions: F(x) = 0.5 * ( x / ( 1 + x^2 )^0.5 ) + 0.5 sr_sig almost perfect fit to sigmoid if x out of range [-3,3]
ls7_sig(x)	Polynomial fitting
mix_sig(x)	mix ls7 & sr sig, use ls7 if |x| < 4 , else use sr.
real_sig(x)
sigmoid(x, sig_type)

Classes:

SigType(value)

An enumeration.

secretflow.utils.sigmoid.t1_sig(x, limit: bool = True)

taylor series referenced from: https://mortendahl.github.io/2017/04/17/private-deep-learning-with-mpc/

secretflow.utils.sigmoid.t3_sig(x, limit: bool = True)

taylor series referenced from: https://mortendahl.github.io/2017/04/17/private-deep-learning-with-mpc/

secretflow.utils.sigmoid.t5_sig(x, limit: bool = True)

taylor series referenced from: https://mortendahl.github.io/2017/04/17/private-deep-learning-with-mpc/

secretflow.utils.sigmoid.seg3_sig(x)

f(x) = 0.5 + 0.125x if -4 <= x <= 4

1 if x > 4 0 if -4 > x

secretflow.utils.sigmoid.df_sig(x)

https://dergipark.org.tr/en/download/article-file/54559 Dataflow implementation of sigmoid function: F(x) = 0.5 * ( x / ( 1 + |x| ) ) + 0.5 df_sig has higher precision than sr_sig if x in [-2, 2]

secretflow.utils.sigmoid.sr_sig(x)

https://en.wikipedia.org/wiki/Sigmoid_function#Examples Square Root approximation functions: F(x) = 0.5 * ( x / ( 1 + x^2 )^0.5 ) + 0.5 sr_sig almost perfect fit to sigmoid if x out of range [-3,3]

secretflow.utils.sigmoid.ls7_sig(x)

Polynomial fitting

secretflow.utils.sigmoid.mix_sig(x)

mix ls7 & sr sig, use ls7 if |x| < 4 , else use sr. has higher precision in all input range. NOTICE: this method is very expensive, only use for hessian matrix.

secretflow.utils.sigmoid.real_sig(x)

class secretflow.utils.sigmoid.SigType(value)

基类：Enum

An enumeration.

Attributes:

REAL
T1
T3
T5
DF
SR
MIX

REAL = 'real'

T1 = 't1'

T3 = 't3'

T5 = 't5'

DF = 'df'

SR = 'sr'

MIX = 'mix'

secretflow.utils.sigmoid.sigmoid(x, sig_type: SigType)

secretflow.utils.testing module

Functions:

unused_tcp_port()	Return an unused port
cluster_def(parties[, runtime_config])	Generate SPU cluster_def for testing
heu_config(sk_keeper, evaluators)

secretflow.utils.testing.unused_tcp_port() → int

Return an unused port

secretflow.utils.testing.cluster_def(parties: List[str], runtime_config=None)

Generate SPU cluster_def for testing

secretflow.utils.testing.heu_config(sk_keeper: str, evaluators: List[str])

Module contents