secretflow.ml.nn.fl.backend.tensorflow package#

Subpackages#

Submodules#

secretflow.ml.nn.fl.backend.tensorflow.fl_base module#

Classes:

BaseModel(builder_base[, builder_fuse])

BaseTFModel(builder_base, *[, _ray_trace_ctx])

class secretflow.ml.nn.fl.backend.tensorflow.fl_base.BaseModel(builder_base: Callable, builder_fuse: Optional[Callable] = None)[source]#

Bases: ABC

Methods:

__init__(builder_base[, builder_fuse])

build_dataset(x[, y, batch_size, ...])

get_weights()

evaluate(x, y[, batch_size, verbose, ...])

__init__(builder_base: Callable, builder_fuse: Optional[Callable] = None)[source]#
abstract build_dataset(x: ndarray, y: Optional[ndarray] = None, batch_size=32, buffer_size=128, repeat_count=1)[source]#
abstract get_weights()[source]#
abstract evaluate(x, y, batch_size=None, verbose=1, sample_weight=None, steps=None)[source]#
class secretflow.ml.nn.fl.backend.tensorflow.fl_base.BaseTFModel(builder_base: Callable[[], Model], *, _ray_trace_ctx=None)[source]#

Bases: BaseModel

Methods:

__init__(builder_base, *[, _ray_trace_ctx])

build_dataset_from_csv(csv_file_path, label)

build tf.data.Dataset

build_dataset(x[, y, s_w, sampling_rate, ...])

build tf.data.Dataset

get_rows_count(filename)

get_weights()

set_weights(weights)

set weights of client model

set_validation_metrics(global_metrics)

wrap_local_metrics()

evaluate([evaluate_steps])

predict([predict_steps])

init_training(callbacks[, epochs, steps, ...])

on_train_begin()

on_epoch_begin(epoch)

on_epoch_end(epoch)

on_train_end()

get_stop_training()

train_step(weights, cur_steps, train_steps, ...)

save_model(model_path)

load_model(model_path)

__init__(builder_base: Callable[[], Model], *, _ray_trace_ctx=None)[source]#
build_dataset_from_csv(csv_file_path: str, label: str, sampling_rate=None, shuffle=False, random_seed=1234, na_value='?', repeat_count=1, sample_length=0, buffer_size=None, ignore_errors=True, prefetch_buffer_size=None, stage='train', label_decoder=None)[source]#

build tf.data.Dataset

Parameters

  • csv_file_path – Dict of csv file path

  • label – label column name

  • sampling_rate – Sampling rate of a batch

  • shuffle – A bool that indicates whether the input should be shuffled

  • random_seed – Randomization seed to use for shuffling.

  • na_value – Additional string to recognize as NA/NaN.

  • repeat_count – num of repeats

  • sample_length – num of sample length

  • buffer_size – shuffle size

  • ignore_errors – if True, ignores errors with CSV file parsing,

  • prefetch_buffer_size – An int specifying the number of feature batches to prefetch for performance improvement.

  • stage – the stage of the datset

  • label_decoder – callable function for label preprocess

build_dataset(x: ndarray, y: Optional[ndarray] = None, s_w: Optional[ndarray] = None, sampling_rate=None, buffer_size=None, shuffle=False, random_seed=1234, repeat_count=1, sampler_method='batch', stage='train')[source]#

build tf.data.Dataset

Parameters

  • x – feature, FedNdArray or HDataFrame

  • y – label, FedNdArray or HDataFrame

  • s_w – sample weight of this dataset

  • sampling_rate – Sampling rate of a batch

  • buffer_size – shuffle size

  • shuffle – A bool that indicates whether the input should be shuffled

  • random_seed – Prg seed for shuffling

  • repeat_count – num of repeats

  • sampler_method – method of sampler

get_rows_count(filename)[source]#
get_weights()[source]#
set_weights(weights)[source]#

set weights of client model

set_validation_metrics(global_metrics)[source]#
wrap_local_metrics()[source]#
evaluate(evaluate_steps=0)[source]#
predict(predict_steps=0)[source]#
init_training(callbacks, epochs=1, steps=0, verbose=0)[source]#
on_train_begin()[source]#
on_epoch_begin(epoch)[source]#
on_epoch_end(epoch)[source]#
on_train_end()[source]#
get_stop_training()[source]#
abstract train_step(weights, cur_steps, train_steps, **kwargs)[source]#
save_model(model_path: str)[source]#
load_model(model_path: str)[source]#

secretflow.ml.nn.fl.backend.tensorflow.sampler module#

Functions:

batch_sampler(x, y, s_w, sampling_rate, ...)

implementation of batch sampler

possion_sampler(x, y, s_w, sampling_rate, ...)

implementation of possion sampler

sampler_data([sampler_method, x, y, s_w, ...])

do sample data by sampler_method
secretflow.ml.nn.fl.backend.tensorflow.sampler.batch_sampler(x, y, s_w, sampling_rate, buffer_size, shuffle, repeat_count, random_seed)[source]#

implementation of batch sampler

Parameters

  • x – feature, FedNdArray or HDataFrame

  • y – label, FedNdArray or HDataFrame

  • s_w – sample weight of this dataset

  • sampling_rate – Sampling rate of a batch

  • buffer_size – shuffle size

  • shuffle – A bool that indicates whether the input should be shuffled

  • repeat_count – num of repeats

  • random_seed – Prg seed for shuffling

Returns

tf.data.Dataset

Return type

data_set

secretflow.ml.nn.fl.backend.tensorflow.sampler.possion_sampler(x, y, s_w, sampling_rate, random_seed)[source]#

implementation of possion sampler

Parameters

  • x – feature, FedNdArray or HDataFrame

  • y – label, FedNdArray or HDataFrame

  • s_w – sample weight of this dataset

  • sampling_rate – Sampling rate of a batch

  • random_seed – Prg seed for shuffling

Returns

tf.data.Dataset

Return type

data_set

secretflow.ml.nn.fl.backend.tensorflow.sampler.sampler_data(sampler_method='batch', x=None, y=None, s_w=None, sampling_rate=None, buffer_size=None, shuffle=False, repeat_count=1, random_seed=1234)[source]#

do sample data by sampler_method

Parameters

  • x – feature, FedNdArray or HDataFrame

  • y – label, FedNdArray or HDataFrame

  • s_w – sample weight of this dataset

  • sampling_rate – Sampling rate of a batch

  • buffer_size – shuffle size

  • shuffle – A bool that indicates whether the input should be shuffled

  • repeat_count – num of repeats

  • random_seed – Prg seed for shuffling

Returns

tf.data.Dataset

Return type

data_set

Module contents#