Extended Specification#

Table of Contents#

cluster.proto#

Messages

data.proto#

Messages
- DeviceObjectCollection
- DeviceObjectCollection.DeviceObject

cluster.proto#

Proto file: secretflow/protos/secretflow/spec/extend/cluster.proto

Messages#

SFClusterConfig#

Runtime Config for a SecretFlow cluster. Besides intrinsic SFClusterDesc, dynamic network configs are provided.

Field	Type	Description
desc	SFClusterDesc	Intrinsic properties.
public_config	SFClusterConfig.PublicConfig	Dynamic runtime public configs.
private_config	SFClusterConfig.PrivateConfig	Dynamic runtime private configs.

SFClusterConfig.PrivateConfig#

Different for each party. Private and unique to each party.

Field	Type	Description
self_party	string	none
ray_head_addr	string	none

SFClusterConfig.PublicConfig#

Public and shared to all parties.

Field	Type	Description
ray_fed_config	SFClusterConfig.RayFedConfig	none
spu_configs	repeated SFClusterConfig.SPUConfig	none

SFClusterConfig.RayFedConfig#

Addresses for the RayFed cluster of each party.

Field	Type	Description
parties	repeated string	none
addresses	repeated string	none
listen_addresses	repeated string	none

SFClusterConfig.SPUConfig#

Contains addresses for one SPU device.

Field	Type	Description
name	string	Should match SPU name in SFClusterDesc.devices.
parties	repeated string	none
addresses	repeated string	none
listen_addresses	repeated string	none

SFClusterDesc#

Intrinsic properties of a SecretFlow cluster, including:

Version info.
Parties: who participate in the computation.
Secret devices including and their configs.

Field	Type	Description
sf_version	string	SecretFlow version.
py_version	string	Python version.
parties	repeated string	Joined entities. e.g. [“alice”, “bob”,…].
devices	repeated SFClusterDesc.DeviceDesc	Description of secret devices
ray_fed_config	SFClusterDesc.RayFedConfig	none

SFClusterDesc.DeviceDesc#

Description for a secret device.

PYUs do not need to claim since they are plaintext devices.
Notes for config: At this moment, you have to provide a JSON string for different devices. We are going to formalize this part in future.
- Example SPU config:

  {
    "runtime_config": {
        "protocol": "REF2K",
        "field": "FM64"
    },
    "link_desc": {
        "connect_retry_times": 60,
        "connect_retry_interval_ms": 1000,
        "brpc_channel_protocol": "http",
        "brpc_channel_connection_type": "pooled",
        "recv_timeout_ms": 1200000,
        "http_timeout_ms": 1200000
    }
}

Referrences: SPU: https://www.secretflow.org.cn/docs/spu/latest/en-US/reference/runtime_config#runtimeconfig HEU: https://www.secretflow.org.cn/docs/secretflow/latest/en-US/source/secretflow.device.device.device#secretflow.device.device.heu.HEU.init

Field	Type	Description
name	string	Name of the device.
type	string	Supported: SPU, HEU, TEEU.
parties	repeated string	Parties of device.
config	string	Specific config for the secret device.

SFClusterDesc.RayFedConfig#

Field	Type	Description
cross_silo_comm_backend	string	Indicates communication backend of RayFed. Accepted: ‘grpc’, ‘brpc_link’ Dafault is ‘grpc’

Enums#

data.proto#

Proto file: secretflow/protos/secretflow/spec/extend/data.proto

Messages#

DeviceObjectCollection#

Descibes public storage info for a collection of Device Objects.

Field	Type	Description
objs	repeated DeviceObjectCollection.DeviceObject	none
public_info	string	Any public information.

DeviceObjectCollection.DeviceObject#

Field	Type	Description
type	string	Supported: `spu \| pyu`
data_ref_idxs	repeated int32	Index of data_ref in the parent DistData message.