Skip to content

Migrate the validator client to the Gloas builder API (Gloas builder API 5/5) #9807

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ethDreamer wants to merge 1 commit into from
Open

Migrate the validator client to the Gloas builder API (Gloas builder API 5/5) #9807

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ members = [
"testing/web3signer_tests",
"validator_client",
"validator_client/beacon_node_fallback",
"validator_client/builder_store",
"validator_client/doppelganger_service",
"validator_client/graffiti_file",
"validator_client/http_api",
Expand Down Expand Up @@ -120,6 +121,7 @@ bincode = "1"
bitvec = "1"
bls = { path = "crypto/bls" }
builder_client = { path = "beacon_node/builder_client" }
builder_store = { path = "validator_client/builder_store" }
builder_types = { path = "common/builder_types" }
byteorder = "1"
bytes = "1.11.1"
Expand Down
1 change: 1 addition & 0 deletions book/src/SUMMARY.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@
* [Redundancy](./advanced_redundancy.md)
* [Release Candidates](./advanced_release_candidates.md)
* [MEV](./advanced_builders.md)
* [Gloas Builder Configuration](./gloas_builder_config.md)
* [Late Block Re-orgs](./advanced_re-orgs.md)
* [Blobs](./advanced_blobs.md)
* [Command Line Reference (CLI)](./help_general.md)
Expand Down
90 changes: 90 additions & 0 deletions book/src/gloas_builder_config.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Builder Configuration

> This applies from the **Gloas** fork onwards. It configures how the validator client sources
> execution-payload bids from external builders under ePBS.

The validator client reads its external-builder settings from a YAML file named
`builder_definitions.yml` in the validator directory
(`<datadir>/validators/builder_definitions.yml`). The file holds two things:

- **A global bid policy** — `min_bid` and `builder_boost_factor`, applied to bids received over p2p
(gossip) and used as the default for any builder that does not set its own.
- **A list of builders** to request bids from directly, each with optional per-builder overrides of
the global policy.

## Example

```yaml
# Global bid policy: applies to p2p (gossip) bids, and is the default for any
# builder below that omits the corresponding field.
min_bid: 0 # gwei — bids below this rank last; one wins only if nothing else is viable
builder_boost_factor: 100 # percent — 100 = neutral, >100 favors builders, 0 = prefer local

builders:
# Minimal builder — inherits the global policy.
- enabled: true
url: "https://builder-a.example.com"
max_execution_payment: 1000000000 # gwei — cap on the trusted execution payment

# Builder overriding the globals and pinning the expected builder key.
- enabled: true
url: "https://builder-b.example.com"
max_execution_payment: 1000000000
min_bid: 500000000 # override the global for this builder
builder_boost_factor: 120 # override the global for this builder
builder_pubkeys: # optional — reject a bid not signed by one of these keys
- "0xa1b2c3d4..."
# auth_data: "0x68747470..." # optional — defaults to the UTF-8 bytes of `url`
```

> **Comments are not preserved.** The validator client rewrites this file when builders are added or
> removed (for example via the keymanager API), which strips YAML comments. Keep an annotated copy
> elsewhere if you rely on inline notes.

## Fields

### Top level (global bid policy)

| Field | Required | Default | Meaning |
| ------- | ---------- | --------- | --------- |
| `min_bid` | no | `0` | Minimum total payment, in gwei, for a p2p bid. A bid below the floor is ranked behind any floor-clearing candidate (including the local block) and only wins when nothing else is viable. Also the default `min_bid` for any builder that omits it. |
| `builder_boost_factor` | no | `100` | Percentage multiplier applied to p2p bids when comparing against the local block. Also the default for any builder that omits it. |
| `builders` | no | `[]` | The list of builders to request bids from directly. |

### Per builder (each entry under `builders`)

| Field | Required | Default | Meaning |
| ------- | ---------- | --------- | --------- |
| `enabled` | **yes** | — | Whether this builder is used. Disabled builders are ignored. |
| `url` | **yes** | — | The builder's `http`/`https` URL. Bids are requested from here at block-production time. |
| `max_execution_payment` | **yes** | — | Cap, in gwei, on the *trusted* execution payment accepted from this builder. |
| `min_bid` | no | *(global)* | Override the global minimum bid for this builder. |
| `builder_boost_factor` | no | *(global)* | Override the global boost factor for this builder. |
| `builder_pubkeys` | no | *(empty)* | The builder's BLS public keys, hex-encoded. If non-empty, a returned bid **not** signed by one of them is rejected. |
| `auth_data` | no | *(UTF-8 of `url`)* | Opaque authentication data, hex-encoded, agreed with the builder out of band. Signed into the request. Must be non-empty when set. Defaults to the UTF-8 bytes of `url`. |

All byte fields (`builder_pubkeys` entries, `auth_data`) are `0x`-prefixed hex strings. All payment values
(`min_bid`, `max_execution_payment`) are in gwei.

## How bids are selected

At block-production time the validator client requests a bid from each enabled builder with a `url`,
and also considers bids seen over p2p. For each candidate bid:

- **`min_bid`** — a bid whose total value is below the applicable `min_bid` is ranked behind any
floor-clearing candidate (including the local block) rather than dropped, so it wins only when
nothing else is viable (e.g. the local build failed). Direct builders use their own (or the
inherited global) value; p2p bids use the global value.
- **`builder_boost_factor`** — the surviving bid's value is scaled by its boost factor
(`boost × value ÷ 100`) before being compared against the locally-built block. A factor below
`100` favors the local block; above `100` favors the builder; `0` always prefers local;
`2^64 − 1` strongly favors the builder. The factor is a multiplier, not an absolute override, so a
zero-value bid still ranks `0` and loses to any non-zero local block.
- **`max_execution_payment`** — bounds how much of a builder's (off-chain) execution payment counts
toward its bid value. This applies only to direct builders; p2p bids carry no trusted execution
payment.
- **`builder_pubkeys`** — for a direct builder, if non-empty, the returned bid must be signed by
one of these keys or it is discarded.

The highest-value bid after these rules wins. Per-builder `min_bid`/`builder_boost_factor` apply
only to bids requested directly by URL; p2p bids are governed by the global values.
182 changes: 0 additions & 182 deletions common/eth2/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3048,188 +3048,6 @@ impl BeaconNodeHttpClient {
opt_response.ok_or(Error::StatusCode(StatusCode::NOT_FOUND))
}

// The legacy `GET v4/validator/blocks/{slot}` client methods below are kept alongside the new
// POST variants until the validator client migrates to POST (later in this PR stack), at which
// point they are removed.

/// `GET v4/validator/blocks/{slot}`
pub async fn get_validator_blocks_v4<E: EthSpec>(
&self,
slot: Slot,
randao_reveal: &SignatureBytes,
graffiti: Option<&Graffiti>,
include_payload: bool,
builder_booster_factor: Option<u64>,
graffiti_policy: Option<GraffitiPolicy>,
) -> Result<(ProduceBlockV4Response<E>, ProduceBlockV4Metadata), Error> {
self.get_validator_blocks_v4_modular(
slot,
randao_reveal,
graffiti,
SkipRandaoVerification::No,
include_payload,
builder_booster_factor,
graffiti_policy,
)
.await
}

/// `GET v4/validator/blocks/{slot}`
///
/// Returns either a bare block or the full [`BlockAndEnvelope`] (block + execution payload
/// envelope + blobs + KZG proofs) depending on the `Eth-Execution-Payload-Included` response
/// header. Note that a builder bid yields a bare block even when `include_payload=true`.
#[allow(clippy::too_many_arguments)]
pub async fn get_validator_blocks_v4_modular<E: EthSpec>(
&self,
slot: Slot,
randao_reveal: &SignatureBytes,
graffiti: Option<&Graffiti>,
skip_randao_verification: SkipRandaoVerification,
include_payload: bool,
builder_booster_factor: Option<u64>,
graffiti_policy: Option<GraffitiPolicy>,
) -> Result<(ProduceBlockV4Response<E>, ProduceBlockV4Metadata), Error> {
let mut path = self
.post_validator_blocks_v4_path(
slot,
randao_reveal,
graffiti,
skip_randao_verification,
include_payload,
graffiti_policy,
)
.await?;

if let Some(builder_booster_factor) = builder_booster_factor {
path.query_pairs_mut()
.append_pair("builder_boost_factor", &builder_booster_factor.to_string());
}

let opt_result = self
.get_response_with_response_headers(
path,
Accept::Json,
self.timeouts.get_validator_block,
|response, headers| async move {
let metadata = ProduceBlockV4Metadata::try_from(&headers)
.map_err(Error::InvalidHeaders)?;
let block_response = if metadata.execution_payload_included {
ProduceBlockV4Response::BlockAndEnvelope(
response
.json::<ForkVersionedResponse<
BlockAndEnvelope<E>,
ProduceBlockV4Metadata,
>>()
.await?
.data,
)
} else {
ProduceBlockV4Response::BlockOnly(
response
.json::<ForkVersionedResponse<
BeaconBlock<E>,
ProduceBlockV4Metadata,
>>()
.await?
.data,
)
};
Ok((block_response, metadata))
},
)
.await?;

opt_result.ok_or(Error::StatusCode(StatusCode::NOT_FOUND))
}

/// `GET v4/validator/blocks/{slot}` in ssz format
pub async fn get_validator_blocks_v4_ssz<E: EthSpec>(
&self,
slot: Slot,
randao_reveal: &SignatureBytes,
graffiti: Option<&Graffiti>,
include_payload: bool,
builder_booster_factor: Option<u64>,
graffiti_policy: Option<GraffitiPolicy>,
) -> Result<(ProduceBlockV4Response<E>, ProduceBlockV4Metadata), Error> {
self.get_validator_blocks_v4_modular_ssz::<E>(
slot,
randao_reveal,
graffiti,
SkipRandaoVerification::No,
include_payload,
builder_booster_factor,
graffiti_policy,
)
.await
}

/// `GET v4/validator/blocks/{slot}` in ssz format
///
/// See [`Self::get_validator_blocks_v4_modular`] for the response semantics.
#[allow(clippy::too_many_arguments)]
pub async fn get_validator_blocks_v4_modular_ssz<E: EthSpec>(
&self,
slot: Slot,
randao_reveal: &SignatureBytes,
graffiti: Option<&Graffiti>,
skip_randao_verification: SkipRandaoVerification,
include_payload: bool,
builder_booster_factor: Option<u64>,
graffiti_policy: Option<GraffitiPolicy>,
) -> Result<(ProduceBlockV4Response<E>, ProduceBlockV4Metadata), Error> {
let mut path = self
.post_validator_blocks_v4_path(
slot,
randao_reveal,
graffiti,
skip_randao_verification,
include_payload,
graffiti_policy,
)
.await?;

if let Some(builder_booster_factor) = builder_booster_factor {
path.query_pairs_mut()
.append_pair("builder_boost_factor", &builder_booster_factor.to_string());
}

let opt_response = self
.get_response_with_response_headers(
path,
Accept::Ssz,
self.timeouts.get_validator_block,
|response, headers| async move {
let metadata = ProduceBlockV4Metadata::try_from(&headers)
.map_err(Error::InvalidHeaders)?;
let response_bytes = response.bytes().await?;
let block_response = if metadata.execution_payload_included {
ProduceBlockV4Response::BlockAndEnvelope(
BlockAndEnvelope::from_ssz_bytes_for_fork(
&response_bytes,
metadata.consensus_version,
)
.map_err(Error::InvalidSsz)?,
)
} else {
ProduceBlockV4Response::BlockOnly(
BeaconBlock::from_ssz_bytes_for_fork(
&response_bytes,
metadata.consensus_version,
)
.map_err(Error::InvalidSsz)?,
)
};

Ok((block_response, metadata))
},
)
.await?;

opt_response.ok_or(Error::StatusCode(StatusCode::NOT_FOUND))
}

/// `GET v1/validator/execution_payload_envelopes/{slot}/{beacon_block_root}`
pub async fn get_validator_execution_payload_envelopes<E: EthSpec>(
&self,
Expand Down
Loading
Loading