Repository navigation
fix(tests): read DLQ entries through scalo - #324
Merged
Merged
Conversation
scalo 2.14.28 writes a dead letter as a DFE event, with the original under `_json` or under `_raw` with an optional `_raw_encoding`. It no longer writes a base64 `payload` key. The integration helpers read `payload` and found no entry. Every full-tier test that checks the file DLQ failed. - The helpers parse each line as `scalo::dlq::DlqEntry`, which restores the original bytes from either field. - The hand-written base64 decoder is gone because scalo's parser decodes `_raw_encoding: base64`.
|
Released in v1.18.68 -- https://github.com/hyperi-io/dfe-loader/releases/tag/v1.18.68 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The release run on main failed three full-tier
grpc_outagetests.scalo 2.14.28 (#323) writes a dead letter with the original under
_json, or_rawwith an optional_raw_encoding. There is no base64payloadkey any more. The readers indead_letter_reasonsandjson_depthstill decodedpayload. Every file DLQ read came back empty.a_table_changed_under_the_cached_schema_is_caught_and_nothing_is_acked_earlycounts an accepted id as early when it is in neither ClickHouse nor the DLQ. Ids 30-32 were dead-lettered (messages_dlq=3) but the reader missed them.The helpers parse each line as
scalo::dlq::DlqEntry. This restores the original from either field. The hand-written base64 decoder is gone. No loader code changes.Full tier on desktop-derek,
--run-ignored all --ignore-default-filter, filtered to the five DLQ-reading modules:json_depth,header_keys, threekafka_offset_floorand two moregrpc_outagetests.