docs: add FAQ for Elasticsearch nanosecond timestamp log ordering (#1…#1253
docs: add FAQ for Elasticsearch nanosecond timestamp log ordering (#1…#1253jinggghui wants to merge 1 commit intojenkinsci:mainfrom
Conversation
| This clock adjustment is required to display in the right ascending order the log messages. | ||
| Note that distributed traces don't require such a clock adjustment because all spans are emitted from the Jenkins Controller. | ||
|
|
||
| ### Why are logs appearing out of order in Elasticsearch/Kibana? |
There was a problem hiding this comment.
This could affect only steps that take less than a millisecond to execute, so the title is misleading.
There was a problem hiding this comment.
BTW, I never saw this behaviour, it is really complex to have a step that takes less than a millisecond to execute. Can you provide a sample pipeline to replicate the error?
| **Fix 2 — Fix the sort order in Kibana:** Even with nanosecond storage, | ||
| Kibana may still show logs out of order because it sorts by milliseconds | ||
| internally. To fix this, set `event.sequence` as the tie-breaker field in | ||
| your Kibana Data View. The Jenkins OpenTelemetry plugin already includes | ||
| `event.sequence` in every log entry, so Kibana can use it to determine | ||
| the exact order logs were written. |
There was a problem hiding this comment.
Weird, IIRC the field to order is timestamp.us, which has nanosecond precision. Which version are you testing?
|
Great work on this documentation addition @jinggghui! I noticed @kuisathaverat raised two important concerns:
Also noticed Java Compiler is showing 1 new issue in the CI checks — |
Fixes #1151
Adds a new FAQ entry to
docs/build-logs.mdexplaining why logs appearout of order in Elasticsearch/Kibana when using the Elastic backend, and
how to fix it.
Testing done
This is a documentation-only change. No code was modified.
The content was verified against the plugin source code:
@timestampfield name confirmed fromElasticsearchFields.javalogs-apm*index pattern confirmed from plugin Kibana dashboard NDJSONfiles (8.14, 8.17, 9.0.x)
logs-apm.app@customcomponent template confirmed as the officialElastic-recommended approach for customizing APM Data Stream mappings
without breaking existing APM field mappings
Submitter checklist