Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 0 additions & 17 deletions .github/linters/.eslintrc.yml

This file was deleted.

18 changes: 18 additions & 0 deletions .github/linters/.markdownlint.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# MD013/line-length - Line length
MD013:
line_length: 120
heading_line_length: 120
code_block_line_length: 120
code_blocks: false
tables: false
headings: true
headers: true
strict: false
stern: false
# MD024/no-duplicate-header - Multiple headers with the same content
MD024:
allow_different_nesting: true
# MD036/no-emphasis-as-heading - Emphasis used instead of a heading
MD036: false
# MD041/first-line-heading - First line in a file should be a top-level heading
MD041: false
2 changes: 2 additions & 0 deletions .github/super-linter.env
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
# It must contain strings like:
# name=value

MARKDOWN_CONFIG_FILE=.markdownlint.yaml

GITLEAKS_LOG_LEVEL=warn
VALIDATE_BASH_EXEC=false
VALIDATE_JAVASCRIPT_PRETTIER=false
Expand Down
13 changes: 13 additions & 0 deletions .github/workflows/check-markdown.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
name: Check Markdown links
on: push

jobs:
markdown-link-check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: gaurav-nelson/github-action-markdown-link-check@v1
with:
use-quiet-mode: 'yes'
use-verbose-mode: 'yes'
20 changes: 20 additions & 0 deletions .github/workflows/pr-conventional-commits.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---

name: Conventional Commits PR Check

on:
pull_request:
types:
- opened
- edited
- synchronize

permissions:
pull-requests: read
jobs:
build:
name: Conventional Commits
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: webiny/action-conventional-commits@v1.3.0
21 changes: 21 additions & 0 deletions .github/workflows/pr-lint-title.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
---

name: "Lint PR Title"

on:
pull_request:
types:
- opened
- edited

permissions:
pull-requests: read

jobs:
main:
name: Validate PR title
runs-on: ubuntu-latest
steps:
- uses: amannn/action-semantic-pull-request@v5
env:
GITHUB_TOKEN: ${{ github.token }}
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@
.idea/
*.iml

# vim
*.swp

# Python
venv/
.mypy_cache/
3 changes: 1 addition & 2 deletions .gitleaksignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
/tmp/lint/docs/installation.md:private-key:660
/github/workspace/docs/installation.md:private-key:660
/github/workspace/docs/configuration.md:curl-auth-header:191
/github/workspace/api/v1alpha1/loggingservice_types.go:generic-api-key:628
/github/workspace/api/v1alpha1/loggingservice_types.go:generic-api-key:631
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ The main features are as follows:

All these documents provided to customer:

* [Architecture](/docs/architecture.md)
* [Installation](/docs/installation.md)
* [Maintenance](/docs/maintenance.md)
* [Troubleshooting](/docs/troubleshooting.md)
* [Architecture](docs/architecture.md)
* [Installation](docs/installation.md)
* [Maintenance](docs/maintenance.md)
* [Troubleshooting](docs/troubleshooting.md)
4 changes: 2 additions & 2 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ This section describes the Logging stack deployment in the on-premise Kubernetes
Non-HA deployment supports for both clients: FluentD and FluentBit. Each client sends processed messages directly in
Graylog.

![Non-HA](/docs/images/architecture/cloud-fluentbit.png)
![Non-HA](../docs/images/architecture/cloud-fluentbit.png)

### HA deployment scheme

Expand All @@ -170,7 +170,7 @@ Other components can't be deployed in more than 1 replica (or it has no sense):
* MongoDB - currently deploying as a sidecar of Graylog so can't be run in more than 1 replica
* Cloud Event Reader - stateless service and can be run in N replicas, but it has no make sense

![Non-HA](/docs/images/architecture/cloud-fluentbit-ha.png)
![Non-HA](../docs/images/architecture/cloud-fluentbit-ha.png)

### DR deployment scheme

Expand Down
32 changes: 16 additions & 16 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ To create a new stream:
3. Specify the required parameters
4. Click **Save**

![Create Stream](/docs/images/graylog/create-streame.png)
![Create Stream](../docs/images/graylog/create-streame.png)

5. Click **Manage Rules** for your stream
6. For example, you can upload one message
Expand Down Expand Up @@ -141,7 +141,7 @@ You can save search criteria as a named query for quick usage in the future.
After the search criteria are set, click **Save**, enter a name for the current query, and save it.
The saved search is available by the **Load** button for quick access.

![Saving custom search](/docs/images/graylog/saved-search.png)
![Saving custom search](../docs/images/graylog/saved-search.png)

Custom saved searches can be added automatically after the Graylog deployment through REST API.

Expand Down Expand Up @@ -189,7 +189,7 @@ Curl example:
curl --location --request POST 'https://GRAYLOG_HOST/api/api/views/search' \
--header 'Content-Type: application/json' \
--header 'X-Requested-By: cli' \
--header 'Authorization: Basic YWRtaW46YWRtaW4=' \
--header 'Authorization: Basic <credentials in base64>' \
-k \
--data '{
"id": "5f33c6369de1b46a5aa085c3",
Expand Down Expand Up @@ -288,7 +288,7 @@ To view and edit the policies:

The configuration for the rotation policy is shown in the following image.

![Configure Rotation Policy](/docs/images/graylog/configure-rotation-policy.png)
![Configure Rotation Policy](../docs/images/graylog/configure-rotation-policy.png)

## Exporting Logs

Expand Down Expand Up @@ -316,7 +316,7 @@ By default, there are next user roles available:
* Operator - The Operator has ReadOnly rights for all streams on the system except of 'Audit logs'.
* AuditViewer - The AuditViewer has ReadOnly rights for all streams on the system.

![Default Roles](/docs/images/graylog/default-roles.png)
![Default Roles](../docs/images/graylog/default-roles.png)

You can also create new user roles.

Expand All @@ -329,7 +329,7 @@ To create a new user role:

A role is created as shown in the following image.

![Create Roles](/docs/images/graylog/create-roles.png)
![Create Roles](../docs/images/graylog/create-roles.png)

### Users

Expand All @@ -341,7 +341,7 @@ By default, there are the following users:

The default users are shown in the following image.

![Default Users](/docs/images/graylog/default-users.png)
![Default Users](../docs/images/graylog/default-users.png)

You can create a new user in the system.

Expand All @@ -354,7 +354,7 @@ To create a new user:

A user is created as shown in the following image.

![Create Users](/docs/images/graylog/create-users.png)
![Create Users](../docs/images/graylog/create-users.png)

For more information about Graylog search query language, refer to
[Export Results as CSV](https://go2docs.graylog.org/5-2/interacting_with_your_log_data/export_results_as_csv.html)
Expand Down Expand Up @@ -397,7 +397,7 @@ It is recommended to write audit logs that combine `log_id` with the value `audi
Generally, more complicated and tricky criteria for audit logs can take place, depending on your application.
An example of using `log_id` with the value `audit` is given in the following image.

![Stream Rule](/docs/images/graylog/stream-rule.png)
![Stream Rule](../docs/images/graylog/stream-rule.png)

## System Logs

Expand Down Expand Up @@ -457,7 +457,7 @@ Logging-backuper is installed on the Graylog host machine by the deploy-logging-

To run the backup process:

1. Navigate to `GRAYLOG_HOST` by ssh.
1. Navigate to `GRAYLOG_HOST` by SSH.
1. Execute the command, `curl -XPOST localhost:8080/backup`.

### Running Restore Process
Expand All @@ -477,7 +477,7 @@ To change the password locally:
1. Navigate to the graylog folder in root home, `sudo -i` then `cd ~/graylog`
2. Use the `change-password.sh` script to change the password with the following parameters:

* `--user (-u)` - To specify the user name
* `--user (-u)` - To specify the username
* `--oldpass (-op)` - To specify the current password
* `--newpass (-np)` - To specify the new password
* `--confirmpass (-cp)` - To confirm the new password
Expand All @@ -490,12 +490,12 @@ To change the password remotely:
2. Use `change-password.sh` script to change the password with the following parameters:

* `--hosts (-h)` - To specify hosts
* `--user (-u)` - To specify the user name
* `--user (-u)` - To specify the username
* `--oldpass (-op)` - To specify the current password
* `--newpass (-np)` - To specify the new password
* `--confirmpass (-cp)` - To confirm the new password
* `--sshuser` - To specify ssh login
* `--sshkey` - To specify the path for ssh key of hosts
* `--sshuser` - To specify SSH login
* `--sshkey` - To specify the path for SSH key of hosts

3. Wait for `[INFO] Password was successfully changed` message

Expand All @@ -506,12 +506,12 @@ To change the password remotely:
Graylog supports custom extensions using plugins.
To configure plugins, navigate to **System/Configurations > Configurations** as shown in the following image.

![Path to Configuration Page](/docs/images/graylog/configuration-page.png)
![Path to Configuration Page](../docs/images/graylog/configuration-page.png)

### Graylog Obfuscation Plugin

The Graylog Obfuscation Plugin is necessary for anonymization of sensitive data in the logs.
For more information, see [Graylog Obfuscation Plugin](graylog_obfuscation_plugin.md).
For more information, see [Graylog Obfuscation Plugin](https://github.com/Netcracker/qubership-graylog-obfuscation-plugin/blob/main/README.md).

## Graylog Audit logs

Expand Down
24 changes: 12 additions & 12 deletions docs/integrations/aws-cloudwatch.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ This document describes how to integrate Logging agents and Graylog with AWS Clo
* [Configure AWS EKS (Kubernetes)](#configure-aws-eks-kubernetes)
* [Configure AWS RDS (PostgreSQL)](#configure-aws-rds-postgresql)
* [Configure AWS Keyspaces (Cassandra)](#configure-aws-keyspaces-cassandra)
* [Configure AWS ElasticSearch / OpenSearch](#configure-aws-elasticsearch--opensearch)
* [Configure AWS OpenSearch](#configure-aws-opensearch)
* [Configure AWS MSK (Kafka)](#configure-aws-msk-kafka)
* [Configure Amazon MQ (Rabbit MQ)](#configure-amazon-mq-rabbit-mq)
* [Configure AWS Kinesis](#configure-aws-kinesis)
Expand Down Expand Up @@ -79,7 +79,7 @@ To create User need:

1. Open IAM, select Users
2. Click `Add Users`
3. Enter user name and do not forgot set `Access key - Programmatic access` to generate access key
3. Enter username and do not forgot set `Access key - Programmatic access` to generate access key
4. Select early created group or permissions
5. Create user and save `Access Key` and `Secret Access Key`

Expand Down Expand Up @@ -195,7 +195,7 @@ To publish PostgreSQL logs to CloudWatch Logs using the AWS console:

**Note:** Please keep in mind that AWS can change UI and some fields can change names or positions.

![Configure RDS to send logs in CloudWatch](/docs/images/cloudwatch/rds-pg-logs.png)
![Configure RDS to send logs in CloudWatch](../images/cloudwatch/rds-pg-logs.png)

_Enable CloudWatch logs for RDS_

Expand All @@ -209,14 +209,14 @@ CloudTrail logs, a service that provides a record of actions taken by a user, ro
CloudTrail captures Data Definition Language (DDL) API calls for Amazon Keyspaces as events. The calls that are captured
include calls from the Amazon Keyspaces console and code calls to the Amazon Keyspaces API operations.

### Configure AWS ElasticSearch / OpenSearch
### Configure AWS OpenSearch

All information about available logs in CloudWatch you can read in official documentation
[Monitoring OpenSearch logs with Amazon CloudWatch Logs](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/createdomain-configure-slow-logs.html).

To enable log publishing to CloudWatch using the AWS console:

1. Open the `Amazon ElasticSearch` console at [https://console.aws.amazon.com/es](https://console.aws.amazon.com/es).
1. Open the `Amazon ElasticSearch` console at [https://console.aws.amazon.com/aos/home](https://console.aws.amazon.com/aos/home).
2. Select the domain you want to update.
3. On the `Logs` tab, select a log type and choose `Setup`.
4. Create a CloudWatch log group, or choose an existing one.
Expand Down Expand Up @@ -246,11 +246,11 @@ To enable log publishing to CloudWatch using the AWS console:

**Note:** Please keep in mind that AWS can change UI and some fields can change names or positions.

![ElasticSearch logs settings](/docs/images/cloudwatch/elasticsearch-logs.png)
![ElasticSearch logs settings](../images/cloudwatch/elasticsearch-logs.png)

_Logs configuration section for ElasticSeach / OpenSearch_

![ElasticSearch logs configuration diaglog](/docs/images/cloudwatch/elasticsearch-logs-creation-dialog.png)
![ElasticSearch logs configuration diaglog](../images/cloudwatch/elasticsearch-logs-creation-dialog.png)

_Logs configuration dialog for ElasticSeach / OpenSearch_

Expand All @@ -268,7 +268,7 @@ To publish Apache Kafka logs to CloudWatch Logs using the AWS console:

**Note:** Please keep in mind that AWS can change UI and some fields can change names or positions.

![Kafka logs settings](/docs/images/cloudwatch/kafka-logs.png)
![Kafka logs settings](../images/cloudwatch/kafka-logs.png)

_Logs configuration section for Apache Kafka_

Expand All @@ -286,7 +286,7 @@ To publish Apache MQ logs to CloudWatch Logs using the AWS console:

**Note:** Please keep in mind that AWS can change UI and some fields can change names or positions.

![RabbitMQ logs settings ](/docs/images/cloudwatch/rabbitmq-log.png)
![RabbitMQ logs settings ](../images/cloudwatch/rabbitmq-log.png)

_Logs configuration section for RabbitMQ_

Expand Down Expand Up @@ -374,21 +374,21 @@ You should now see Flow Logs being written into your Kinesis stream.

Also state of Kinesis streams you can see in AWS Console:

1. Open the `Amazon Kinesis` console at [https://console.aws.amazon.com/kinesis].
1. Open the `Amazon Kinesis` console at [https://console.aws.amazon.com/kinesis/home].
2. In the navigation pane, choose `Data Streams` then choose the one of configured streams.
3. In the opened page you can see some parameters of stream configuration and see metrics (In/Out data and so on).

## Configure Graylog with AWS Plugin

**Warning!** AWS Plugin has been deprecated in favor of the new
[AWS Kinesis/CloudWatch](http://docs.graylog.org/en/3.1/pages/integrations/inputs/aws_kinesis_cloudwatch_input.html#aws-kinesis-cloudwatch-input)
[AWS Kinesis/CloudWatch](https://go2docs.graylog.org/current/getting_in_log_data/aws_kinesis_cloudwatch_input.html)
input in
[graylog-integrations-plugin](https://github.com/Graylog2/graylog-plugin-integrations).

Now go into the Graylog Web Interface and start a new `AWS Kinesis/CloudWatch` input.
It will ask you for some simple parameters like the Kinesis Stream name you are writing your Flow Logs to.

![Graylog AWS Plugin](/docs/images/cloudwatch/aws-logs-plugin.png)
![Graylog AWS Plugin](../images/cloudwatch/aws-logs-plugin.png)

_Configuration in Graylog AWS plugin_

Expand Down
2 changes: 1 addition & 1 deletion docs/migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ fluentbit:
```

The other required or necessary parameters are listed in the
[Installation Guide](/docs/installation.md).
[Installation Guide](installation.md).

## HWE for HA FluentBit

Expand Down
Loading
Loading