diff --git a/administration/configuring-fluent-bit/classic-mode/configuration-file.md b/administration/configuring-fluent-bit/classic-mode/configuration-file.md index 2e188fa55..caa4406a2 100644 --- a/administration/configuring-fluent-bit/classic-mode/configuration-file.md +++ b/administration/configuring-fluent-bit/classic-mode/configuration-file.md @@ -57,7 +57,7 @@ The following is an example of a `SERVICE` section: Log_Level debug ``` -For scheduler and retry details, see [scheduling and retries](../../scheduling-and-retries.md#Scheduling-and-Retries). +For scheduler and retry details, see [scheduling and retries](../../scheduling-and-retries.md). ## Config input diff --git a/administration/configuring-fluent-bit/yaml/service-section.md b/administration/configuring-fluent-bit/yaml/service-section.md index 0dffdddcd..d62a3a54d 100644 --- a/administration/configuring-fluent-bit/yaml/service-section.md +++ b/administration/configuring-fluent-bit/yaml/service-section.md @@ -103,7 +103,7 @@ In Fluent Bit versions 5.1.0 and 5.1.1, setting `FLB_LOG_LEVEL` and `log_level` FIPS mode is available in Fluent Bit version 5.1 and greater. It requires OpenSSL 3.0 or greater with the FIPS provider installed on the host. -When `security.fips_mode` is enabled, Fluent Bit activates the OpenSSL FIPS provider during startup and verifies that a FIPS-approved algorithm can be fetched from it. If the provider isn't available or can't be activated, Fluent Bit logs the OpenSSL errors and exits instead of starting with non-compliant cryptography. You can set the same behavior from the command line with the [`--enable-fips`](../../configuring-fluent-bit.md#require-fips-mode-with---enable-fips) flag. +When `security.fips_mode` is enabled, Fluent Bit activates the OpenSSL FIPS provider during startup and verifies that a FIPS-approved algorithm can be fetched from it. If the provider isn't available or can't be activated, Fluent Bit logs the OpenSSL errors and exits instead of starting with non-compliant cryptography. You can set the same behavior from the command line with the [`--enable-fips`](../../configuring-fluent-bit.md#require-fips-mode-with-enable-fips) flag. FIPS mode can't be changed by [hot reload](../../../administration/hot-reload.md). If a reloaded configuration changes `security.fips_mode`, Fluent Bit halts the reload and keeps running with the previous configuration. @@ -176,7 +176,7 @@ The following storage-related keys can be set as children to the `storage` key: For storage and buffering details, see [Buffering](../../../pipeline/buffering.md) and [Backpressure](../../backpressure.md). -For scheduler and retry details, see [Scheduling and retries](../../scheduling-and-retries.md#Scheduling-and-Retries). +For scheduler and retry details, see [Scheduling and retries](../../scheduling-and-retries.md). ## Configuration example diff --git a/installation/upgrade-notes.md b/installation/upgrade-notes.md index 9cd10e326..3d3aba9ac 100644 --- a/installation/upgrade-notes.md +++ b/installation/upgrade-notes.md @@ -209,7 +209,7 @@ If you have existing queries based on the resource's `project_id,` update your q The migration from v1.4 to v1.5 is pretty straightforward. -- The `keepalive` configuration mode has been renamed to `net.keepalive`. Now, all Network I/O keepalive is enabled by default. To learn more about this and other associated configuration properties read the [Networking Administration](https://docs.fluentbit.io/manual/administration/networking#tcp-keepalive) section. - If you use the Elasticsearch output plugin, the default value of `type` [changed from `flb_type` to `_doc`](https://github.com/fluent/fluent-bit/commit/04ed3d8104ca8a2f491453777ae6e38e5377817e#diff-c9ae115d3acaceac5efb949edbb21196). Many versions of Elasticsearch tolerate this, but Elasticsearch v5.6 through v6.1 require a `type` without a leading underscore. See the [Elasticsearch output plugin documentation FAQ entry](https://docs.fluentbit.io/manual/pipeline/outputs/elasticsearch#faq-underscore) for more. +- The `keepalive` configuration mode has been renamed to `net.keepalive`. Now, all Network I/O keepalive is enabled by default. To learn more about this and other associated configuration properties read the [Networking Administration](https://docs.fluentbit.io/manual/administration/networking#connection-keepalive) section. - If you use the Elasticsearch output plugin, the default value of `type` [changed from `flb_type` to `_doc`](https://github.com/fluent/fluent-bit/commit/04ed3d8104ca8a2f491453777ae6e38e5377817e#diff-c9ae115d3acaceac5efb949edbb21196). Many versions of Elasticsearch tolerate this, but Elasticsearch v5.6 through v6.1 require a `type` without a leading underscore. See the [Elasticsearch output plugin documentation FAQ entry](https://docs.fluentbit.io/manual/pipeline/outputs/elasticsearch#mapping-type-names-cant-start-with-underscores) for more. ## Fluent Bit v1.4 diff --git a/pipeline/filters/kubernetes.md b/pipeline/filters/kubernetes.md index 7f84274af..90d5973ee 100644 --- a/pipeline/filters/kubernetes.md +++ b/pipeline/filters/kubernetes.md @@ -80,7 +80,7 @@ The plugin supports the following configuration parameters: | `tls.verify_hostname` | When enabled, turns on hostname validation for certificates. | `Off` | | `tls.vhost` | Set an optional TLS virtual host for the Kubernetes API server connection. | _none_ | | `use_journal` | When enabled, the filter reads logs in `Journald` format. | `Off` | -| `use_kubelet` | Optional feature flag to get metadata information from Kubelet instead of calling Kube Server API to enhance the log. This could mitigate the [Kube API heavy traffic issue for large cluster](kubernetes.md#optional-feature-using-kubelet-to-get-metadata). If used when any [Kubernetes Namespace Meta](#kubernetes-namespace-meta) fields are enabled, Kubelet will be used to fetch pod data, but namespace meta will still be fetched using the `kube_url` settings. In v5.1.3 or later, the pod name and namespace are matched exactly against the Kubelet pod list instead of by prefix. | `Off` | +| `use_kubelet` | Optional feature flag to get metadata information from Kubelet instead of calling Kube Server API to enhance the log. This could mitigate the [Kube API heavy traffic issue for large cluster](kubernetes.md#using-kubelet-to-get-metadata). If used when any [Kubernetes Namespace Meta](#kubernetes-namespace-meta) fields are enabled, Kubelet will be used to fetch pod data, but namespace meta will still be fetched using the `kube_url` settings. In v5.1.3 or later, the pod name and namespace are matched exactly against the Kubelet pod list instead of by prefix. | `Off` | | `use_pod_association` | Deprecated alias for `aws_use_pod_association`. Kept for backward compatibility with AWS Observability users. | `Off` | | `use_tag_for_meta` | When enabled, Kubernetes metadata (for example, `pod_name`, `container_name`, and `namespace_name`) will be extracted from the tag itself. Connection to Kubernetes API Server won't get established and API calls for metadata won't be made. See [Workflow of Tail and Kubernetes Filter](#workflow-of-tail-and-kubernetes-filter) and [Custom tag For enhanced filtering](#custom-tags-for-enhanced-filtering) to better understand metadata extraction from tags. | `Off` | diff --git a/pipeline/inputs/standard-input.md b/pipeline/inputs/standard-input.md index ec267ff28..1ab57df98 100644 --- a/pipeline/inputs/standard-input.md +++ b/pipeline/inputs/standard-input.md @@ -32,7 +32,7 @@ The plugin supports the following configuration parameters: If no parser is configured for the `stdin` plugin, it expects valid JSON input data in one of the following formats: - A JSON object with one or more key-value pairs: `{ "key": "value", "key2": "value2" }` -- A 2-element JSON array in [Fluent Bit Event](../../concepts/key-concepts.md#event-or-record) format, which can be: +- A 2-element JSON array in [Fluent Bit Event](../../concepts/key-concepts.md#events-or-records) format, which can be: - `[TIMESTAMP, { "key": "value" }]` where TIMESTAMP is a floating point value representing a timestamp in seconds. - From Fluent Bit v2.1.0, `[[TIMESTAMP, METADATA], { "key": "value" }]` where _`TIMESTAMP`_ has the same meaning as previous and _`METADATA`_ is a JSON object. diff --git a/pipeline/inputs/tail.md b/pipeline/inputs/tail.md index 0075cff77..37cd2ae83 100644 --- a/pipeline/inputs/tail.md +++ b/pipeline/inputs/tail.md @@ -47,7 +47,7 @@ The plugin supports the following configuration parameters: | `skip_empty_lines` | Skips empty lines in the log file from any further processing or output. | `false` | | `skip_long_lines` | When a monitored file reaches its buffer capacity due to a very long line (`buffer_max_size`), the default behavior is to stop monitoring that file. `skip_long_lines` alters that behavior and instructs Fluent Bit to skip long lines and continue processing other lines that fit into the buffer size. | `false` | | `static_batch_size` | Set the maximum number of bytes to process per iteration for the monitored static files (files that already exist upon Fluent Bit start). | `50M` | -| `tag` | Set a tag with `regexextract` fields that will be placed on lines read. For example, `kube....`. Tag expansion is supported: if the tag includes an asterisk (`*`), that asterisk will be replaced with the absolute path of the monitored file, with slashes replaced by dots. See [Workflow of Tail + Kubernetes Filter](../filters/kubernetes.md#workflow-of-tail--kubernetes-filter). | _none_ | +| `tag` | Set a tag with `regexextract` fields that will be placed on lines read. For example, `kube....`. Tag expansion is supported: if the tag includes an asterisk (`*`), that asterisk will be replaced with the absolute path of the monitored file, with slashes replaced by dots. See [Workflow of Tail + Kubernetes Filter](../filters/kubernetes.md#workflow-of-tail-and-kubernetes-filter). | _none_ | | `tag_regex` | Set a regular expression to extract fields from the filename. For example: `(?[a-z0-9](?:[-a-z0-9]*[a-z0-9])?(?:\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*)_(?[^_]+)_(?.+)-(?[a-z0-9]{64})\.log$`. | _none_ | | `thread.ring_buffer.capacity` | Number of slots in the ring buffer for data entries when running in [threaded](../../administration/multithreading.md) mode. Each slot can hold one data entry. | `1024` | | `thread.ring_buffer.window` | Percentage threshold (1-100) of the ring buffer capacity at which a flush is triggered when running in [threaded](../../administration/multithreading.md) mode. | `5` | diff --git a/pipeline/outputs/opensearch.md b/pipeline/outputs/opensearch.md index 865ae243c..dda89483e 100644 --- a/pipeline/outputs/opensearch.md +++ b/pipeline/outputs/opensearch.md @@ -21,7 +21,7 @@ This plugin supports the following parameters: | `aws_profile` | AWS profile name. | `default` | | `aws_region` | Specify the AWS region for Amazon OpenSearch Service. | _none_ | | `aws_role_arn` | AWS IAM Role to assume to put records to your Amazon cluster. | _none_ | -| `aws_service_name` | Service name to be used in AWS Sigv4 signature. For integration with Amazon OpenSearch Serverless, set to `aoss`. See the [FAQ](opensearch.md#faq) section on Amazon OpenSearch Serverless for more information. | `es` | +| `aws_service_name` | Service name to be used in AWS Sigv4 signature. For integration with Amazon OpenSearch Serverless, set to `aoss`. See the [FAQ](#fluent-bit-and-amazon-opensearch-serverless) section on Amazon OpenSearch Serverless for more information. | `es` | | `aws_sts_endpoint` | Specify the custom STS endpoint to be used with STS API for Amazon OpenSearch Service. | _none_ | | `buffer_size` | Specify the buffer size used to read the response from the OpenSearch HTTP service. Use for debugging purposes where it's required to read full responses. The response size grows depending of the number of records inserted. Set this value to `False` to set an unlimited amount of memory. Otherwise set the value according to the [Unit Size](../../administration/configuring-fluent-bit.md#unit-sizes) specification. | `512k` | | `compress` | Set payload compression mechanism. Allowed value: `gzip`. Enabling compression reduces network bandwidth usage but might increase CPU usage. | _none_ | @@ -54,7 +54,7 @@ This plugin supports the following parameters: | `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` | | `write_operation` | Operation to use to write in bulk requests. | `create` | -The parameters `index` and `type` can be confusing if you are new to OpenSearch. If you have used a common relational database before, they can be compared to the `database` and `table` concepts. Also see [the FAQ](opensearch.md#faq). +The parameters `index` and `type` can be confusing if you are new to OpenSearch. If you have used a common relational database before, they can be compared to the `database` and `table` concepts. Also see [the FAQ](#frequently-asked-questions). {% hint style="info" %} diff --git a/pipeline/outputs/stackdriver.md b/pipeline/outputs/stackdriver.md index d8c2d134d..ef34cf8a5 100644 --- a/pipeline/outputs/stackdriver.md +++ b/pipeline/outputs/stackdriver.md @@ -27,7 +27,7 @@ This plugin uses the following configuration parameters. For more details about | `metadata_server` | Prefix for a metadata server. | Value of environment variable `$METADATA_SERVER`, or `http://metadata.google.internal` if unset. | | `namespace` | A namespace identifier, such as a cluster name or environment. If the resource type is either `generic_node` or `generic_task`, this field is required. | _none_ | | `node_id` | A unique identifier for the node within the namespace, such as a hostname or IP address. If the resource type is `generic_node`, this field is required. | _none_ | -| `project_id_key` | The value of this field is used by the Stackdriver output plugin to find the GCP `projectID` from `jsonPayload` and then extract the value of it to set the `PROJECT_ID` within LogEntry `logName`, which controls the GCP project that should receive these logs. | `logging.googleapis.com/projectId`. See [Stackdriver Special Fields](https://github.com/fluent/fluent-bit-docs/blob/master/pipeline/outputs/stackdriver_special_fields.md#log-entry-fields) for more info. | +| `project_id_key` | The value of this field is used by the Stackdriver output plugin to find the GCP `projectID` from `jsonPayload` and then extract the value of it to set the `PROJECT_ID` within LogEntry `logName`, which controls the GCP project that should receive these logs. | `logging.googleapis.com/projectId`. See [Stackdriver Special Fields](https://github.com/fluent/fluent-bit-docs/blob/master/pipeline/outputs/stackdriver_special_fields.md#logentry-fields) for more info. | | `resource` | Set resource type of data. Supported resource types: `k8s_container`, `k8s_node`, `k8s_pod`, `global`, `generic_node`, `generic_task`, and `gce_instance`. | `global` | | `resource_labels` | An optional list of comma-separated strings specifying resource label plain text assignments (`new=value`) and mappings from an original field in the log entry to a destination field (`destination=$original`). Nested fields and environment variables are also supported using the [record accessor syntax](https://docs.fluentbit.io/manual/administration/configuring-fluent-bit/classic-mode/record-accessor). If configured, all resource labels will be assigned using this API only, with the exception of `project_id`. See [Resource Labels](#resource-labels) for more details. | _none_ | | `service_account_email` | Account email associated with the service. Only available if no credentials file has been provided. | Value of environment variable `$SERVICE_ACCOUNT_EMAIL` | @@ -261,7 +261,7 @@ By default, the plugin will write to the following log name: /projects//logs/ ``` -You might be in a scenario where being more specific about the log name is important (for example, [integration with Log Router rules](https://docs.cloud.google.com/logging/docs/routing/overview) or [controlling cardinality of log based metrics](https://cloud.google.com/logging/docs/logs-based-metrics/troubleshooting#too-many-time-series)). You can control the log name directly on a per-log basis by using the [`logging.googleapis.com/logName` special field](https://github.com/fluent/fluent-bit-docs/blob/master/pipeline/outputs/stackdriver_special_fields.md#log-entry-fields). You can configure a `log_name_key` if you'd like to use something different from `logging.googleapis.com/logName`. For example, if the `log_name_key` is set to `mylognamefield` will extract the log name from `mylognamefield` in the log. +You might be in a scenario where being more specific about the log name is important (for example, [integration with Log Router rules](https://docs.cloud.google.com/logging/docs/routing/overview) or [controlling cardinality of log based metrics](https://cloud.google.com/logging/docs/logs-based-metrics/troubleshooting#too-many-time-series)). You can control the log name directly on a per-log basis by using the [`logging.googleapis.com/logName` special field](https://github.com/fluent/fluent-bit-docs/blob/master/pipeline/outputs/stackdriver_special_fields.md#logentry-fields). You can configure a `log_name_key` if you'd like to use something different from `logging.googleapis.com/logName`. For example, if the `log_name_key` is set to `mylognamefield` will extract the log name from `mylognamefield` in the log. ## Troubleshooting diff --git a/pipeline/parsers/regular-expression.md b/pipeline/parsers/regular-expression.md index 1098dce90..60b6c840b 100644 --- a/pipeline/parsers/regular-expression.md +++ b/pipeline/parsers/regular-expression.md @@ -2,7 +2,7 @@ Use the _regular expression_ parser format to create custom parsers with Ruby regular expressions. These regular expressions use named capture to define which content belongs to which key name. -Use [Tail multiline](../inputs/tail.md#multiline) when you need to support regular expressions across multiple lines from a `tail`. The Tail input plugin treats each line as a separate entity. +Use [Tail multiline](../inputs/tail.md#multiline-support) when you need to support regular expressions across multiple lines from a `tail`. The Tail input plugin treats each line as a separate entity. {% hint style="warning" %}