Repository navigation
Loki output: set tenant header based on k8s label instead of static value #1145
Description
Describe the solution you'd like
I want to define 1 default Loki ClusterOutput but not with a static tenant.
With this, tenants don't get to see the config (it's cluster level) and for 100 tenant's I don't need then to create 100 namespaced output's with the same loki endpoint and auth config with is unsecure and harder to maintain if something changes.
e.g.:
apiVersion: logging.banzaicloud.io/v1beta1
kind: ClusterOutput
metadata:
name: default
spec:
loki:
url: http://loki:3100
tenant_from_label: "capsule.clastix.io/tenant"
configure_kubernetes_labels: true
buffer: ...
Additional context
We create multi tenant k8s clusters and use capsule to create Tenant's in the cluster.
It's not relevant to know how Capsule works internally, only that namespaces and pods have a certain label with their tenant id.
For logging, I need to extract this label to be forwarded as the tenant header (X-Scope-OrgId) to Loki.
I see in the code that something like this is available for kafka (headers_from_record in the crds):
kafka:
properties:
ack_timeout:
type: integer
...
headers_from_record:
additionalProperties:
type: string
type: object
More important: Fluentd also supports this:
https://docs.fluentd.org/output/http#headers_from_placeholders
For non-loki users it might be interesting to also add this feature (with a more generic name: headers_from_record) to the http output
Activity
atamgp commentedon Dec 19, 2022
Update:
It seems logging-framework uses the loki output from fluentd.
fluentd does not yet support tenant_id_key which fluentbit does support:
https://docs.fluentbit.io/manual/v/1.8/pipeline/outputs/loki
I opened a ticket with fluend to also support this, then the logging-framework needs to pass through only?
fluent/fluentd#3992
aslafy-z commentedon Dec 19, 2022
I believe this is already supported by the fluentd loki plugin, see https://grafana.com/docs/loki/latest/clients/fluentd/#tenant.
Also, the associated logging-operator output field is already passed through to the fluentd configuration: https://github.com/banzaicloud/logging-operator/blob/42402497f2116f014f14ba867fdce13aba0bf171/pkg/sdk/logging/model/output/loki.go#L74
stale commentedon Apr 12, 2023
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!
pepov commentedon Apr 12, 2023
thanks @aslafy-z for clarifying this! @fekete-robert is there a good place in the docs to add this?
fekete-robert commentedon Apr 18, 2023
I think we can do two things:
pepov commentedon Apr 18, 2023
I would opt for the separate page, but first someone should actually try this out whether it works correctly or not. I closed this prematurely, sorry.
stale commentedon Jun 17, 2023
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!
stale commentedon Aug 25, 2023
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!