[BUG] auto_service_type omitted from exported flow logs for pod_group and pod_cluster endpoints

[rosenlo]

Search before asking

I searched the issues and found no similar report.

DeepFlow Component

Server (ingester exporters)

What you expected to happen

auto_service_type_{0,1} should be present on every exported flow log, the way
auto_instance_type_{0,1} is. Instead it is omitted entirely whenever the
endpoint's auto_service resolves to a pod_group or a pod_cluster — while
auto_service_{0,1} right next to it is filled in normally.

Sampled from our Kafka flow-log export, auto_service_type_1 is absent from
8,049 of 14,911 L7 documents (54%). The absences are not random: they fall
exactly on auto_instance_type_1 = pod and = pod_node, and on every other
resource type the two fields carry the same string.

auto_instance_type_1	auto_service_type_1	docs
pod	absent	8,037
chost	chost	5,710
lb	lb	866
pod	pod_service	152
ip	ip	131
pod_node	absent	12
internet_ip	internet_ip	3

A single exported document looks like this — name present, type gone:

{
  "auto_instance_1": "some-pod-6c58585cf5-sg4s2",
  "auto_instance_type_1": "pod",
  "auto_service_1": "some-workload"
}

Cause

common.GetAutoService (server/ingester/common/common.go) does not return
TYPE_POD_GROUP for a pod_group. It returns the workload controller's own
subtype:

} else if podGroupID > 0 {
    return podGroupID, podGroupType     // VIF_DEVICE_TYPE_POD_GROUP_*, 130-135
} else if gpID > 0 {
    ...
} else if podClusterID > 0 {
    return podClusterID, PodClusterType // 103
}

deviceTypeStrings in
server/ingester/exporters/universal_tag/universal_tag.go only defines
0, 1, 5, 6, 9-16, 101, 102, 120, 255. DeviceType.String() indexes that
slice directly, so 103, 104 and 130-135 resolve to "", and an empty tag value
is omitted from the serialized output.

The name survives because it comes from a different lookup —
deviceMap[uint64(autoType)<<32|uint64(autoID)] — and those keys do include
130-135, since ch_device.go populates them through
RESOURCE_POD_GROUP_TYPE_MAP. Hence name present, type absent.

For reference, the ClickHouse dictionary flow_tag.node_type_map (generated
from tagrecorder's RESOURCE_TYPE_TO_NODE_TYPE) maps all six pod_group
subtypes to "pod_group", 103 to "pod_cluster" and 104 to
"custom_service" — so the querier side of the same data is unaffected. Only
the exporter's own table has the holes, even though the comment above it says
it mirrors that dictionary.

How to reproduce

1. Enable a flow-log exporter (Kafka or OTLP) with auto_service_type among
   the exported tags.
2. Send L7 traffic to a pod that is not fronted by a Kubernetes Service, so
   GetAutoService takes the podGroupID > 0 branch.
3. Inspect an exported document: auto_service_1 carries the workload name,
   auto_service_type_1 is absent. A pod_node destination reproduces it the
   same way through the podClusterID > 0 branch (type 103).

A pod that is fronted by a Service takes the podServiceID > 0 branch and
returns 11, which is in the table — those documents come out correct, which is
why the field looks intermittently present rather than plainly broken.

DeepFlow version

Observed on a deployment based on v7.0.8. The code path is unchanged on main
as of cef3448 — both GetAutoService's return values and
deviceTypeStrings — so it reproduces there as well.

Anything else

Every time, for every affected endpoint. It is not intermittent per-endpoint:
whether the tag appears is fully determined by which branch GetAutoService
takes.

Consumers downstream of the exporter cannot distinguish "DeepFlow resolved
nothing for this endpoint" (ip) from "resolved a workload" when the tag is
simply missing, which is how we ran into it.

Are you willing to submit a PR?

Yes I am willing to submit a PR! — sent as a follow-up to this issue.

Code of Conduct

I agree to follow this project's Code of Conduct.