Grafana Alloy

Grafana Alloy is the telemetry collector from Grafana Labs. It reads logs, metrics, traces and profiles where they are produced, changes them in a pipeline, and sends them to a backend such as Loki, Prometheus, Tempo or Pyroscope. One agent can do the work that used to take Promtail for logs, a Prometheus agent for metrics and an OpenTelemetry Collector for traces. The official docs own the component reference.

What it is

Alloy is an OpenTelemetry Collector distribution with built-in Prometheus pipelines. It ships the upstream otelcol.* receivers, processors and exporters, plus Grafana's own prometheus.*, loki.*, pyroscope.* and discovery.* components. The licence is Apache 2.0.

The current stable line is v1.20 (v1.20.1, released 2026-09-28). A v1.21.0 release candidate shipped on 2026-10-14 and is marked as not for production. The official Helm chart is grafana/alloy (chart 1.13.1 carries app version v1.20.1).

Alloy is the successor to two retired Grafana agents:

• Grafana Agent (Static, Flow and Operator) reached end of life on 1 November 2025. Alloy uses the same components and concepts that Grafana Agent Flow introduced.
• Promtail, the original Loki log shipper, reached end of life on 2 March 2026. alloy convert --source-format=promtail translates an existing Promtail config.

It sits in the Grafana LGTM family next to Loki (logs), Grafana (dashboards), Tempo (traces), Mimir (metrics) and Pyroscope (profiles), but nothing ties it to that family: the otelcol.exporter.* and prometheus.remote_write components send to any compatible backend.

How it works

An Alloy config is a set of components. Each component does one job and exports values that other components read. A discovery.kubernetes block lists pods, a discovery.relabel block turns pod metadata into labels, a loki.source.* block reads lines, a loki.process block parses or labels them, and loki.write batches and sends them. Alloy builds a dependency graph from those references and keeps it running: when discovery finds a new pod, the source component starts a new tailer without a restart.

The flow below is the common Kubernetes log case: one Alloy pod per node reads the logs of the containers on that node and pushes them to Loki.

Two things follow from this design. First, Alloy holds no durable queue by default: the loki.write write-ahead log is still experimental, so an outage longer than the retry window loses lines. Second, every label added in the pipeline becomes a Loki stream label, so the relabel step decides index size and query cost.

Two ways to read pod logs

Aspect	loki.source.file + node log mount	loki.source.kubernetes
Reads from	Log files on the node	The Kubernetes API (kubelet log stream)
Deployment	DaemonSet with the host log directory mounted	Can be one Deployment for the whole cluster
Privileges	Needs host filesystem access	No privileged container, no host mounts
Cost	Cheapest	More network traffic and kubelet CPU
Sharing work	Each pod reads its own node	clustering block shards targets across Alloy pods

The configuration syntax

The syntax is block-based with expressions. A block has a component name and a label (loki.write "default" { … }), and other blocks reference its exports (loki.write.default.receiver). Functions such as sys.env("HOSTNAME") and coalesce() fill values at runtime. A POST to /-/reload or a SIGHUP reloads the file. If the new file is invalid, Alloy keeps running the last valid graph.

When to use it

Reach for it when:

• You run Loki and need a log shipper. Promtail is end of life, and Alloy is the supported path.
• You want one agent for several signals: logs to Loki, metrics to Prometheus remote-write, traces over OTLP to Tempo.
• You need to edit telemetry in flight: drop noisy lines, parse JSON, redact fields, add labels before data reaches storage.
• You are migrating from Grafana Agent, Promtail or a Prometheus agent and want a converter (alloy convert) to start from.
• You want to see the pipeline while debugging: the UI shows each component, its health and the data passing through it.

Skip it when:

• You want a vendor-neutral, upstream-only collector. The upstream OpenTelemetry Collector (contrib build) has no Grafana-specific components.
• Your logs go to Elasticsearch, OpenSearch or a SIEM and never to Loki. Fluent Bit or Vector fit that path with less config.
• Metrics are already handled by Prometheus in agent mode and you need nothing more than metrics. Adding Alloy would add a second scraper.

At a glance

Core concepts

• Component: one named block that does one job: discover, scrape, receive, process or write.
• Exports and references: a component's outputs (targets, receivers) that other components read. The references form the pipeline graph.
• Stability level: each component is generally available, public preview or experimental. --stability.level (default generally-available) sets the minimum allowed.
• Clustering: Alloy pods gossip with each other and split targets with consistent hashing. It is turned on per component, not globally.
• Modules: reusable pieces of config that are imported from a file, a Git repo or a string.

Key facts

Item	Value
Licence	Apache 2.0
Stable line	v1.20 (v1.20.1, 2026-09-28)
Helm chart	grafana/alloy from https://grafana.github.io/helm-charts
Default chart controller	DaemonSet (Deployment and StatefulSet also supported)
HTTP server and UI	Port 12345 (127.0.0.1:12345 for a bare binary)
loki.write retry defaults	500 ms to 5 m backoff, 10 retries, 429s retried
loki.write batching	1 MiB or 1 s, whichever comes first
Predecessors	Grafana Agent (EOL 2025-11-01), Promtail (EOL 2026-03-02)

How to integrate

Helm install as a DaemonSet

# values.yaml for the grafana/alloy chart
controller:
  type: daemonset
alloy:
  mounts:
    varlog: true          # host log directory, needed by loki.source.file
  extraEnv:
    - name: CLUSTER
      value: prod
  configMap:
    content: |
      discovery.kubernetes "pods" {
        role = "pod"
        selectors {
          role  = "pod"
          field = "spec.nodeName=" + coalesce(sys.env("HOSTNAME"), constants.hostname)
        }
      }

      discovery.relabel "pods" {
        targets = discovery.kubernetes.pods.targets
        rule {
          source_labels = ["__meta_kubernetes_namespace"]
          target_label  = "namespace"
        }
        rule {
          source_labels = ["__meta_kubernetes_pod_name"]
          target_label  = "pod"
        }
        rule {
          source_labels = ["__meta_kubernetes_pod_container_name"]
          target_label  = "container"
        }
      }

      loki.source.kubernetes "pods" {
        targets    = discovery.relabel.pods.output
        forward_to = [loki.process.pods.receiver]
      }

      loki.process "pods" {
        stage.static_labels {
          values = { cluster = sys.env("CLUSTER") }
        }
        forward_to = [loki.write.default.receiver]
      }

      loki.write "default" {
        endpoint {
          url = "http://loki.example.com:3100/loki/api/v1/push"
        }
      }

helm repo add grafana https://grafana.github.io/helm-charts
helm repo update
helm upgrade --install alloy grafana/alloy \
  --namespace alloy --create-namespace \
  --version 1.13.1 -f values.yaml

The node selector in discovery.kubernetes keeps each DaemonSet pod on its own node's pods, so no clustering is needed. To read files instead of the API, swap loki.source.kubernetes for local.file_match plus loki.source.file, keep varlog: true, and add a __path__ relabel rule as the Kubernetes logs guide shows.

Check it is working

kubectl -n alloy port-forward ds/alloy 12345:12345   # open http://localhost:12345
alloy fmt config.alloy                                # syntax check a local file
alloy convert --source-format=promtail \
  --output=config.alloy promtail.yaml                 # migrate a Promtail config

The UI graph view shows each component as healthy or unhealthy. The loki_write_sent_entries_total and loki_write_dropped_entries_total metrics show whether lines arrive.

In the GL stack

Grafana Alloy v1.20.1 runs as a DaemonSet on all three K3s clusters behind builddaily.io, paiddaily.io and the other products: the management cluster, the beta cluster and the prod cluster. Its only job is logs.

• What it does. On every node it tails pod logs, adds a cluster label, and pushes the lines to Loki on the management cluster. That label is what lets one Loki and one Grafana serve all three clusters: a query filters by cluster first.
• Where the data goes. Loki 3.7.8 runs as a single binary on the management cluster and is the central log store. Grafana 13.2.4 on the same cluster queries it next to Prometheus and Tempo.
• What it does not do. Metrics do not go through Alloy. The beta and prod clusters run Prometheus in agent mode, with node-exporter and kube-state-metrics, and remote-write to the Prometheus hub on the management cluster. Traces are trialled on beta through the OpenTelemetry Operator and go to Tempo.
• Failure mode. If the management cluster is down, Alloy on beta and prod retries for a few minutes and then drops the lines. Logs from that window do not reach Loki.

Gotchas

• Drops are silent unless alerted. After max_backoff_retries the batch is gone. Alert on loki_write_dropped_entries_total, or set max_backoff_retries = 0 to retry forever and accept memory growth.
• The WAL needs experimental mode. The wal block in loki.write is experimental. In the v1.21.0 release candidate, a config with the WAL on fails to load unless Alloy runs with --stability.level=experimental.
• Label cardinality. Every label set in discovery.relabel or loki.process becomes a Loki stream label. Do not map pod UIDs, request IDs or user IDs to labels. Put them in the log line or in structured metadata.
• Do not cluster a log DaemonSet. The docs say a log-collecting DaemonSet gains nothing from clustering, because each pod only reads its own node.
• API tailing costs the kubelet. loki.source.kubernetes uses more network and kubelet CPU than reading files. On a busy node, prefer loki.source.file.
• Breaking changes land in minor releases. v1.19 removed prometheus.write.queue and made prometheus.operator.servicemonitors reject endpoints that read local files by default. v1.20 renamed default attribute names in otelcol.processor.k8sattributes. Read the release notes before each minor bump.
• Promtail conversions need review. The converter works on a best-effort basis. Alloy's meta-monitoring metrics have different names, so alerts and dashboards built on Promtail metrics need updates. The positions file is saved in a different location, so a Persistent Volume that held it needs the new path. Promtail tracing does not migrate. The converter assumes pod discovery is for a DaemonSet, so other deployment shapes need the generated discovery.kubernetes block changed.

Risks

• Single-vendor stewardship. Grafana Labs owns the roadmap. The code is Apache 2.0, but two predecessor agents were retired within a few years; plan for migrations.
• Release pace. A minor release lands about every month, often with breaking changes in OpenTelemetry components that follow upstream. Pin the chart and app version.
• Lock-in is low. The otelcol.* components speak OTLP, and Loki and Prometheus push APIs are open. Moving to the upstream OpenTelemetry Collector or Vector means rewriting config, not changing backends.
• Security. On the file path, a DaemonSet with host log mounts can read every container's output on the node, including any secrets an application logs. Restrict who can edit its config.

Alternatives

• OpenTelemetry Collector (contrib): the upstream, vendor-neutral collector. Pick it when every signal is OTLP and no Grafana-specific component is needed.
• Fluent Bit: a small C log processor with many outputs, including Loki. Pick it when memory per node is tight or logs go to several non-Grafana destinations.
• Vector: Datadog's Rust pipeline tool with a transform language (VRL). Pick it for heavy log transformation and routing to many sinks.
• Prometheus in agent mode: pick it for metrics only, when Alloy's extra signals are not needed.

Related

• Loki, Grafana and Tempo: the log store Alloy writes to and the tools that read it.
• Prometheus and OpenTelemetry: the metrics and trace paths that run beside Alloy.
• K3s, Lima and Rancher: the clusters Alloy runs on.
• Official: Alloy docs, grafana/alloy releases, loki.write reference, Migrate from Promtail.