# Logs, metrics and traces

> Tap the Nodes into Analytics and research the visibility track, so you can see error causes, history and the slow hop of each request instead of guessing.

Tap each Node into Analytics, and add the research that shows errors by cause, history, percentiles and per-hop traces.

## Move up when

- The p99 moves and you cannot say which hop caused it.
- Errors look like one number, with no cause behind it.
- You cannot tell a failing Node from a slow one without opening each card.

## The Design



Traffic flows through the Load Balancer, the App Server and the Database. Analytics receives a copy by Link and samples it. It carries no traffic itself.

## Why it works

Logs show errors split by cause. Metrics add history graphs and the Dashboard. Percentiles show p50, p95 and p99 next to the mean. Tracing names the slowest hop of each request kind, and researching it needs each kind sampled at several different Component types, so link Nodes of different types.

A lower sample rate keeps CPU and latency low on the tapped hops, and a higher one gives more Insight. Tail-based sampling keeps slow and failed traces, so it earns more Failure Insight from the same failures. Retention sets how long traces stay.

## What it costs

Sampling adds CPU and latency to every tapped Node and storage Burn to Analytics. Samples past its capacity are dropped evenly, so size it for the rate you set. Real tracers cost the same way and use sampling for the same reason. The game has no separate log store for application text, so Analytics is the one place the Board shows what it sees.

## In Quorum

Place [Analytics](https://wiki.playquorum.dev/components/analytics.md) and link Nodes into it. Research [Structured logs](https://wiki.playquorum.dev/research/logs.md), [Metrics](https://wiki.playquorum.dev/research/metrics.md), [Percentiles](https://wiki.playquorum.dev/research/percentiles.md), [Distributed tracing](https://wiki.playquorum.dev/research/tracing.md) and [Tail-based sampling](https://wiki.playquorum.dev/research/tail-sampling.md). Read [Insight and observability](https://wiki.playquorum.dev/explanation/observability.md) and [distributed tracing](https://wiki.playquorum.dev/glossary/distributed-tracing.md). The next rung, [services with a mesh](https://wiki.playquorum.dev/patterns/split-into-services-with-a-mesh.md), needs this view to be debuggable.

Source: https://wiki.playquorum.dev/patterns/logs-metrics-and-traces