Dashboards & BI
Self-serve principles, metric layers, dashboard hygiene, and the "vanity vs decision" distinction every stakeholder feels but never names.
Why BI gets undervalued (and shouldn't be)
BI is the layer where data science meets the rest of the company. A great model is invisible if its output never reaches the decision-maker. The most-effective senior DS treat BI as a serious craft — not "spinning up a dashboard" but designing artifacts that compound trust and drive behavior.
Bad BI signals:
- Dashboards nobody opens (check usage logs — most BI tools have them).
- Stakeholders building their own spreadsheets to reconcile your numbers.
- Two dashboards reporting the same metric with different values.
- "Quick questions" you keep answering manually because the dashboard doesn't quite cover it.
Dashboards that get used
A used dashboard has four properties:
- One audience. Don't make a dashboard for "everyone." A board deck, a product team weekly, and a sales pipeline view are three different artifacts.
- One decision it informs. Be able to answer "what would a viewer do differently because of this dashboard?" If the answer is "be more informed," it's a vanity dashboard — kill it.
- Trust. Numbers match what the consumer can verify elsewhere. Definitions are visible. The dashboard explains itself.
- Low maintenance cost. Built on stable upstream models, breaks rarely, doesn't require manual refresh.
Tool tradeoffs
| Tool | Strengths | Weaknesses |
|---|---|---|
| Looker | Metric layer (LookML), explore-style self-serve, enterprise governance | Expensive, LookML lock-in, slow to iterate on |
| Mode | Analyst-friendly, fast iteration, SQL + Python notebooks | No real metric layer, governance is weak |
| Hex | Notebooks + dashboards in one, modern UX, app-like interactivity | Newer, smaller talent pool, governance still maturing |
| Metabase | Cheap or free, good enough for early-stage | Outgrown around Series B |
| Tableau | Sophisticated viz, enterprise fixture | Disconnected from warehouse modeling, expensive |
| Sigma / ThoughtSpot | Spreadsheet-style UX for execs, NL queries | Less analyst-friendly |
At HeyGen-stage AI startup, Mode or Hex is usually the right pick — fast iteration matters more than enterprise governance. Looker becomes worth its cost around Series C when you have multiple analyst teams that need a shared metric layer.
Metric layer
The single most under-discussed BI investment. A metric layer defines metrics once, with version control, tests, and lineage — and serves them to every BI tool, app, and notebook.
Without it
- "Conversion rate" is defined three different ways in three dashboards.
- The sales team's CRM dashboard disagrees with the product team's BI dashboard, and reconciling takes a week.
- A junior analyst ships a new dashboard with a subtly different definition, and the number propagates into a board deck.
Options
- LookML (Looker-native).
- dbt semantic layer (works across many tools).
- Cube (open-source, headless).
- MetricFlow (the engine underneath dbt's semantic layer).
Treat metric definitions as code: version-controlled, code-reviewed, tested. The metric layer is where this lives. Selling this investment to leadership is part of the senior-DS pitch — the ROI shows up as fewer reconciliation meetings and faster onboarding for new analysts.
Dashboard hygiene
The discipline that separates production-quality BI from notebook-style ad-hoc:
- Timestamp on every chart: "data through <date>". Catches stale dashboards.
- Definition tooltips: hover any metric, see how it's computed.
- Date filters with sensible defaults: trailing 90 days is usually right.
- No more than ~7 charts per dashboard: more and viewers don't read any of them.
- Comparison context on every metric: "MAU = 1.2M (+5% WoW, +12% YoY)". A number without context is noise.
- Anomaly callouts: highlight metrics outside historical ranges so viewers don't have to spot them.
- Owner and refresh cadence visible: who maintains this, when does data refresh?
Vanity vs decision metrics
A vanity metric is one a leader likes to see but nobody changes behavior based on. A decision metric is one that, when it moves, triggers action. Senior DS spend their time making decision metrics visible — and politely retiring vanity ones.
Examples
| Vanity | Decision metric (same area) |
|---|---|
| Total signups all-time | Weekly new activations (signed up AND completed setup) |
| Page views | Time-to-first-key-action |
| "Total users" | Weekly active users by cohort |
| Total revenue all-time | Net revenue retention by cohort |
| NPS | 30-day repeat-purchase rate |
Anti-patterns
- "Everything in one dashboard." Different audiences want different things. Split.
- Charts without context. A 5% MoM lift looks impressive — until you see that the metric has 8% week-over-week noise.
- Dashboards that compute on the fly with complex SQL. Slow, breaks easily. Move logic into dbt models.
- Pie charts with too many slices. Use a bar chart.
- Y-axes that don't start at zero (when they should). Visually exaggerates moves.
- Color encoding that doesn't reuse a consistent palette. Confuses viewers across dashboards.
- "Final" dashboards that nobody maintains. Document an owner. Sunset dashboards quarterly.
Interview probes
Show probe 1: "How do you decide what goes on a dashboard?"
Start with the audience and the decision they're making. A weekly product review dashboard for a single team has ~5 charts: north-star metric, leading indicators, top 2–3 experiments, and a guardrails section. If a chart doesn't change a decision, it doesn't belong. The "kill criterion" is: would removing this chart prompt any viewer to ask where it went?
Show probe 2: "Two dashboards report MAU and they disagree. How do you fix it?"
Long-term: build a metric layer so MAU is defined once. Short-term: trace both definitions, identify the divergence (is one filtering bots, the other not? Different date truncation? Different definition of 'active'?), pick the right one, kill the other, and write up the decision so it doesn't happen again. Communicate the discrepancy to stakeholders proactively — they probably noticed.
Show probe 3: "What's your process for retiring a dashboard?"
Quarterly sweep: pull view counts (BI tools log this). Anything not opened in 60 days is a candidate. Reach out to the named owner and the last viewer. If neither defends it, mark for sunset with 30 days notice. Sunset means archived, not deleted — easier to restore than rebuild if someone needed it.
Show probe 4: "Should every metric have a confidence interval on a dashboard?"
No. Most production dashboards report point estimates because the audience won't process the intervals. Use intervals on experiment dashboards and forecasting dashboards where the uncertainty drives decisions. On steady-state product dashboards, instead show comparison context: WoW, MoM, YoY — that's a more digestible form of "is this normal."