[ OK ]Initializing kernel...
~/im/blog
Hire Me

Let's Talk

Interested in working together or have a question? I'm always open to discussing new projects.

Get in touch

Connect

Find me on social media and professional networks.

Privacy PolicyTerms of Conditions
© 2026 Irfan MiralDeveloped byirfanMiral.com
HomeAbout/ResumeBlogContact
2026-10-28• 5 min read

Log Management on a Single Server

DevOps Logging Linux Administration Observability

When something goes wrong on a single server, the logs are usually all there, scattered across /var/log/nginx/, /var/log/postgresql/, the systemd journal, and whatever the application itself writes wherever it felt like. Finding the relevant line means knowing which of five places to look, and if the app logs plain text instead of anything structured, finding "all requests from this IP in the last hour, across nginx and the app" means several different grep commands with different formats. None of this is broken, exactly, it's just slower than it needs to be, especially under pressure.

The full ELK stack (Elasticsearch, Logstash, Kibana) solves this, but it's genuinely heavy for one server, Elasticsearch alone wants a meaningful chunk of memory just to run. The useful middle ground for a single server is smaller than that.

Structured logs make everything else easier

The highest-leverage single change is getting application logs into a structured format (JSON) instead of free-text. Compare:

2026-10-28 14:32:01 ERROR Failed to process order 4821 for user 1029: timeout
{"time":"2026-10-28T14:32:01Z","level":"error","msg":"failed to process order","order_id":4821,"user_id":1029,"reason":"timeout"}

The second one isn't more readable to a human glancing at it, but it's queryable: "every error for user 1029" or "every timeout in the last hour" becomes a filter on a field instead of a regex that might also match unrelated text containing the same numbers. Most logging libraries support structured output with a configuration change, not a rewrite.

journald is already doing more than people use

For anything running as a systemd service (which is most things on a modern Linux server), journalctl already centralizes stdout/stderr with timestamps, and supports filtering that a lot of people don't reach for:

# Everything from the app, last hour
journalctl -u myapp --since "1 hour ago"

# Follow logs from multiple units at once
journalctl -u myapp -u nginx -f

# Only errors, across everything
journalctl -p err --since today

That last one, filtering by priority across the entire system, is something a scattered collection of log files can't do at all without external tooling. If application logs go to stdout in JSON (rather than to a separate file), journalctl's output for that unit is already structured, and can be piped to jq for filtering on specific fields.

A lightweight stack, if you want dashboards

For something closer to a real log viewer without Elasticsearch's footprint, Grafana Loki plus Promtail is the commonly recommended lightweight pairing, Promtail tails log files (or reads the journal) and ships them to Loki, which Grafana can then query and display alongside the metrics dashboards from Prometheus if that's already set up. Loki's resource footprint is small enough to run comfortably alongside an application on the same VPS, which is the main reason it fits here where Elasticsearch wouldn't.

# promtail-config.yml (minimal)
scrape_configs:
  - job_name: journal
    journal:
      max_age: 12h
    relabel_configs:
      - source_labels: ['__journal__systemd_unit']
        target_label: 'unit'

Don't forget rotation

None of this matters if disk fills up with logs nobody's looking at. logrotate is usually already configured for common services, but application logs written to a custom path need their own entry:

# /etc/logrotate.d/myapp
/var/log/myapp/*.log {
    daily
    rotate 14
    compress
    missingok
    notifempty
}

And for the journal itself, capping its size avoids the journal quietly growing unbounded:

# /etc/systemd/journald.conf
SystemMaxUse=1G

The actual goal

None of this needs to be elaborate. The goal is that when something breaks, the question "what happened, and when" has one answer-able place to look, with enough structure to filter by what matters (a user ID, an error type, a time range) rather than scrolling through plain text hoping the relevant line stands out. Structured logging plus journalctl's filtering covers a lot of that for free. Loki and Grafana add a browsable interface on top, for when "for free" stops being enough.

PreviousMonitoring a Small VPS with Prometheus and GrafanaNext Is SSH 2FA Worth the Hassle?