I’m curious as to which tools and technologies you all are using to keep track of all those services you are deploying, whether it be resource tracking, network traffic, logs, traces, or uptime.

As a bonus question, how have you organized your network or your services to reduce the overhead of implementing observability?

    • SayCyberOnceMore@feddit.uk
      link
      fedilink
      English
      arrow-up
      5
      arrow-down
      1
      ·
      2 days ago

      What’s not to like?

      Still alive & still maintained, and less resources than Grafana & Prometheus…

      • non_burglar@lemmy.world
        link
        fedilink
        English
        arrow-up
        4
        ·
        1 day ago

        Sorry, no disrespect intended, but I’m shocked. It’s like hearing someone say they drive a 1988 Toyota Cressida because it’s great… It was great at the time, but we’ve moved on and the smokeping website should tell you how ancient it is; sponsors from 2007 by companies that don’t exist anymore.

        Still alive & still maintained

        Alive, maybe. I think any maintenance is down to bodges to keep it running in modern environments. Neither Toby nor Niko have worked on sp in over a decade.

        Smokeping is fine if all you do is look at its own graphs and you have enough traffic to see patterns in latency.

        But:

        • more or less unmaintained for a long time
        • cgi scripts and scraping
        • jitter is estimated from rping, not calculated with real values from more than one point on a route (that means a lot when you have a DMZ)
        • Perl
        • rrd tool is… Not great. Not very configurable, also unmaintained, lua support is not good, etc
        • difficult to customize unless you use their submenu system
        • no reporting, you get what you get with smokeping

        I had to retire 2 smokeping monitors because their CGI implementations were security risks. That was 2015. Not a good reason for homelab, but CGI is a pretty ancient and insecure way to interact with the web server.

        Smokeping is good in a big organization with lots of traffic and a few broadcast domains. It isn’t really great at monitoring remote sites because ICMP doesn’t tell you what segment in route is causing the issue, even with rping.

        I used smokeping a lot in my career from about 2005 to 2015, when security audits made me retire it. Just even using rrdtool with an exporter and graphana would be preferable to smokeping itself.

        • SayCyberOnceMore@feddit.uk
          link
          fedilink
          English
          arrow-up
          1
          ·
          7 hours ago

          Sorry, no disrespect intended, but I’m shocked. It’s like hearing someone say they drive a 1988 Toyota Cressida because it’s great…

          How did you know I had an 88 Cressida? (jk)

          You have good points there, but for a homelab, it does show some interesting insights (ie, I did an upgrade of my Raspberry Pi Zeros and saw how all their latency changed after the reboot… weird)

          I’ve been using this for years as it just worked, but a recent update has broken it (a simple “v1.10 is earlier than v1.9” issue) and I had to rollback and have been waiting for someone to fix the bug for a few weeks, so, yep, the slowness of response is a fair point.

          Ah, dunno, maybe I’ll look at some alternatives… one day…

        • Denys Nykula@piefed.social
          link
          fedilink
          English
          arrow-up
          1
          ·
          1 day ago

          What’s insecure about CGI? I thought the main reason it isn’t popular now is because runtimes are slow to start. Though IIRC when I used busybox httpd CGI with a small runtime, quickjs, speed wasn’t an issue.

          • non_burglar@lemmy.world
            link
            fedilink
            English
            arrow-up
            3
            ·
            1 day ago

            From a security perspective, it’s a big mess of inputs and can have really inappropriate access to local filesystem on the web server unless you really know what you’re doing. Combine that with Perl (also a pita to secure), and it’s now a liability.

            These are good tech, I used them a lot myself. But the structure of a language and how it builds things is important too, and that’s why very few ppl bother with Perl or CGI now (besides them also being fails on certain security audits.)