Remote Linux sysadmin: quiet job or 24/7 on-call duty?

The story of a remote sysadmin watching movies while getting paid clashes with the reality of 24/7 on-call shifts, back-to-back outages and a salary ceiling many question.

English · Original discussion in Spanish · Published

Remote Linux sysadmin: quiet job or 24/7 on-call duty?
The myth of the sysadmin living in his underwear

Getting up at odd hours, connecting via SSH with a public/private key to the server in question, checking that the services (web server, MySQL) are running, updating repositories and patches, setting up a monitoring application with scheduled alerts and then watching a movie until the shift ends. That is the story that sparks the debate: the remote Linux system administrator as the dream job, no interaction with anyone, while the application developer burns out their brain learning frameworks that mutate every year. The discussion, however, falls apart into a thousand pieces as soon as someone who has actually been on call grabs the keyboard.

The workday that never ends

The first blow to the story is the schedule. A system administrator's day does not end at six. Those who have covered real on-call shifts describe entire nights without sleep: interventions at 02:00, alerts at 04:00 and a final call at 06:30 that already makes it impossible to get back to sleep, so you down a double coffee and start catching up on the backlog from the day before. All of this, moreover, with managers who cannot understand why you have not finished what was pending.

On top of that, outages don't send a warning. A hard drive can fry at any moment; a server can come under attack at a random hour. Monitoring exists precisely to flag those problems, which means someone has to react fast, at any time. The programmer, by contrast, has a schedule and disconnects when it ends. The system administrator remains permanently available, and that availability is what wrecks the personal life.

The day everything goes down at once

The image of the technician scratching his belly lasts until the first outage. When a service goes down for no apparent reason, won't start and nobody has touched anything, the party begins. And it rarely comes alone: two machines down at once, an attack, a performance drop, another chained failure. Some describe that stretch as losing four kilos of weight in a few days and the realization that the job is not what it seemed.

The most optimistic calculation assumes that, with your homework done, an on-call shift can be relaxing. The pessimistic scenario starts from the fact that observability and metrics exist precisely because a system can run "without errors" and still have a perversos success rate due to a misconfiguration. Nobody argues that on-call duty shouldn't be paid; the argument is whether the price is only financial.

A dead-end job or a solid foundation?

The second front is salary and career. A system administrator, the argument goes, has limited growth: they maintain what others create, their work gains no added value, and their role resembles that of a civil servant of IT rather than an entrepreneur. A developer, by contrast, can build an application and sell it to thousands of users. That asymmetry is real, though it is also true that developers eat the overtime when deadlines arrive.

Against that view, some argue the job has become so complex that the initial story doesn't even imagine it: cloud identities, continuous integration and deployment pipelines, containers, high availability and scaling. What a trainee used to do on their first day —creating directories, initializing services, managing certificates— is today just the starting point of a chain of tools that fight among themselves. The full calculation, with all the layers involved, paints a considerably less rosy picture than that of the technician watching movies.

Monitoring, observability and the art of renaming the wheel

One of the most curious offshoots of the exchange is the discussion about jargon. What the industry calls observability, the argument goes, is nothing more than the continuous supervision that industrial SCADA systems have been doing for decades: capturing historical data, alarms when a parameter goes out of range and process control. The difference, they say, is that a business layer is now added on top —how many orders come in, how many are lost, how much money goes down the drain— and a prettier Grafana is put on it. Reinventing the wheel and giving it a new name, in short, is a tradition of the sector.

The profile that really keeps the system running

Beyond the salary and the on-call shifts, the portrait that emerges is that of a professional who automates tasks with scripts and their own tools while others keep grinding away at spreadsheets for weeks. A job that can be simple and poorly paid, or complex and paid only in scraps, but which in any case requires constant measuring, even when there are no alarms. The question, in the end, is not whether the job is comfortable. It is whether anyone wants to be the one holding up the infrastructure when everything else fails, knowing the phone can ring at three in the morning.

Also available in: Deutsch

Summary of a discussion on Burbuja.info - Foro de economía, actualidad y política., translated from Spanish and reviewed before publication. Read the full discussion (99 replies).

More summaries

All summaries in English →

-
Back