# The Attacker's Discipline There is a specific kind of cognitive dissonance that comes from reverse engineering a malware sample in one terminal while your own app's backend runs in another. I spent the better part of the last few weeks doing exactly that. During the day I was shipping features and deciding effective logging strategies for Beacon, an attendance management app that I am building. In the evenings, I was pulling apart Android Malware sample from a smishing campaign, tracing through obfuscated DEX bytecode trying to understand what the app did. At some point, the two started bleeding into each other; not because the problems are similar on the surface, but because the discipline underneath them is identical. The malware authors were solving similar engineering problems as me, they just had the opposite intention. * * * ## What the Malware Authors got right A striking observation from reversing a multi-stage malware sample was that it doesn't simply run, it checks first. Before any payload logic executes, there's an environment validation routine. Is this a real device or an emulator? Is there a debugger attached? Are there any hooking frameworks like Frida or Xposed present in the process? Each check contributed to a score, which on exceeding a certain threshold causes the payload to abort cleanly. This isn't paranoia. The authors know that their payload will land on thousands of devices across a wide range of devices. They do all they can to delay an analyst from identifying how the app runs. Nearly all the methods performing these checks had no logging at all. On the other hand, the functions executing the core logic of the malware was fairly sprinkled with logging statements. The malware was built to function correctly on a wide range of devices, with certain code branches for devices running even on Android 7. The functions themselves were cleanly separated with SRP, ensuring that result of every operation could be checked by the orchestrating function while also preserving the modularity of the code. The authors had also added background services which randomly ran to check if their installed listeners and services are still running, and re-starting them if not so. By the time I mapped this out, I had a grudging appreciation for the craft. These are real engineering decisions made deliberately, in service of a clear goal. The goal is malicious. The decisions. though are not sloppy. * * * ## The SRE mirror I do not have production incidents in my history. I have not been paged at 3am, I have not stared at a spiking dashboard while a service falls over. But as I was mapping out the malware's anti-observability decisions, I got curious about what the discipline looked like from the other side, the people whose entire job is reconstructing what went wrong. It turns out the vocuablury is different, but the problem is identical. SRE teams usually work off three signals. Metrics tell you something is wrong. Logs tell you what a specific service did at a specific moment. Traces tell you where in a requests path things broke down. None of these alone tell anything useful. A CPU spike with no trace showing which request caused it is just a number. Stitching the three together under pressure, while things are still on fire is where the actual works happens. There is also a fourth signal that apparently not enough teams track properly: events. Deploys, config changes, feature flags being flipped etc. Most teams let this live scattered across Slack messages and CI logs with nothing tying it to anything else. Almost every incident starts with the same question: what changed right before this happened? Events are very useful to answer these questions. The parallel to what I was looking at was hard to miss. The malware authors had built their payload to generate as few events as possible. Randomized polling intervals so the heartbeat does not look mechanical. No exception logging in the anti-analysis layer. Clean aborts with no trace when the environment check fails. They were engineering against observability the same way SRE practice engineers for it. AI helped me on the malware side in a specific, limited way. Reasoning through obfuscated code across dozens of DEX files, surfacing patterns across classes, cross-referencing decrypted strings. Work that would have taken much longer got done faster. But every conclusion I arrived at was still mine to make. AI cleared the ground. The jump was still manual. From what I read, the same holds on the SRE side. AI in incident response is good at retrieval: digging through past incidents, correlating alerts, surfacing similar failures. The judgment call, deciding what the evidence means, remains on a person. The honest version of the AI claim is that it compresses what one person can reconstruct alone. It does not replace the reconstruction. * * * ## What I'm stealing for Beacon Beacon is not a complex system. It is an attendance and workforce management app for remote-field workers, a Go backend, an Android client, a web admin panel. The failure modes are not dramatic. But they are real, and I have already been caught by a few of them. The events signal is the one that stuck with me most. When something broke in Beacon during development, my first instinct was always to go look at logs. Logs told me what happened. They almost never told me what changed. I had no systematic way of tracking the things that actually cause most failures: a config value I updated, a dependency I bumped, a background job I modified. That information lived in my head or in a commit message I had to go dig up. The malware authors were deliberate about the opposite. They knew exactly what their payload was doing at every point, because they designed it that way. Every action was intentional and accounted for. I was trying to reconstruct intent from fragments. I do not want to be in that position with my own system. The second thing is graceful degradation. The malware was built to fail quietly and contained. A failed check did not crash the app. It returned a result, the orchestrating function handled it, and execution continued or aborted cleanly depending on the score. No noise, no collateral damage. Beacon does not do this consistently yet. Some failure paths are handled well. Others surface as unhandled exceptions that tell me something broke but not what, or why, or what state the system was in when it did. That is the next thing I am fixing. The irony is not lost on me. I learned more about building observable, resilient systems from reversing malware than from any SRE playbook. The authors were not trying to teach me anything. They were trying to avoid being understood. It worked, for a while. But the discipline they applied to staying hidden is the same discipline I want to apply to staying legible * * * ### A note This article started as a conversation with Ayush More from [Sherlocks AI](https://sherlocks.ai), who reached out after reading some of my earlier writing. When I pitched this angle, he put together a genuinely useful set of notes on how SRE teams approach observability and incident reconstruction, which filled in the other side of the parallel I was trying to draw. If the SRE mirror section holds up, a good part of the credit goes to him for grounding it in something real. Go check out what they are building at Sherlocks AI if incident response tooling is relevant to your world.