Join my mailing list!

I'll send out occasional emails with new Articles, Projects, and Updates.

How DarkNet websites are pushing the limits of Web Development

Written Sat Jun 01 2024, Updated Mon Jun 03 2024
Bored of modern web-dev? You might need to go a little deeper than a new JS framework.

If you've ever found yourself tired with the endless permutations of rendering hypertext modern devs love to serve out as the bold, "innovative" framework of the month, you're not alone. Although in many corners of the web, things seem either stagnant or massively overcomplicated, there's one place that has embraced its minimalistic constraints and continues to pioneer new safer and intriguing approaches to problems like phishing, authentication, captchas, and rock-solid OPSEC. All despite the major technological limitations incredible privacy imposes on the seedy underbelly of the deep web.

The challenges of staying safe in a sea of danger

Your security and privacy on the web is an evolving battle. Every couple of months it seems like there's some new type of scam, cyber threat, private data leak, or vulnerability in the legacy technologies many services still rely on. Javascript's legacy in those cursed days of Internet Explorer still rings a little too loud in the minds of security-minded skeptics. Most hidden services explicitly require users to disable JS before even accessing their pages. With the constant push for more dynamic interactivity of modern platforms, we've come a long way from refreshing a forum post every few minutes. But as the old adage uhh... adds constraints breed creativity. And so, the DarkNet has continued to evolve with the rest of UI and backend design trends, but in a sense, from a diverging perspective.

A DarkNet TL;DR

I won't get too technical, but you can think of the web being just that, a web. A hub of connections between different resources hosted on web servers. However, accessing these resources is in many ways a centralized endeavor. Content that cannot be accessed through links part of the wider "web," can be thought of as the "Deep Web" This would include private resources on the internet like your emails and protected content, everything not indexable by search engines. Though technologies like SSL, VPNs, or proxies can attempt to give you greater anonymity and privacy online, because of the infrastructure and architecture of the modern web, they can't circumvent the influence of centralization from ISPs, big tech companies, VPN providers, and their willingness to cooperate with governments.

The TOR Network is a unique sort of internet protocol that encrypts your traffic through layers of decentralized nodes on the network, hosted by dedicated users across the world wanting to sustain the infrastructure. So-called hidden services can host servers that allow users to connect through extremely secure and private means with a greater degree of anonymity. This of course is not without its drawbacks, and the technology needed to make these services functional and user-friendly is something developers accustomed to working on the clearnet might be fascinated to learn about.

Modern browsers and CSS has come a long way!

No Javascript might sound like a dealbreaker, and an instant banishment back to the 90s web 1.0 days. But a lot of DarkNet sites are surprisingly dynamic. Both in terms of interface and reactivity. CSS has gotten extremely functional! In most cases, I prefer to do things with good CSS and HTML over JavaScript, and it's possible for some pretty complicated and dynamic components including stuff like tab views, Modals, nested submenus, and very clean, functional UI. Something many years ago would require mountains of JQuery or some very involved UI implementation.


But even with all of this being the case, you might still assume surely updating the page dynamically in real-time is impossible without techniques like fetch(). But creative use of some cool backend tech still enables these features in a limited capacity. A technique called Long Polling (I talked about in my article about progress bars with code examples.) The short of it is that the server keeps the client's HTTP connection open, holding it even past the time of the page render. It continues to stream the page as hypertext as the user browses, and certain actions can trigger subsequent requests to the server and in turn, render new HTML... WITHOUT JS! There are even some hidden services that enable full live chat with this system. Once again, the Darknet can be slow, and this tactic can enable things like lazy-loading, giving the user a snappier browsing experience. These sites have evolved with the modern web in really interesting ways that a modern web dev seldom deals with.

The Captcha Arms Race

Surprisingly, DDoS attacks are an alarmingly frequent issue site hosts are having to confront. Where these attacks originate is often quite speculative. Perhaps quarreling markets or vendors, others speculate federal agencies. A service growing increasingly popular called EndGame works with the site such that legitimate requests can only be made to it after they've communicated with another server running the open-source EndGame captcha service. This thing can produce some crusty, grimy captchas, hard and annoying enough for most regular users, let alone AI. Captcha services are usually hosted on different servers that can act like a bouncer for only legitimate traffic. This is a very neat look into DevOps on the TOR Network, and could be compared to something like DIY CloudFlare.

While the clearnet has found centralized and frankly creepy ways to verify a user is on their service rather than a bot, hidden services have cooked up some cool captcha systems that really stoke that Wild West type of magical vibe reminiscent of the early days of the web. Some almost feel like puzzles or a developer's attempt to program something radical in terms of UI design or backend functionality. They have to circumvent increasingly competent AI agents making web automation breezier by the day, but still provide something functional and user friendly. Once again the lack of Javascript demands some creative use of things like CSS, spans, and some backend magic and often procedural image generation! Way more fun than being reduced to training some megacorp's AI that'll probably take your job or something.

The Phish are Dead in the Water.

If you access content on the DarkNet you'll notice the "onion URLS" are not like traditional domains. They are long, autogenerated strings assigned when a new host appears on the network. To get a custom "domain", hosts can choose to purchase a service that will "mine" for a URL prefixed with your desired website name. There are even fraudsters that will set up fake Phishing websites to collect users' login data and change things like crypto addresses. Since these long URLs are hard to decipher, sites have created fascinating ways to verify the true URL in the user's browser and the legitimate URL mirrors for the page. Some sites even make users sign up with a private and public username, that only they could know, and then force them to verify it before logging in.

No passwords, PGP to the rescue!

One experimental development is passwordless login. An amazing cryptography tool called GNU PGP allows you to create an "identity" with a matching pair of private and public keys. Other users with your public key can encrypt messages to send you that are only computationally possible to decrypt with your public key. Messages can also have a signature, that only your private key could create, and that, with your public key, other users can verify you created. So the technology allows you to be authentic while still being anonymous, as well as communicate in a manner that only you and the person you're messaging can read.

It can also be used as a replacement for mountains of passwords. In theory, if only you have that key, it can safely serve as verification. Some platforms only allow users to log in with a PGP key alone. It's a genuinely fascinating and innovative approach to user authentication and something I could see taking off in a less obscure manner on the clearnet. Password management is a pain and a whole lot of people don't even do the bare minimum, so a single "token" or process you could perform to authenticate might be an innovation. Something you have, rather than something you know.

Tessa Painter - 2022

Find me elsewhere

Site last updated Wed, 05 Jun 2024 00:49:01 GMT