The System Does Not Reward the Seam-Patchers
目录
- Walking on the Edge
- Walking Along Component Edges
- “Don’t Reinvent the Wheel”
- The More You Fix, the More You Can Fix
- The Split Between Software Development and Operations
- Important Seams Are Often Not Obvious
- “Frontend Thinking”
- Responsibility Inversion
- Asymmetric Payoff
- Can AI Patch the Seams?
- Conclusion
- References
“Frontend Thinking,” Responsibility Inversion, and Asymmetric Payoff
Aka, Walking Along Component Edges; or, Why the Computer Industry Punishes People Who Don’t Reinvent the Wheel
Aka, Why Open Source Is Being Hit by AI Contributions
Walking on the Edge
The 201 PRs I've submitted to other people's projects (click to expand)
Mojo-Webqq #255、Mojo-Webqq #261、Mojo-Webqq #276、docker #6、Install-Lion #2、QvPlugin-Command #5、Magisk #4613、ATLauncher #507、ATLauncher #511、Android-Architecture-knowledge-2- #4、bilimiao2 #8、magisk-unifiednlp-aosp-enabler #2、download-navi #263、bilimiao2 #17、bilimiao2 #18、NewPipeExtractor #936、LibreTube #3624、LibreTube #3632、linux-enable-ir-emitter #104、oiledmachine-overlay #81、distrobox #886、AM #58、AM #59、ansible-runner #1306、restic-android #199、bilimiao2 #67、QLDependency #50、Get_JDCookie #5、config #210、CachyOS-kernels #13、CachyOS-kernels #14、grpc-rust #1692、displaycal-py3 #380、distrobox #1455、eselect-llvm #3、stickerpicker #78、bilimiao2 #160、xarblu-overlay #507、InfiniTime #2143、distrobox #1622、hev-socks5-tunnel #208、hev-socks5-tunnel #219、LibreChat #7584、AM #1589、NCM-Downloader #14、installkernel-gentoo #36、GradleAndroidRustPlugin #10、GradleAndroidRustPlugin #11、kernel-patches #132、kvm-guest-drivers-windows #1473、kvm-guest-drivers-windows #1474、kvm-guest-drivers-windows #1475、kvm-guest-drivers-windows #1479、virtio-win-guest-tools-installer #85、kernel-patches #135、distrobox #1982、distrobox #1987、virtio-win-guest-tools-installer #87、virtio-win-guest-tools-installer #88、GradleAndroidRustPlugin #14、zram-init #57、zram-init #59、rclone #1、kvm-guest-drivers-windows #1536、kvm-guest-drivers-windows #1537、SillyTavern #5309、SillyTavern #5316、SillyTavern #5333、SillyTavern #5334、AM #2176、AM #2177、SillyTavern #5351、python-evdev #251、freebsd-src #2116、guru #1、rclone #2、reframe #25、reframe #26、hermes-webui #2189、hermes-webui #2190、honcho #689、opencode-models-discovery #3、plannotator #805、OmniRoute #2975、OmniRoute #3064、plannotator #843、OmniRoute #3094、OmniRoute #3146、OmniRoute #3188、OmniRoute #3422、OmniRoute #4293、OmniRoute #4359、neomacs #134、neomacs #135、neomacs #136、neomacs #137、neomacs #138、pi-hindsight #444、pi-honcho-memory #16、pi-mcp-adapter #159、OmniRoute #6322、OmniRoute #6323、OmniRoute #6330、OmniRoute #6336、OmniRoute #6702、OmniRoute #6714、OmniRoute #6727、pi-subagents #136、pi-subagents #139、OmniRoute #7004、OmniRoute #7012、OmniRoute #7027、pi-mcp-adapter #181、pi #6697、pi #6764、pi-subagents #153、pi-subagents #154、OmniRoute #7673、OmniRoute #7700、OmniRoute #7747、OmniRoute #7776、OmniRoute #7843、pi-subagents #169、OmniRoute #8367、pi-subagents #173、pi-tasks #42、pi-subagents #180、OmniRoute #8908、pi-hermes-memory #140、pi-subagents #191、pi-hermes-memory #141、pi-hermes-memory #142、OmniRoute #9199、OmniRoute #9205、pi-subagents #198、OmniRoute #9262、OmniRoute #9290、OmniRoute #9313、OmniRoute #9329、pi-intercom #87、OmniRoute #9555、OmniRoute #9569、pi-subagents #839、pi-hermes-memory #165、OmniRoute #9788、pi-subagents #922、OmniRoute #9983、OmniRoute #10066、codebase-memory-mcp #1530、OmniRoute #10072、pi-honcho-memory #21、OmniRoute #10079、pi-subagents #973、OmniRoute #10123、OmniRoute #10125、OmniRoute #10127、OmniRoute #10162、pi-caveman #15、OmniRoute #10243、OmniRoute #10247、pi-subagents #1141、pi-hermes-memory #189、pi-subagents #1214、rpiv-mono #174、OmniRoute #10712、OmniRoute #10716、OmniRoute #10723、OmniRoute #10806、deno #36644、pi-subagents #1297、pi-hermes-memory #191、pi-hermes-memory #192、OmniRoute #10953、OmniRoute #10963、sub2api #6139、CLIProxyAPI #5246、CLIProxyAPI #5247、CLIProxyAPI #5264、EternalTerminal #1、pi-subagents #1747、EternalTerminal #803、wenai #1、tproxy-config #36、sub2api #7215、sub2api #7217、rpiv-mono #241、honk #5、neomacs #412、rpiv-mono #262、CLIProxyAPI #6189、gpt-with-taste #6、sub2api #8191、cpa-quota-estimator #4、CLIProxyAPI #6760、sub2api #8302、aisix #1370、aisix #1387、youki #3862、youki #3866、youki #3867、honk #450。
I love the feeling of fixing the things I use every day and making them better for everyone, and I always assumed that doing enough would mean getting more back—earning more, or becoming better known in the open-source community and landing a better job. But unfortunately, I’ve gradually realized that my labor has become a stepping stone for others, while my name disappears without a sound.
To someone like me who loves open source, it feels as if people in the open-source community are hurting each other. The more a person does for compatibility, edge cases, and the features they themselves need, the more they fail—and the more the project owner succeeds. I don’t doubt others’ success, but I can’t help reflecting: am I doing this only for humanity—a humanity that doesn’t include me?
For someone like me, who was exposed to open source on the very first day I wrote code, the earlier you explore open source, the more you fail—it turns your worldview on its head. But when I think it through, the problem isn’t open source; it’s the structural inversion of the computer industry. And this structural inversion fundamentally damages any community built on computer software, and it’s also why “wheel-building” has become the phrase everyone hears most often.
Walking Along Component Edges
“Don’t Reinvent the Wheel”
Not reinventing the wheel means reusing components. The more you reuse, the more seams there are between components. Systems are split up according to job roles, but problems don’t occur along role boundaries.
The more you favor reuse, the more you end up “doing it all”; the more you do it all, the less specialized you become. But I think being able to do it all is naturally an advantage, because it means you can learn, go deep, and solve problems across most dimensions. Breadth isn’t just about avoiding duplicated work; it also avoids problems caused by insufficient breadth. For example, a Java programmer can recite JVM optimizations from memory but has no clue how the operating system schedules the JVM itself, so they end up repeatedly optimizing along a futile path.
And the ultimate form of “don’t reinvent the wheel” is virtualization. I dream of being able to virtualize a single, networked operating system that spans hardware. And that means I need to do a bit of everything, so I need to learn a bit of everything.
The More You Fix, the More You Can Fix
A few years ago, I was still fully involved in building open-source software for Android apps and Home Assistant. Back then, I just quietly repeated the mantra: “It’s good for open source to have bugs—it shows everyone can implement their own features, so there are bugs and you can still fix them yourself.” I was still full of hope then, just puzzled about why open-source software always had an endless supply of obvious bugs and missing features, with a crowd of people filing issues but no one lifting a finger.
For example, comment replies in NewPipe: someone filed it back in 2019, and in 2022 I wrote NewPipe #9020 (later moved to #9410), plus the companion NewPipeExtractor #936 to display the reply count; in Syncthing-Fork, syncthing-android #914 to make network requests go over a specified network; in LibreTube, LibreTube #3624 so you can control the player while viewing comments; and Home Assistant’s Bluetooth device tracking didn’t work on newer Python versions, so I filed core #108513; the corresponding issue has 123 comments.
And then, over the following year, I began to discover, as a Linux developer, the messiness of Linux and why the big companies all have their own forks.
My PRs with Red Hat started with Ansible, for a rather accidental reason: the RHCE exam covers Ansible. So while studying, I smoothly discovered the problem that Ansible couldn’t exit when running piped commands, and after investigating, I ended up with ansible-runner #1306.
Then later, immutable distros took off. I thought containerization really was quite good in practical development, so I tried distrobox—and successfully started fixing distrobox (a misspelled package name under Alpine, distrobox #886; package installation failing during Gentoo Stage3 image initialization, #1455; the runtime sync fallback in distrobox-init swallowing errors, #1622; and a workaround for podman stop and rm timing out with rootless plus --pid host, #1982, submitted in January 2026 and still open as of now). Eventually I fixed my way down into podman and runc/crun/youki (issue: podman #30078; PRs: podman #30079, container-libs #1307, crun #2427, runc #5594, youki #3862).
Then later, after my years working at Klavis AI came to an end, I decided to settle down and study virtualization again. I came across paravirtualization, and somehow I got the urge to install a paravirtualized Windows, and then I began a year-long stint of fixing virtio-win. The PRs I filed that year: kvm-guest-drivers-windows #1473, kvm-guest-drivers-windows #1474, kvm-guest-drivers-windows #1475, kvm-guest-drivers-windows #1479, virtio-win-guest-tools-installer #85, virtio-win-guest-tools-installer #87, virtio-win-guest-tools-installer #88, kvm-guest-drivers-windows #1536, kvm-guest-drivers-windows #1537, virtio-win-guest-tools-installer #108, virtio-win-guest-tools-installer #109.
Most of them aren’t edge cases you only hit when chasing maximum performance; they’re basic contracts like failure cleanup, state truthfulness, and terminal (TTY) semantics:
- kvm-guest-drivers-windows #1473: In the error path for initialization failure, the logging code accessed a member pointer that hadn’t been assigned yet. It was supposed to report the failure, but reporting the failure itself failed.
- kvm-guest-drivers-windows #1475: After framebuffer initialization failed, the resources and IDs that had already been created weren’t fully reclaimed.
- virtio-win-guest-tools-installer #87: In maintenance mode, the UI showed that the new feature was installed, but the actual files and services weren’t deployed.
- ansible-runner #1306: It mistook “has an input file descriptor” for “needs a TTY,” so when output was redirected it incorrectly entered interactive behavior. Submitted in September 2023 and only merged in February 2026.
- podman #30079: A bug from last year finally has a lead thanks to it.
All these problems are stuck at the seams: between Ansible’s input/output semantics and the container’s TTY allocation, between the installer’s UI state and the actual installation after privilege escalation, between the behavior of the container engine, the runtime, and cgroups. Each part may have someone responsible for it, but “connect these parts, trace it to the real cause, then fix it upstream” is too hard for anyone to take on in full. This is exactly the work I’ve long been used to doing, but I don’t enjoy it, because it tortures me: no one pays you a salary for it, and the more you do it, the more your résumé looks like an SRE’s or a full-stack developer’s—as if it’s increasingly proving that you’re just a superficial programmer.
But because I need to use it, I have no choice but to do it. That’s also how my downstream fork xz-dev/pi came about. “This is the first time I’ve hated a tool I like this much, so I forked it.” The Pi team is very slow at handling community contributions and fixing bugs, and always says “no”; it originated from a conflict in Issue #6234. I forked it because when I send a message mid-task, I can’t interrupt the current execution in time, and pressing Esc can’t stop stuck hooks and streaming responses either. Plus many, many other fixes. But this time I’m genuinely tired. I have direct PR and issues permissions on upstream pi, but I gave in. I realized that what I’d been doing, and suffering over, was being a mouthpiece. In a community, the mouthpiece often ends up being given the glorious title of “coordinator” and then forced into systematic exploitation. So this time, I’m trying another approach: put it here and let someone else do it. And this is also what big companies do as enterprises—they can open-source and contribute upstream, but their patches go live in their own project first.
All patches in this fork are free to submit upstream directly, or to use as a reference for your own upstream PR. I waive any code ownership over these downstream patches; please credit this repository as the source in your PR.
It tracks upstream main with a minimal downstream patch stack, using downstream-fork-maintain-skill as the blueprint for ongoing maintenance.
Because I realized that if I still have any hope of continuing to develop, then I can’t be the “coordinator.”
The Split Between Software Development and Operations
I was forced to start out from the backend, but I always consoled myself that only by becoming familiar with software development would I later better understand why the operating system is designed the way it is. So gradually I came to see this enormous split.
In software development, you really can manage processes yourself, because you wrote all the code yourself and can manage almost everything in fine detail—and everyone genuinely believes the operating system will handle the fallback. But most people don’t know the operating system deeply. That is, they actually don’t know much about what kind of environment their software runs in.
Meanwhile, ops sees process leaks or inefficient software every day, but they won’t think it’s something that needs optimizing—or they don’t have the ability to fix it. First, they’re not developers and don’t know the details. Second, most ops people lack the capability (this lack of ops capability is the other side of the structural problem I’ll discuss next), and the company doesn’t give them the resources to do it.
So it turns into this: the people writing code don’t know what actually happens at runtime, and the people who know the runtime state don’t know the specifics. That’s also been my experience at work: many programmers know, recite, feel, or guess at the environment their program runs in, but they basically don’t have an accurate feel for it. As for how long a function should actually run, no one can give an estimate.
The people who maintain can’t write; the people who write can’t maintain.
Podman #30078 is a concrete example:
- It uses
--pid=host, with no private PID namespace to serve as the boundary for cleaning everything up as a unit; - In the rootless case, creating a cgroup fails, but the runtime allows that failure to be ignored under certain conditions and continues starting;
- When stopping, it in turn relies on the cgroup to signal the entire workload, and when there’s no cgroup path, the relevant function simply returns success;
- The upper layer waits for exit, eventually times out, and the process may still be alive.
The bug is obvious, and so is the fix. You could say this is just an oversight in the long development cycle of a big project like crun. Or is it?
Going one layer deeper: to this day, the Linux kernel still has no way—independent of PID namespaces and cgroups—to manage descendants via a reaper the way FreeBSD does. FreeBSD’s procctl(PROC_REAP_KILL) can send a specified signal to the descendants managed by a reaper, and can also select only direct children or a particular subtree. On Linux, PR_SET_CHILD_SUBREAPER mainly handles reparenting of orphaned processes, exit notification, and reclamation via wait(); a process group is not a descendant tree; pidfd solves the problem of stable target identity, avoiding misdirected signals caused by PID reuse; cgroup.kill manages a cgroup subtree, can only send SIGKILL, and its scope isn’t determined by process lineage either. Nor can this be filled in by writing a recursive traversal: an application controls its own source code, but it can’t control lifecycle changes like third-party child processes, concurrent forks, and reparenting.
cgroup.kill itself was only added later, entering the Linux 5.14 generation in 2021. And its addition didn’t end things: in 2025 people were still fixing a race between fork and cgroup.kill, and in 2026 they fixed another issue where children created with CLONE_INTO_CGROUP were killed by mistake.
I’ve always felt that Linux drivers, bug fixes, performance optimizations, and so on can all be improved quickly, while details stuck in the middle like process management get completely ignored, and no one is willing to push them forward. To put it more precisely: the path to progress is fundamentally different for problems that are inside a subsystem and have clear ownership, versus problems that span components and lack shared acceptance criteria.
This split has at least three levels:
- The knowledge split: the people who know the business intent don’t necessarily know the runtime semantics of processes, signals, resources, and failures; the people who see the runtime symptoms don’t necessarily know the code path that produced them.
- The authority split: the people who can see a leak may not be able to modify source code, ship a release, or change the runtime environment; the people who can change code may not have access to the real failure scene.
- The incentive split: delivering a feature has direct acceptance criteria, while fixing a cross-layer lifecycle requires coordination across several projects and teams, and the payoff is dispersed to future maintainers and other users.
The most dangerous state isn’t that no one is responsible; it’s that every layer has someone responsible, yet no one is responsible for proving that the layers still work correctly when combined. Information, capability, authority, and responsibility don’t sit within the same closed loop. So handing this kind of problem to either development or ops doesn’t naturally constitute a solution: the former may only be asked to be responsible for features, while the latter is asked to be responsible for a system they cannot sufficiently change. A lack of ops capability makes the problem worse, but even if you replace them with excellent ops people, without development time, modification permissions, and the resources to push things across projects, they won’t automatically gain the conditions to solve it.
So we often see smoke tests, but they can only express “I started it”; they don’t mean the whole system can now express and honor “I am responsible for it and all the work it produces, through to the end.” Seams become weak points not only because some interface is missing, but even more because, among the interface, the runtime environment, and the organizational division of labor, there is no verifiable lifecycle chain of responsibility. And what’s even more fatal is that there actually is a solution—chaos testing, as proposed by Netflix. But designing chaos testing correctly itself requires this ability to work at the “seams” over the long term. Yet in fact, as all of the above shows, the computer industry does not reward behavior that cultivates this ability, so this becomes a chicken-and-egg problem.
Important Seams Are Often Not Obvious
virtio-win’s issue #765 is another proof. It was opened in May 2022 and still hasn’t been closed; it has 33 comments, and the reported versions range from 0.1.215 all the way to 0.1.271. The people reporting it each found their own way around it: using Microsoft’s troubleshooting tool to force an uninstall, temporarily switching the system disk to SATA, manually right-clicking to install the INF, unchecking some component, or just rolling back to 0.1.248. Once the problem is bypassed, the failure scene is gone too. People involved in the investigation replied back in 2022 that they couldn’t reproduce it and asked everyone to provide logs; in February 2026 someone analyzed a log and still didn’t know the cause, and a fresh install followed by an upgrade on a brand-new Win2025 machine didn’t reproduce it either.
I filed PR #85 for it, but after I finished writing it, I couldn’t reproduce it myself either. Later I used GPT and Claude to reproduce it case by case on virtual machines, spending three days and nights without stopping, before I found the trigger conditions: one of them was that the machine had a previously interrupted old installation, while the brand-new test machine happened to lack that history; other failures also depended on the device installation policy, hidden child devices, services rewritten by the new INF, and the signature of the x86 driver package. The maintainer replied half a year later, saying they’d also tried a similar approach and encountered cases where it didn’t work correctly, so they decided to leave it for now and keep the PR open.
So the problems at this seam are:
- There is no specification that could truly cover every case, so things can look wildly different across systems;
- When it happens, it’s usually ops or the delivery team that finds it first. If it does manage to propagate to development, the information may be distorted, or the original environment may have been lost long ago—that is, it was temporarily fixed by other means, but the failure environment can no longer be restored;
- Fixing it is fixing a bug, which only counts as “compensation,” not a benefit;
- The responsibility of fixing it outweighs the payoff, because it’s just “maintenance,” so the return is far less than the value of developing a prototype or a “large system”;
- Fixing them is often not simple—or rather, it requires accumulated experience, accumulated knowledge, and time for fixing and debugging.
Eries also said, “After all, there’s a limit to how fast a person can absorb knowledge.”
“Frontend Thinking”
The “frontend thinking” I’m talking about doesn’t refer to the frontend profession; it names a way of working: not solving problems from the standpoint of correct implementation, but the feature-first implementation companies push you into, replacing complete implementation with the visible path to success.
The standard for “done” gets quietly shrunk to: the button clicks, the API responds, the demo works—therefore the feature is complete. But the questions that actually determine whether software can be entrusted are left to someone else: What is the state after a failure halfway through? Can it be safely retried? Does cancel actually stop it? Who releases the resources? How does the user confirm the result? Who bears the cost when performance degrades?
A fix that papers over the problem can pass acceptance just the same. The official virtio-win installer can return success under device installation policy restrictions or in the case of a partial upgrade, while the device doesn’t actually get its driver installed, because installation and uninstallation errors are swallowed.
Even a big SRE company like Red Hat has missed a great many things—at the very least, even I can run into this many long-standing, glaring problems. Just look at my fixes to ops tooling, and the virtualization-related fixes.
Now that AI is here, frontend developers can write backend code, and things get even worse: performance is now completely divorced from the direction of computer optimization. Ops projects like AI gateways are left with nothing but crashes and makeshift fixes, plus a crowd of people watching and following a garbage project written by a frontend developer; most companies say AI is important, yet they don’t even have a stable channel for an AI gateway.
So I think that, with AI becoming widespread, the computer industry is rapidly exposing a large-scale rot of “frontend thinking” rooted in “responsibility inversion”—no longer confined to ops and the OS layer, but expanded into the business layer. Are we really going to be left hoping that when software fails it just spits out a string of JSON errors (I was stunned the other day when I opened the Hema app: the whole blank page had nothing but a JSON string), because nobody tests at all and nobody thinks about how to handle errors gracefully; that users will only be able to operate software through AI agents; and that software performance will be so bad that only AI really has the patience to use it?
DORA’s 2026 analysis of AI also sees a similar pattern: the time saved by code generation is often spent right back on review and verification; the more AI is used, the higher the delivery throughput—and the higher the delivery instability. AI is more like an amplifier of an organization’s existing capabilities and flaws than a tool that automatically fixes the engineering system. The danger is that companies only cash in on “writing faster” but don’t pay for “verifying more,” and then use agents to cope with software failures: if the software is hard to use, let the agent take a detour; if a defect doesn’t affect closing deals for now, the priority of fixing it keeps dropping; and so the software becomes even harder to use.
The problem isn’t JSON itself, either. Structured errors aimed at programs can be perfectly good design; the problem is that the software simply throws out an error, without accounting for what it did, what it didn’t do, whether the state is determinate, or whether the next step is safe. Handling errors gracefully doesn’t mean swapping in a nicer-sounding message; it means fulfilling those responsibilities. Software vendors today find it easier and easier to package unfinished engineering responsibilities as adaptability that users or agents are supposed to have. “AI can operate it for me” and “this software has been correctly implemented” must always remain two separate things.
And this “speed-as-correctness” ethos breeds another problem of “frontend thinking”: endless reinventing of the wheel. And because, as all of the above shows, the system does not reward seam-patching, software development can only be achieved through wheel-building. (XRay suddenly comes to mind—the whole V2Ray fork, Long live Chairman R!)
Responsibility Inversion
I call this responsibility inversion. Because you can’t directly or intuitively see the payoff it generates, you don’t give the role much pay or resources; so no one is willing to go deep in it; without anyone going deep, this area gets even worse; and the worse it gets, the fewer people are willing to stay. And with no one going deep, companies conclude that ops is simply garbage, and naturally won’t spend much to hire. The end result is like OpenSSL: the foundational components rot. It’s not that everyone is pretending not to see it—no one genuinely cares at all, and even the people who do care can’t solve the problem.
Most companies are even stranger: ops invariably gets the lowest pay. As a result, the part that most affects the company—and matters most—doesn’t get enough money to hire people to fix it; instead, more money and effort go into the business side to compensate. The people managing virtualization are all ops, and cheap ops have no idea how to properly understand virtualization. I often complain: many times the SRE-to-business ratio feels badly out of balance, and plenty of ops tooling has long-standing unfixed bugs.
But I’m not saying Red Hat pays poorly. Red Hat doesn’t pay low wages; the problem is that the resources around Red Hat are too few: Red Hat can’t fix every bug, and the people using this software clearly aren’t capable of fixing them either—otherwise they wouldn’t have been sitting there for years, waiting for someone like me, who’s in the habit of fixing the seams between systems, to come fix them. A single vendor’s engineering capacity can’t automatically cover the problems accumulated across the whole downstream ecosystem. And downstream isn’t just a question of “are there smart people”: recognizing symptoms, locating cross-layer causes, writing the fix, adding tests, and getting the fix accepted upstream are different abilities, and they also require time and organizational permission. Miss any one link, and usually all you’re left with is rebooting, config workarounds, and pinning old versions.
I think this is also why AI this year has brought such a massive “seam-patching” shock to open-source software.
Steve Marquess, who raised money for OpenSSL back in the day, wrote about it in concrete detail: commercial customers are willing to pay for specific deliverables, but they typically don’t fund basic maintenance—version management, code review, refactoring, performance, security. It isn’t that people don’t know it matters; it’s that “depending on it” doesn’t automatically become “paying to maintain it.”
Within an organization, responsibility inversion has three layers:
- Benefits accrue to features, costs accrue to maintenance. New features count as project achievements; subsequent failures count as ops work, not as part of the original delivery.
- Those who carry the responsibility lack decision rights. Ops is required to guarantee stability, but isn’t given enough say in design, enough time to fix things, or the authority to block a substandard release.
- Successful firefighting masks underinvestment. Manual firefighting and workarounds keep the system running, so management sees “it still works”—not how much specialized labor it takes to keep it “still working.”
Back to those five points at the seam: fixing it is only “compensation”; the responsibility of fixing it outweighs the reward; and fixing it takes years of experience, knowledge, and debugging time. So all told, we’ve created a white-collar job where the effort exceeds the reward, and where—as the problems pile up—people will inevitably leave, or get pulled onto other, more economically rewarding projects.
There are two mutually reinforcing gaps here. At the ecosystem level, a large number of users reap the benefits while a small number of maintainers shoulder the public maintenance, and investment can’t automatically grow with the scale of dependence. At the company level, employees are judged on immediate delivery, while cross-layer understanding, upstream fixes, and long-term efficiency don’t necessarily earn corresponding rewards. So people who know how to chase things to the root are both scarce and insufficiently incentivized to keep taking on this work. And by making “get the feature running first” cheaper, AI may widen this gap further rather than automatically closing it.
So society’s or companies’ exploitation of the open-source community isn’t actually deliberate—it’s structurally inevitable exploitation. Because only open-source maintainers have the faith to run on love.
Asymmetric Payoff
The kind of work I do now pays less than those frontier fields—e-commerce mobile development and Web development in the past, AI now. But today’s AI also has the “frontend thinking” problem I mentioned earlier, and many people even use AI as a direct replacement for their brains: they don’t understand the principles at all, they only know where the answer is, and they have no idea what they’re doing—yet they make more money than people doing this kind of work. So this is a kind of mismatch between responsibility and authority—or rather, an asymmetric payoff.
I’ve discussed this with friends: companies don’t reward you for understanding technology better. Maybe you write more efficiently, but if someone else can also get it done, the company doesn’t care that much. So I always joke that the state should pay me for reading technical books.
It’s not necessarily that companies don’t see the technical difference. They may see it and still have no incentive to pay extra for it. What a company buys is “this role gets the immediate task done,” not paying tiered by a person’s depth of understanding of computing. If someone else with a shallower understanding can also pass acceptance, deeper ability doesn’t necessarily become a pay difference. And the benefits of these improvements may land on the people who take over maintenance in the future, on other companies using the same upstream software, on failures and capacity pressure that haven’t happened yet, on the time users save waiting, troubleshooting, and taking detours. These benefits can be real, but they don’t necessarily flow back to the current department’s budget, much less to your own salary. Technical value, the value the company can capture, and the compensation an individual can receive are three different things. So you can’t simply expect that “the market will naturally reward people who understand technology better.”
So when people say “others can do it too,” how far did they actually get? If two implementations really show no meaningful difference in real-world load, failure rate, and maintenance cost, then it’s a normal trade-off for the company not to pay for extra technical depth; greater technical sophistication doesn’t automatically mean a more worthwhile commercial investment. But the common case is this: once it’s shown that both can run right now, they’re treated as equivalent across the entire lifecycle. One implementation needs constant firefighting; the other handles failure correctly. One consumes more resources; the other scales stably. If those differences end up being borne by ops, users, or a future team, whoever is delivering it now can still be seen as having “done it too.” That’s what makes a makeshift completion competitive: not that it has no cost, but that the cost isn’t counted in the same ledger.
Job hunting works the same way. Today’s job market expects people to be able to work the moment they arrive, without any learning, so naturally it wants just the right amount of vertical depth—or a candidate who is deep enough in a vertical direction that interviewers can imagine they have both the ability and the willingness to dig deep in whatever role is needed. The more you do it all, the less specialized you look to interviewers.
The industry relies heavily on cross-layer understanding and long-term maintenance, but it hasn’t established investment and reward mechanisms that match that reliance. It’s not that technology has no value; it’s that the necessary technical value has no stable payer or bearer.
Can AI Patch the Seams?
Maybe we tell ourselves that AI has arrived and can solve this. Eries once joked to me that in the future “no one will understand the code; there’ll just be a prompt that makes AI generate the ELF directly at compile time.” I replied: “Linus said AI is a compiler.”
But every project ultimately serves people, and therefore people are responsible for it too. AI really can fix some things in many places, but as a developer, can you truly understand the fix? If you really can, then you automatically end up back in the loop I described earlier; if you can’t, then no one is truly responsible for that fix. AI also can’t recover those failure scenes that have already vanished.
I myself use AI extensively to write code; in xz-dev/pi almost none of the code is handwritten by me. I fully embrace AI, but I also genuinely feel the need to actively “snatch” control of the project away from AI. AI is definitely an accelerator, but whether you can use it well depends on whether you use AI with a clear head—and using it with a clear head means “you really are just treating AI as an interpreter,” rather than treating yourself as the “Jev model” (a choice router).
AI-assisted development and systems engineering responsibility are not in conflict; the dividing line is whether you continue to interrogate the semantics after the code is generated. And once you start, “you” become that seam-patcher.
Conclusion
Eries: That’s why you have to look at so much low-level stuff.
Eries: Like how long a single disk write takes, or an RPC call, that kind of thing.
Me: But is anyone really going to give us a raise for this, boss?
Eries: No.
Eries: That’s the contradiction.
References
- Podman: issue #30078, PR #30079
- virtio-win: kvm-guest-drivers-windows #1473, #1475, issue #765; virtio-win-guest-tools-installer #87, #85
- ansible-runner #1306
- xz-dev/pi, intro post, earendil-works/pi issue #6234
- xz-dev/AI-gateway
- FreeBSD procctl(2), Linux PR_SET_CHILD_SUBREAPER, Linux pidfd_send_signal(2), Linux cgroup v2 documentation
- Linux commits: 2021, introducing cgroup.kill, 2025, fixing the race between fork and cgroup.kill, 2026, fixing CLONE_INTO_CGROUP children being killed by mistake
- Steve Marquess: Of Money, Responsibility, and Pride
- DORA’s 2026 analysis of AI
- Further reading: Bengt Holmstrom, Paul Milgrom (1991), Multitask Principal-Agent Analyses: Incentive Contracts, Asset Ownership, and Job Design; Harvard Business School, The Value of Open Source Software