Porting the Moebius 0.2B picture inpainting mannequin to run within the browser with Claude Code

Porting the Moebius 0.2B picture inpainting mannequin to run within the browser with Claude Code

twenty second June 2026

This morning on Hacker Information I noticed Moebius: 0.2B Light-weight Picture Inpainting Framework with 10B-Degree Efficiency, describing a small however efficient inpainting mannequin—a mannequin the place you may mark areas of a picture to take away and the mannequin imagines what ought to fill the area. The launched mannequin required PyTorch and NVIDIA CUDA, however because it described itself as 0.2B I made a decision to attempt to get it working utilizing WebGPU in a browser. TL;DR: I bought it working, and you may attempt the demo at simonw.github.io/moebius-web/. Learn on for the main points.

The completed software

Right here’s a video demo of the completed software:

You’ll be able to open any picture in it (non-square photographs get letterboxed), spotlight areas to take away, click on the “Run inpaint” button and anticipate the mannequin to do its magic.

A parallel agent side-project

My foremost undertaking for right now was touchdown a significant characteristic in Datasette: a UI for creating and altering tables, as a follow-up to the insert and edit rows characteristic I launched final week.

I used to be engaged on that in Codex Desktop (right here’s the PR) and infrequently discovered myself spending 5-10 minutes spinning my fingers ready for it to finish a mid-sized refactor or add the ending touches to a change to the UI.

(An amusing factor about coding brokers is that the more durable an issue is the extra time it’s a must to get distracted when you anticipate them to complete crunching!)

So I made a decision to spin up Claude Code in a terminal window and see how far I may get at porting Moebius to the net.

Some agentic analysis to kick off the undertaking

My first step was to ask common Claude in regards to the feasibility of this undertaking. In Claude.ai, which has the flexibility to clone repos from GitHub:

Clone https://github.com/hustvl/Moebius/ and inform me in the event that they printed the code and weights to run this mannequin wherever

(I hadn’t noticed the hyperlink to the weights but, that’s tucked away within the “Information” part.)

Then:

For Moebius what are the choices for working it proper now – Python and NVIDIA CUDA solely or different choices too?

And:

Muse on the feasibility of porting it to Transformers.js or comparable and working it in a browser

I like telling fashions to “muse on X”, it’s the shortest method I’ve discovered of expressing that I need them to ponder an issue for me with out offering them with a concrete aim.

Right here’s that chat transcript. I copied out the final reply and saved it as analysis.md for Claude Code to learn later.

Claude urged utilizing ONNX Runtime Net on the WebGPU backend—the layer beneath the Transformers.js library I had urged.

That was sufficient to persuade me it was price setting Claude Code unfastened and seeing how far it may get.

I normally begin initiatives like this by gathering as a lot data because the coding agent would possibly want as attainable. Since I didn’t anticipate this undertaking to really work I did every thing in my /tmp folder:

Setting off Claude Code

I created a listing for the remainder of the undertaking and ran git init in that so Claude may begin committing code notes:

I fired up a claude occasion within the /tmp/Moebius folder, the extent above all the analysis supplies I had ready for it. I prompted:

Learn ./moebius-web/analysis.md – your aim is to port this mannequin to ONNX and WebGPU so we will run it immediately in a browser, with a easy UI

Because it began to work I dropped on this follow-up (typos included):

Bulid this in /tmp/Moebius/moebius-web and commit early and infrequently, additionally preserve a notes.md file in there with notes about what you determine alongside the way in which – additionally begin by writing out a plan.md in there and replace that plan as oy work too

I typically ask brokers to maintain notes like this—the tip result’s typically attention-grabbing, each for myself and for the following agent session that touches the identical undertaking. Right here’s what that notes.md file appeared like on the finish of the undertaking.

I kicked it off and went again to my foremost undertaking, checking in often to see how Claude was doing. When it appeared prefer it may need one thing that labored I prompted:

Inform me what URL I can go to in my very own browser to do that

Then I attempted it out in Chrome and pasted some errors (and screenshots of errors) again into Claude Code.

After just a few rounds of this we had one thing that appeared to work! Time to place it on the web so different individuals may use it.

How would we publish this to Hugging Face such that the mannequin weights have been on there and the HTML demo would present up in Hugging Face areas?

Claude Code is aware of use the hf CLI software, so I created a mannequin repo on Hugging Face, then created a token that might write to that repo and dropped it right into a /tmp/Moebius/token.txt file so Claude may use it.

It printed the 1.24GB of transformed ONNX weights to huggingface.co/simonw/Moebius-ONNX for me.

I’d seen different demos load weights into the browser from Hugging Face earlier than, so I knew it was attainable. I made a decision to host my very own frontend code on GitHub Pages, so I stated:

I need to publish the moebius-web folder to GitHub, minus the massive recordsdata (so perhaps minus the fashions/ folder), such that after I activate GitHub Pages for that repo navigating to https://simonw.github.io/moebius-web/ serves the UI

Telling it the ultimate URL was vital in case it wanted to repair the URLs within the demos that it was constructing so they’d work when deployed to manufacturing.

After just a few extra rounds of iteration, in between engaged on my foremost undertaking, we bought to a working, deployed model!

Besides… every time I reloaded the web page it appeared to obtain ~1.3GB of mannequin weights. Browser caching appeared fairly vital for this!

something intelligent we will do with serviceworkers or comparable to assist cache these things? It appears to reload each time, I’m involved that there is likely to be one thing bizarre about the way in which HF redirects work that imply we do not profit from browser caching

I knew that Transformers.js initiatives may deal with this correctly, so I grabbed a duplicate of the Whisper Net demo, dropped it into /tmp/Moebius/whisper-web and stated:

look in /tmp/Moebius/whisper-web (with a subagent) and see how they do that

That undertaking was totally obfuscated, constructed JavaScript recordsdata so I figured utilizing a subagent would keep away from spending the remainder of my top-level token context deciphering these recordsdata.

Claude discovered that it was utilizing caches.open(“transformers-cache”)—the CacheStorage API—and added that to our undertaking.

I’ve shared the complete Claude Code transcript for this undertaking (printed utilizing my claude-code-transcripts software).

What did I study from all of this?

This positively counts as vibe coding: I didn’t have a look at a single line of code from the undertaking, proscribing my enter to testing, suggesting small characteristic enhancements (like a progress bar for the massive file downloads) and pointing the mannequin within the route of examples of how I needed issues to work.

Since I didn’t write any code the quantity I realized in regards to the underlying applied sciences—WebGPU, ONNX, and the Moebius mannequin itself—was very restricted.

As is normally the case with this sort of undertaking a very powerful issues I realized involved what was attainable:

Claude Opus 4.8 is able to changing a PyTorch mannequin to ONNX, publishing the consequence to Hugging Face after which constructing out an internet software and interface that may load and execute that mannequin.
Chrome, Firefox and Safari are all now able to working this sort of mannequin—I attempted it in all three.
The CacheStorage API works with ~1.3GB mannequin recordsdata.
… which implies we will have inpainting as a characteristic of a client-only internet software! (If our customers can tolerate the 1.3GB obtain.)

I felt like I ought to in all probability attempt to study a little bit extra about my undertaking. I fired up Claude.ai and prompted:

Clone https://github.com/simonw/moebius-web/ and use it to show me all in regards to the mannequin and ONNX and the method of changing a mannequin to ONNX and WebGPU and principally every thing I would have to know to be able to absolutely perceive this repo

Right here’s the transcript and the understanding.md Markdown file it created, which I’ve now added to the GitHub repo. I discovered the reason of ONNX significantly enlightening:

ONNX (Open Neural Community Alternate) is a transportable, framework-neutral file format for neural networks. An .onnx file is basically two issues bundled collectively:

A computation graph — a directed graph of nodes, the place every node is an operator (Conv, MatMul, Add, Einsum, Softmax, Collect, Resize, …) wired collectively by named tensors flowing between them. That is the “recipe” for the ahead move.

The weights — the realized parameter tensors (the convolution kernels, the embedding desk, and many others.), saved as initializers in that very same graph.

Crucially, ONNX describes what to compute, abstractly, with out saying how or on what {hardware}. The operator set is versioned by an opset quantity (this repo makes use of opset 18), which pins down precisely which operators exist and what their semantics are.

It seems PyTorch has in-built mechanisms for exporting to ONNX, as seen right here in export_onnx.py:

torch.onnx.export(
dec, (lat,), dec_path, opset_version=args.opset,
input_names=[“latent”], output_names=[“image”],
dynamic_axes={“latent”: {0: “B”}, “picture”: {0: “B”}},
)

Claude additionally included a helpful glossary and an only-slightly-broken ASCII-art diagram displaying how the mannequin pipeline suits collectively.