\n<\/p>\n
twenty second June 2026<\/p>\n
This morning on Hacker Information I noticed Moebius: 0.2B Light-weight Picture Inpainting Framework with 10B-Degree Efficiency, describing a small however efficient inpainting mannequin\u2014a mannequin the place you may mark areas of a picture to take away and the mannequin imagines what ought to fill the area. The launched mannequin required PyTorch and NVIDIA CUDA, however because it described itself as 0.2B I made a decision to attempt to get it working utilizing WebGPU in a browser. TL;DR: I bought it working, and you may attempt the demo at simonw.github.io\/moebius-web\/. Learn on for the main points.<\/p>\n
Right here\u2019s a video demo of the completed software:<\/p>\n
You’ll be able to open any picture in it (non-square photographs get letterboxed), spotlight areas to take away, click on the \u201cRun inpaint\u201d button and anticipate the mannequin to do its magic.<\/p>\n
My foremost undertaking for right now was touchdown a significant characteristic in Datasette: a UI for creating and altering tables, as a follow-up to the insert and edit rows characteristic I launched final week.<\/p>\n
I used to be engaged on that in Codex Desktop (right here\u2019s the PR) and infrequently discovered myself spending 5-10 minutes spinning my fingers ready for it to finish a mid-sized refactor or add the ending touches to a change to the UI.<\/p>\n
(An amusing factor about coding brokers is that the more durable an issue is the extra time it’s a must to get distracted when you anticipate them to complete crunching!)<\/p>\n
So I made a decision to spin up Claude Code in a terminal window and see how far I may get at porting Moebius to the net.<\/p>\n
My first step was to ask common Claude in regards to the feasibility of this undertaking. In Claude.ai, which has the flexibility to clone repos from GitHub:<\/p>\n
\nClone https:\/\/github.com\/hustvl\/Moebius\/ and inform me in the event that they printed the code and weights to run this mannequin wherever<\/p>\n<\/blockquote>\n
(I hadn\u2019t noticed the hyperlink to the weights but, that\u2019s tucked away within the \u201cInformation\u201d part.)<\/p>\n
Then:<\/p>\n
\nFor Moebius what are the choices for working it proper now – Python and NVIDIA CUDA solely or different choices too?<\/p>\n<\/blockquote>\n
And:<\/p>\n
\nMuse on the feasibility of porting it to Transformers.js or comparable and working it in a browser<\/p>\n<\/blockquote>\n
I like telling fashions to \u201cmuse on X\u201d, it\u2019s the shortest method I\u2019ve discovered of expressing that I need them to ponder an issue for me with out offering them with a concrete aim.<\/p>\n
Right here\u2019s that chat transcript. I copied out the final reply and saved it as analysis.md for Claude Code to learn later.<\/p>\n
Claude urged utilizing ONNX Runtime Net on the WebGPU backend\u2014the layer beneath the Transformers.js library I had urged.<\/p>\n
That was sufficient to persuade me it was price setting Claude Code unfastened and seeing how far it may get.<\/p>\n
I normally begin initiatives like this by gathering as a lot data because the coding agent would possibly want as attainable. Since I didn\u2019t anticipate this undertaking to really work I did every thing in my \/tmp folder:<\/p>\n
cd<\/span> \/tmp
\nmkdir Moebius
\ncd<\/span> Moebius
\n#<\/span> Seize the Moebius python code<\/span>
\ngit clone https:\/\/github.com\/hustvl\/Moebius
\n#<\/span> And the mannequin weights (Claude figured this out):<\/span>
\nGIT_LFS_SKIP_SMUDGE=0 git clone
\n https:\/\/huggingface.co\/hustvl\/Moebius Moebius-weights
\n#<\/span> Lastly a few libraries we’d use:<\/span>
\ngit clone https:\/\/github.com\/huggingface\/transformers.js
\ngit clone https:\/\/github.com\/microsoft\/onnxruntime<\/div>\nSetting off Claude Code<\/h4>\n
I created a listing for the remainder of the undertaking and ran git init in that so Claude may begin committing code notes:<\/p>\n
mkdir \/tmp\/Moebius\/moebius-web
\ncd<\/span> \/tmp\/Moebius\/moebius-web
\ngit init
\n#<\/span> Copy in that analysis.md from earlier<\/span>
\ngit add analysis.md
\ngit commit -m “<\/span>Preliminary analysis by Claude Opus 4.8“<\/span><\/span><\/div>\nI fired up a claude occasion within the \/tmp\/Moebius folder, the extent above all the analysis supplies I had ready for it. I prompted:<\/p>\n
\nLearn .\/moebius-web\/analysis.md – your aim is to port this mannequin to ONNX and WebGPU so we will run it immediately in a browser, with a easy UI<\/p>\n<\/blockquote>\n
Because it began to work I dropped on this follow-up (typos included):<\/p>\n
\nBulid this in \/tmp\/Moebius\/moebius-web and commit early and infrequently, additionally preserve a notes.md file in there with notes about what you determine alongside the way in which – additionally begin by writing out a plan.md in there and replace that plan as oy work too<\/p>\n<\/blockquote>\n
I typically ask brokers to maintain notes like this\u2014the tip result’s typically attention-grabbing, each for myself and for the following agent session that touches the identical undertaking. Right here\u2019s what that notes.md file appeared like on the finish of the undertaking.<\/p>\n
I kicked it off and went again to my foremost undertaking, checking in often to see how Claude was doing. When it appeared prefer it may need one thing that labored I prompted:<\/p>\n
\nInform me what URL I can go to in my very own browser to do that<\/p>\n<\/blockquote>\n
Then I attempted it out in Chrome and pasted some errors (and screenshots of errors) again into Claude Code.<\/p>\n
After just a few rounds of this we had one thing that appeared to work! Time to place it on the web so different individuals may use it.<\/p>\n
\nHow would we publish this to Hugging Face such that the mannequin weights have been on there and the HTML demo would present up in Hugging Face areas?<\/p>\n<\/blockquote>\n
Claude Code is aware of use the hf CLI software, so I created a mannequin repo on Hugging Face, then created a token that might write to that repo and dropped it right into a \/tmp\/Moebius\/token.txt file so Claude may use it.<\/p>\n
It printed the 1.24GB of transformed ONNX weights to huggingface.co\/simonw\/Moebius-ONNX for me.<\/p>\n
I\u2019d seen different demos load weights into the browser from Hugging Face earlier than, so I knew it was attainable. I made a decision to host my very own frontend code on GitHub Pages, so I stated:<\/p>\n
\nI need to publish the moebius-web folder to GitHub, minus the massive recordsdata (so perhaps minus the fashions\/ folder), such that after I activate GitHub Pages for that repo navigating to https:\/\/simonw.github.io\/moebius-web\/ serves the UI<\/p>\n<\/blockquote>\n
Telling it the ultimate URL was vital in case it wanted to repair the URLs within the demos that it was constructing so they’d work when deployed to manufacturing.<\/p>\n
After just a few extra rounds of iteration, in between engaged on my foremost undertaking, we bought to a working, deployed model!<\/p>\n
Besides… every time I reloaded the web page it appeared to obtain ~1.3GB of mannequin weights. Browser caching appeared fairly vital for this!<\/p>\n
\nsomething intelligent we will do with serviceworkers or comparable to assist cache these things? It appears to reload each time, I’m involved that there is likely to be one thing bizarre about the way in which HF redirects work that imply we do not profit from browser caching<\/p>\n<\/blockquote>\n
I knew that Transformers.js initiatives may deal with this correctly, so I grabbed a duplicate of the Whisper Net demo, dropped it into \/tmp\/Moebius\/whisper-web and stated:<\/p>\n
\nlook in \/tmp\/Moebius\/whisper-web (with a subagent) and see how they do that<\/p>\n<\/blockquote>\n
That undertaking was totally obfuscated, constructed JavaScript recordsdata so I figured utilizing a subagent would keep away from spending the remainder of my top-level token context deciphering these recordsdata.<\/p>\n
Claude discovered that it was utilizing caches.open(“transformers-cache”)\u2014the CacheStorage API\u2014and added that to our undertaking.<\/p>\n
I\u2019ve shared the complete Claude Code transcript for this undertaking (printed utilizing my claude-code-transcripts software).<\/p>\n
What did I study from all of this?<\/h4>\n
This positively counts as vibe coding: I didn\u2019t have a look at a single line of code from the undertaking, proscribing my enter to testing, suggesting small characteristic enhancements (like a progress bar for the massive file downloads) and pointing the mannequin within the route of examples of how I needed issues to work.<\/p>\n
Since I didn\u2019t write any code the quantity I realized in regards to the underlying applied sciences\u2014WebGPU, ONNX, and the Moebius mannequin itself\u2014was very restricted.<\/p>\n
As is normally the case with this sort of undertaking a very powerful issues I realized involved what was attainable:<\/p>\n
Claude Opus 4.8 is able to changing a PyTorch mannequin to ONNX, publishing the consequence to Hugging Face after which constructing out an internet software and interface that may load and execute that mannequin.
\nChrome, Firefox and Safari are all now able to working this sort of mannequin\u2014I attempted it in all three.
\nThe CacheStorage API works with ~1.3GB mannequin recordsdata.
\n… which implies we will have inpainting as a characteristic of a client-only internet software! (If our customers can tolerate the 1.3GB obtain.)<\/p>\nI felt like I ought to in all probability attempt to study a little bit extra about my undertaking. I fired up Claude.ai and prompted:<\/p>\n
\nClone https:\/\/github.com\/simonw\/moebius-web\/ and use it to show me all in regards to the mannequin and ONNX and the method of changing a mannequin to ONNX and WebGPU and principally every thing I would have to know to be able to absolutely perceive this repo<\/p>\n<\/blockquote>\n
Right here\u2019s the transcript and the understanding.md Markdown file it created, which I\u2019ve now added to the GitHub repo. I discovered the reason of ONNX significantly enlightening:<\/p>\n
\nONNX (Open Neural Community Alternate) is a transportable, framework-neutral file format for neural networks. An .onnx file is basically two issues bundled collectively:<\/p>\n
A computation graph \u2014 a directed graph of nodes, the place every node is an operator (Conv, MatMul, Add, Einsum, Softmax, Collect, Resize, \u2026) wired collectively by named tensors flowing between them. That is the \u201crecipe\u201d for the ahead move.<\/p>\n
The weights \u2014 the realized parameter tensors (the convolution kernels, the embedding desk, and many others.), saved as initializers in that very same graph.<\/p>\n
Crucially, ONNX describes what to compute, abstractly, with out saying how or on what {hardware}. The operator set is versioned by an opset quantity (this repo makes use of opset 18), which pins down precisely which operators exist and what their semantics are.<\/p>\n<\/blockquote>\n
It seems PyTorch has in-built mechanisms for exporting to ONNX, as seen right here in export_onnx.py:<\/p>\n