diffusers.js WebGPU demo: Revolutionizing Image Generation
The diffusers.js WebGPU demo is a powerful tool that brings image generation to the web. It requires specific browser settings for optimal performance. Users need the latest Chrome with the 'Experimental WebAssembly' flag and 'Experimental WebAssembly JavaScript Promise Integration (JSPI)' flag enabled. Additionally, Chrome Canary 119 or newer is necessary, with Chrome Canary 121 or higher for FP16 support. A browser with WebGPU support is also a prerequisite.
This demo comes with a range of features. It utilizes diffusers.js, Prompt, and Negative Prompt for creating images. The number of inference steps is customizable, and the Guidance Scale controls the similarity of the generated image to the prompt. The Seed feature creates initial random noise.
Once the model is downloaded by pressing the provided button, it is stored in the browser cache, and all settings become editable. The demo also addresses common issues. For example, if one encounters a protobuf parsing failed error, they can open DevTools, go to Application -> Storage, and press 'Clear site data'. If the sbox_fatal_memory_exceeded error occurs, it might indicate insufficient RAM to run the software. Reloading the tab or browser can be attempted as a solution.
The development of this demo involved porting StableDiffusionPipeline from python to JS and patching onnxruntime and the emscripten+binaryen WebAssembly compiler toolchain to support allocating and using >4GB memory. It also required working with the WebAssembly spec and the V8 engine.
In conclusion, the diffusers.js WebGPU demo offers a unique and powerful image generation experience on the web, with the potential to transform the way images are created and used in various applications.
