Discussion Thread

this is the privacy angle people sleep on. rag on device means the retrieval index never leaves the machine, which changes the threat model completely. small models lose some quality but for docs summarization and search over a private corpus, wasm inference is honestly good enough already