I'm working on a similar project but vision only. You run a companion app that takes care of a Qwen3.5 vision. For the prototype, I’ve managed cropping and tagging one object at a time in less than 4s (p50) usually 2 to 3s via Wi-Fi. I’ve also managed object permanence across app starts, and now I’m working on tag corrections (rare, but needed, also bc you might want to name your laptop MacBook instead of laptop, etc.) and I’ll then work on a mode to not crop and tag multiple objects at a time along with a search bar for fuzzy search of an object.... not sure where im going with it. no model conversation, and I've dropped YOLO to a fallback when then companion is not available for some reason. ill drop by and leave a RC once i'm done if i can remember.