https://hackerpaper.kaipfister.de/p/2404.05719/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMThttps://news.ycombinator.com/item?id=42707559urn:hackerpapers:mention:2404.05719:42707559Wed, 15 Jan 2025 05:12:57 GMT<p>We finetune our own VLMs for this -- unfortunately prefer not to share which ones we use specifically! ClickClickClick looks awesome, have you heard of FerretUI (https://arxiv.org/pdf/2404.05719)? Pretty similar idea.</p><p><a href="https://hackerpaper.kaipfister.de/p/2404.05719/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.CVhttps://news.ycombinator.com/item?id=40065255urn:hackerpapers:submission:2404.05719:40065255Wed, 17 Apr 2024 14:35:42 GMT<p>Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs</p><p></p><p>1 point; 0 discussion replies on HN. <a href="https://hackerpaper.kaipfister.de/p/2404.05719/">Paper and source history</a>. A submission is not an endorsement.</p>cs.CVhttps://news.ycombinator.com/item?id=40007521urn:hackerpapers:mention:2404.05719:40007521Thu, 11 Apr 2024 22:35:43 GMT<p>They recently published a paper where an on device model has UI awareness. https://arxiv.org/abs/2404.05719 Siri is about to get swole.</p><p><a href="https://hackerpaper.kaipfister.de/p/2404.05719/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.CVhttps://news.ycombinator.com/item?id=39984252urn:hackerpapers:submission:2404.05719:39984252Tue, 09 Apr 2024 21:06:05 GMT<p>Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs</p><p></p><p>1 point; 1 discussion reply on HN. <a href="https://hackerpaper.kaipfister.de/p/2404.05719/">Paper and source history</a>. A submission is not an endorsement.</p>cs.CVhttps://news.ycombinator.com/item?id=39983879urn:hackerpapers:mention:2404.05719:39983879Tue, 09 Apr 2024 20:27:45 GMT<p>I haven&#x27;t read through it yet, but there&#x27;s FerretUI from Apple (mobile-specific, but I think a lot of learnings are generic) https://arxiv.org/abs/2404.05719</p><p><a href="https://hackerpaper.kaipfister.de/p/2404.05719/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.CVhttps://news.ycombinator.com/item?id=39982962urn:hackerpapers:mention:2404.05719:39982962Tue, 09 Apr 2024 19:04:21 GMT<p>Google claims SoTA but it appears that, according to Apple, they may already be out of date: https://arxiv.org/abs/2404.05719</p><p><a href="https://hackerpaper.kaipfister.de/p/2404.05719/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.CVhttps://news.ycombinator.com/item?id=39978036urn:hackerpapers:submission:2404.05719:39978036Tue, 09 Apr 2024 10:41:03 GMT<p>Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs</p><p></p><p>12 points; 1 discussion reply on HN. <a href="https://hackerpaper.kaipfister.de/p/2404.05719/">Paper and source history</a>. A submission is not an endorsement.</p>cs.CVhttps://news.ycombinator.com/item?id=39977671urn:hackerpapers:submission:2404.05719:39977671Tue, 09 Apr 2024 09:24:58 GMT<p>Apple Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs</p><p></p><p>53 points; 7 discussion replies on HN. <a href="https://hackerpaper.kaipfister.de/p/2404.05719/">Paper and source history</a>. A submission is not an endorsement.</p>cs.CV