Mistral 7B · HackerPapers discussions

Mistral 7B — mentioned by woadwarrior01

Sun, 17 Dec 2023 09:39:07 GMT

One thing that's worth mentioning about llama.cpp wrappers like ollama, LM Studio and Faraday is that they don't yet support[1] sliding window attention, and instead use vanilla causal attention from llama2. As noted in the Mistral 7B paper[2], SWA has some benefits in terms of attention span over regular causal attention. Disclaimer: I have a competing universal macOS/iOS app[3] that does support SWA with Mistral models (using mlc-llm). [1]: https://github.com/ggerganov/llama.cpp/issues/3377 [2]: https://arxiv.org/abs/2310.06825 [3]: https://apps.apple.com/us/app/private-llm/id6448106860

Paper and discussion history. A mention is not an endorsement.

Mistral 7B — mentioned by lukasga

Fri, 20 Oct 2023 15:42:05 GMT

Original paper: https://arxiv.org/abs/2310.06825 More digestible announcement: https://mistral.ai/news/announcing-mistral-7b/

Paper and discussion history. A mention is not an endorsement.

Mistral 7B — HN submission

Wed, 11 Oct 2023 09:56:39 GMT

Mistral 7B

267 points; 123 discussion replies on HN. Paper and source history. A submission is not an endorsement.