Cross-Domain Content Generation with Domain-Specific Small Language Models · HackerPapers discussions

Cross-Domain Content Generation with Domain-Specific Small Language Models — mentioned by ankit219

Wed, 11 Jun 2025 06:29:39 GMT

I see this and immediately relived the last two years of the journey. I think some of the mental model that helped me might help the community too. What people expect from finetuning is knowledge addition. You want to keep the styling[1] of the original model, just add new knowledge points that would help your task. In context learning is one example of how this works well. Just that even here, if the context is out of distribution, a model does not "understand" it and would produce guesswork. When it comes to LoRA or PEFT or adapters, it's about style transfer. And if you focus on a specific…

Paper and discussion history. A mention is not an endorsement.

Cross-Domain Content Generation with Domain-Specific Small Language Models — mentioned by ankit219

Tue, 18 Mar 2025 15:54:07 GMT

We did a research in this regard: https://arxiv.org/abs/2409.17171 The idea is to create bespoke models for org at 90% lower compute. (we cheat a little, where we use an underlying open source model and freeze the existing knowledge). Currently building a specialized model + agent for bioresearch labs. we hope to bring down the costs in long term so that these evolved into continuous learning systems that can be updated everyday. The idea is exactly this: model customization + infra gives you the advantages Prompting + tooling cannot.

Paper and discussion history. A mention is not an endorsement.

Cross-Domain Content Generation with Domain-Specific Small Language Models — HN submission

Fri, 11 Oct 2024 13:45:24 GMT

Cross-Domain Content Generation with Domain-Specific Small Language Models

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.