Lightweight Safety Classification Using Pruned Language Models — HN submission

Thu, 19 Dec 2024 17:49:07 GMT

Lightweight Safety Classification Using Pruned Language Models

Layer Enhanced Classification (LEC) is a novel technique that outperforms current industry leaders like GPT-4o, LlamaGuards 1 and 8B, and deBERTa v3 Prompt Injection v2 for content safety and prompt injection tasks. We prove that the intermediate hidden layers in transformers are robust feature extractors for text classification. On content safety, LEC models achieved a 0.96 F1 score vs GPT-4o's 0.82 and Llama Guard 8B's 0.71.The LEC models were able to outperform the other models with only 15 training examples for binary classification and 50 examples for multi-class classification across 66…

19 points; 3 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Lightweight Safety Classification Using Pruned Language Models · HackerPapers discussions

Lightweight Safety Classification Using Pruned Language Models — HN submission