Cross-Care: Assessing the Healthcare Implications of Pre-training Data on Language Model Bias — mentioned by saaas23

Sat, 11 May 2024 15:31:40 GMT

We made this (www.crosscare.net) to visualize what disease+demographic information are in LLM pre-training data, and how they propagate to downstream bias! (https://arxiv.org/abs/2405.05506) Things we found that are kinda interesting: 1. Models have no idea about the true prevalence (fair they are not fact-check/retrieval machines anyway) 2. models fit to pre-training data's co-occurances 3. Alignment methods deepen these biases 4. Base models SFT/DPO on certain language datasets (usually English only) will only impact their understanding of these concepts in English

Paper and discussion history. A mention is not an endorsement.

Cross-Care: Assessing the Healthcare Implications of Pre-training Data on Language Model Bias · HackerPapers discussions

Cross-Care: Assessing the Healthcare Implications of Pre-training Data on Language Model Bias — mentioned by saaas23