This study explores the behavior of large language models under near-unconstrained conditions by removing chat templates and explicit instructions. The researchers demonstrate that different model families exhibit distinct semantic preferences and knowledge priors, often defaulting to specific topics like programming or literature even with neutral prompts. These findings suggest that raw model generations reveal underlying training distributions and systemic biases that standard conditional benchmarks often overlook.
Key points
Language models possess inherent semantic priors that manifest when system prompts and chat templates are removed.
Different model families gravitate toward distinct topic distributions, revealing variations in training data influence.
Near-unconstrained generation can expose training data leakage, including potential personally identifiable information.
Standard conditional benchmarks are insufficient for auditing model behavior as they often mask a model's natural generative tendencies.