Post-Training Small Language Models for Editorial Compliance Checking: How Output Structure Inverts the Ranking of Alignment Methods

Compliance checking is the process of ensuring that content adheres to a set of editorial, legal and stylistic guidelines. Frontier language models can automate it, but a publisher cannot always route its incoming stream through one: unpublished commercial content raises data-residency obligations, on-premise policies forbid third-party inference, and at production volume the per-token bill becomes a recurring cost. The alternative is a small open-weight model fine-tuned on data generated by a larger one and served locally. This thesis asks which post-training method that student should receive, and finds that the answer depends on a variable the literature treats as fixed: the shape of the output the model is asked to produce. We compare six methods, Supervised Fine-Tuning (SFT), DPO, SimPO, ORPO, KTO and GRPO, including stacked variants, across six models from three families in the 1.7B–8B range, on a real editorial compliance task with 390 human labels and a single consumer GPU. Preference data comes from an augmentation pipeline in which large models rewrite real articles into each compliance class while paraphrasing away from the literal trigger phrases the guidelines enumerate, with labels assigned by a disjoint model jury and human adjudication. When the model must emit a single JSON containing both the fields copied from the input article and the compliance verdict, SFT is the best method and every preference algorithm trails it by three to four points of F1 while beating it on structural similarity to the gold output. We trace this to signal dilution: on a completion of one to two thousand tokens, chosen and rejected differ in style,formatting and length, and in only the two or three tokens that carry the verdict. Neither data-level remedy we test closes the gap, since normalizing the pairs pushes the rejected completion off-policy and on-policy pair mining improves over vanilla DPO without reaching SFT. Decomposing the task, so that deterministic code copies the extraction fields and the model produces only the analysis, inverts the ranking: preference optimization gains roughly three points of F1 over SFT and ten points of structural similarity over the base policy, concentrated on the rare class that SFT leaves untouched,at an unchanged served output. Finally, the recipe transfers unevenly, and we organize the exceptions into three regimes indexed by the state of the base policy rather than by parameter count: a healthy policy, a degenerate one on which no preference objective can define a signal, and one carrying a structural zero on a class,which every reference-anchored objective inherits. The regime is identifiable from a per-class confusion matrix before any training. We close by pricing the annotation budget, showing that the aggregate metric saturates after a few hundred training articles while the class carrying the legal exposure does not. The resulting system screens roughly a thousand articles a day for an Italian digital publisher and has reduced manual editorial review by about seventy percent.