publications

A. Feature absorption occurs when one sparse autoencoder atom carries several concepts. For penalties based on pairwise feature overlaps raised to p, curvature at an absorbed solution changes sign at p = 2 in the orthogonal idealization. Below 2, sufficient penalty removes the local minimum; above 2, no such threshold exists. Correlated features make p = 2 weakly destabilizing, offering a practical balance against reconstruction cost. On Pythia-160M and Gemma-2-2B, full absorption events fall 89%, which is not explained by reduced feature firing. The effect can reverse in independent-threshold architectures. This identifies when the penalty works, rather than a universal solution to absorption.

A. Models actively learn to use it. Targeted ablations degrade performance in proportion to each head’s shift-invariant signal, and similar structure appears in models trained with absolute positional embeddings. Its strongest observed role is in offset-structured retrieval and positional routing; how to train it for richer semantic invariances remains an open question.

A. Across four models and 230 twenty-turn sessions, probes recover constraint identity before many checker-failed answers. That establishes decodable information while the instruction remains in context, not functional use. At unit strength, combined activation-direction and attention steering improves checker pass rates by 21.7–29.8 percentage points over baseline and beats either alone, but its gains are subadditive. Across the full grid of steering strengths, adding activation-direction steering to attention loses more successes than it gains in 24 of 36 comparisons. These are exploratory policy comparisons with imperfect checks and independently generated histories; they do not identify the natural cause of compliance decay.

A. Across 37 personas and three models, mixtures from a 60-trait dictionary use a median of six traits and capture roughly half the persona direction’s length. Adding and removing these mixtures changes recognition more than matched random edits; related mixtures also affect sycophancy and refusal. Broad persona resemblance is easier to recover than distinctions between similar characters. Because the trait dictionary is correlated and linearly dependent, these recipes are not uniquely identified psychological primitives, and the joint interventions do not isolate each trait’s causal contribution.

A. No. MorphSAE gives each named dictionary a TopK quota, guaranteeing returned indices rather than positive activations or semantic ownership. The optional cross-dictionary loss reduces its sampled overlap statistic, but quota-only comparisons show no reliable reconstruction advantage, and none of the 12 unsupervised evaluations passes the study’s full set of component-recovery checks. A supervised synthetic copy task does yield useful named components, yet a named model with one shared budget performs similarly. The positive result supports that supervised decomposition recipe; it does not attribute the benefit to reserved slots or establish automatic component recovery.

A. Not in the tested LLaVA-OneVision-0.5B setting. Training all seven non-empty subsets of the vision encoder, projector, and language model on 13 geometry tasks gives multiple single-module repairs for about 44–45% of base-model errors across two seeds. Joint training mostly reallocates existing repairs. In one case studied in depth, vision-only and language-only adaptation use different causal routes that meet at one late decision state. Where parameters change, where the repair is computed, and which module carries it can therefore differ. Repair availability remains specific to the model, training recipe, and answer-scoring protocol.

A. Not necessarily. Across architecture-matched 13B models, English bias questions and their Japanese and Korean translations can have similar representation geometry while internal language preferences vary with training history. English-hub signals weaken after language-specific adaptation and under balanced bilingual training. Crucially, activation patching of English-hub states does not transplant the corresponding bias behavior. The benchmark scores therefore capture language-specific stereotype preferences, rather than simply reflecting English routing. High geometric alignment alone does not establish language-agnostic representations of bias or make the underlying constructs directly comparable across languages.

A. No. We construct a residual-stream direction from paired stereotype and anti-stereotype items, then compare adding it with projecting it out at the decision token. Adding the direction increases stereotypical choices; removal mainly changes confidence, with few decisions flipping because most items lie far from the decision boundary. Component rankings offer limited guidance: predicted signs are unreliable, and one in seven to one in four top-ranked components push against the intended edit. A direction that can amplify a stereotype does not necessarily provide an effective way to erase it.

A. Usually earlier. Across 150 Pythia training trajectories spanning three model scales and five tasks, spectral measurements show that structural reorganization precedes measurable capability in 94% of runs, often by thousands of steps. Ablations at pre-emergence checkpoints show that already-prominent components carry much of the capability that appears later. Whether behavioral emergence looks sharper than structural change depends on the task: subject-verb agreement and induction show sharper behavioral transitions, while factual recall and docstrings show the two changing together. The timing gap is widespread in these experiments, but the shape of emergence is task-dependent.

A. In a controlled 7-by-7 Sokoban setting, eight supervision conditions share a 50.5M-parameter Transformer and action-training corpus. Across three seeds, supervising state, action effects, and goal occupancy raises complete-state recovery from 20.8% to 95.5%, while held-out 32–60-step puzzle solving changes from 74.5% to 73.5%. Supervision helps continuation after supplied detours, but doubling training shrinks that advantage on matched handovers. This separates recoverable state information from its behavioral utility; it does not show that world knowledge is unnecessary or establish the mechanism behind the supervision benefit.

A. Two contracts, declared before any method is proposed. The Research Contract fixes what is being explained, to whom, at what intervention unit, within what evaluation bounds, under what adversarial conditions, and with what audit trail. The Agent Contract states the behavior the system was supposed to produce, as obligations, permissions, prohibitions, violation criteria, and accountability chains, which is the baseline a deviation gets explained against. Without both, a reported score is not comparable across papers and cannot be carried into a deployment decision.

A. A question-specific switching model calibrated on eight graph–adversary configurations predicts capture on 52 other configurations with mean error 0.012 in the tested six-agent setup. Graph rankings transfer better than absolute capture probabilities, which need recalibration under new questions or attacks. When the adversary chooses its worst-case position, concentrating links on one agent is especially risky; with random placement, rankings can reverse. The most accurate topology also depends on the initial answer distribution and agents’ majority-following behavior. The study thus supports conditional topology selection, not a universally safest graph.

A. Under explicit curvature and stationarity assumptions, two-sided loss bounds predict when the smaller task vector’s task suffers more. GLUE experiments support this ordering under unequal training exposure, while matching exposure compresses norm differences and removes reliable norm-based ordering. On four tested pairs in two model families, shrinking only the larger vector protects the smaller task better than uniform scaling; equalizing at fixed total norm can instead stretch the smaller vector and harm both tasks. Shrinkage can cost the larger task, and the bounds are conditional rather than certified capacity limits for real networks.

A. CLASP adds a budget controller to an existing router. Offline, it uses cached cost and latency measurements to exclude actions that exceed the allowed violation rate and calibrates two penalty weights. Online, it subtracts weighted cost and latency predictions from the router’s quality score before selecting a topology and model tier. Changing a budget requires updating the mask and weights rather than retraining the router. The weights also adapt to observed deployment outcomes, so compliance depends on calibration and monitoring as conditions change.

A. We present PolicyLLM, which offline compiles policy documents into a decision graph and uses the Z3 SMT solver to find contradictions among the compiled rules, settling them by an explicit priority ordering. At inference time it both steers generation with policy-derived scaffolding and checks the response against that bundle, routing it to pass, auto-correct, regenerate, or escalate, and appending the decision to a hash-chained log. The solver certifies the rules it was handed, not that those rules were read correctly out of the policy documents, so a response it cannot verify goes to a human reviewer rather than to the user.

A. Yes, on the Seoul apartment transactions studied here. We aggregate records by apartment complex and combine complex attributes with macroeconomic indicators to handle sparse, irregular observations. Time-series attention follows each complex over a twelve-month window, while geographic attention links neighboring complexes in the same month. The model outperforms the statistical, machine-learning, and time-series deep-learning baselines compared on this dataset. This supports direct transaction-price prediction in the tested setting; it does not establish performance in other cities or market conditions.

A. We assemble 878,928 Seoul transactions from 2015 to 2020 with 42 variables, where earlier studies used a few thousand records and chose their variables by hand. After addressing multicollinearity, we compare five variable-selection methods across four models. Tested on 2021, the model recovers the spatial pattern of expensive apartment complexes but systematically underestimates prices. The larger dataset enables a systematic comparison, but price-level error remains a practical limitation.