cs.LG · 2026-11-06 · No. 165

Machine Learning, 2026-11-06.

76 new papers in cs.LG. Titles, authors, abstracts. Links to arXiv. Want this in your inbox every morning? Subscribe →

01 — The papers

76 entries
  1. 01

    User Embeddings are Superpositions of Interpretable Behavioral Modes

    Jonathan Kozaczuk, Zheng Dong, Noyan Evirgen, Keivan Majidi, Eddie K. H. Ng, Chris Porter, Qi Zhao, Jia Geng,...

    cs.LG

    User embeddings are central components of modern recommendation, personalization, and behavioral modeling systems. Though these embeddings can compactly encode rich behavioral information, they are inherently dense and opaque, limiting interpretability. To address this gap, we argue that user embeddings can be understood as sparse superpositions of interpretable behavioral modes. This structure is directly predicted by generative models of...

    arxiv.org/abs/2611.08487 · PDF

  2. 02

    XStack: A Benchmark for ML Repo Migration

    Linrong Cai, Taiming Lu, Haozhe Chen, Zhuang Liu

    cs.LG · cs.AI

    ML code should run where researchers choose, not where it was first written. Today, machine learning code mainly lives in two ecosystems, PyTorch on NVIDIA GPUs and JAX on Google TPUs. Migrating repositories between them means building and testing across frameworks and hardware. This migration is (1) a practical need, since it unlocks different hardware for researchers, and (2) a natural benchmark: it involves different frameworks and...

    arxiv.org/abs/2611.08472 · PDF

  3. 03

    Same Behavior, Different Mechanism: What Finite Experiments Can Certify About Integrated Information

    Linqian Wu, Diaoxuan He

    cs.LG · cs.AI · q-bio.NC

    Interpretability, AI auditing and assessments of machine consciousness increasingly rest on mechanism-level quantities: functionals of a system's causal transition law that describe how it computes, not what it outputs. Experiments see that law only through finite, partial and noisy interventions. We ask when such a quantity is identifiable from an experiment's access, continuous in the mechanism and certifiable from finite data, and answer...

    arxiv.org/abs/2611.08469 · PDF

  4. 04

    Towards interpreting any residual stream direction by scaling activation inversion

    Celeste De Schamphelaere, Juan J. Vazquez, Ari Spiesberger, Tomáš Gavenčiak, Gavin Leech, Euan Ong, Jiahai Feng

    cs.LG · cs.AI

    To interpret a direction in a language model's residual stream, we can look for text that strongly activates it. But when such text is found by scanning a corpus, it is limited to what that corpus contains. We develop Maximally-Activating Example Metamodels (MAEMs), language models which map residual stream directions from a subject model to text exemplars that strongly activate those directions. Our objective is to maximise the cosine...

    arxiv.org/abs/2611.08465 · PDF

  5. 05

    Adversarial Training with Large Step Sizes: Implicit Bias, Optimization, and Sharpness

    Yi Feng, Andrea Paudice

    cs.LG · math.OC

    Adversarial Training (AT) is an important method for improving model robustness. We study the dynamics of deterministic Gradient Descent (GD) in AT. Most existing works in this direction focus on the small or vanishing step sizes regime, where GD resembles a continuous-time flow. However, such step sizes can make training inefficient and are rarely used in practice. In this paper, we study the AT dynamics of GD with large constant step sizes....

    arxiv.org/abs/2611.08457 · PDF

  6. 06

    Unpacking the Compute Efficiency of Strong-to-Weak OPD against GRPO

    Zeyu Leo Liu, Yang Li, Bo Pang, Emre Can Acikgoz, Xuan-Phi Nguyen, Anurag Koul, Srijan Bansal, Erik Nijkamp, Shafiq...

    cs.LG · cs.CL

    On-policy distillation (OPD) trains a student on its own rollouts under a frozen teacher's per-token supervision. Does this dense signal reduce the training FLOPs needed to reach a target accuracy compared with reinforcement learning? We systematically compare vanilla reverse-KL OPD with Group Relative Policy Optimization (GRPO) across task difficulty, student and teacher size, domain, and optimization choices, measuring held-out accuracy...

    arxiv.org/abs/2611.08453 · PDF

  7. 07

    A Hidden Markov Preference Model for Online Prediction of Adversary Actions

    Angel Daruna, Michael Cogswell, Tixiao Shan, Yi Yao, Han-Pang Chiu, Ajay Divakaran

    cs.LG

    A defender anticipating an attacker needs a probability distribution over the attacker's next action from information available before that action occurs. Two structures complicate this: the attacker's behavioral regime persists and can switch within an episode, and each action combines an operation with state-dependent arguments whose feasible values change over time. We capture both with a hidden Markov model whose latent state is the...

    arxiv.org/abs/2611.08445 · PDF

  8. 08

    Data-driven discovery of intrinsic Lie point symmetries of differential equation systems

    Ilker Gurcan, Kadir Ustun, Ramazan Gokberk Cinbis

    cs.LG · math.DG · math.DS · math.GR · math.NA

    Discovering continuous symmetries directly from differential-equation data can reveal geometric structure for model reduction and scientific machine learning without symbolic determining equations or a closed-form representation of the governing system. We formulate intrinsic Lie point-symmetry discovery as an operator-theoretic problem on sampled jet-space data. Admissible generators are identified from a sampled determining operator and...

    arxiv.org/abs/2611.08421 · PDF

  9. 09

    On the limitations of molecular representations

    Acer Blake, Fergus Imrie, Charlotte M. Deane

    cs.LG

    Molecular representations such as fingerprints and learned embeddings play a central role in cheminformatics and small-molecule machine learning. They serve as the primary input for a diverse array of tasks, including clustering, dataset splitting, and virtual screening, yet their evaluation is often limited to downstream model performance on narrow benchmark tasks. We carried out a systematic investigation of the quality of molecular...

    arxiv.org/abs/2611.08420 · PDF

  10. 10

    Spectral and Subspace Dynamics of LoRA in Supervised Fine-Tuning and Reinforcement Learning

    Chenruo Liu, Zihan Wang, Yijun Dong, Qi Lei

    cs.LG · stat.ML

    Low-rank adaptation (LoRA) is commonly used in both supervised fine-tuning (SFT) and reinforcement learning (RL), often under similar optimizer settings. Yet, how the different dynamics of the two stages manifest in the low-rank increment that LoRA optimizes remains unclear. Tracking the singular spectrum and subspaces of LoRA updates on mathematical reasoning tasks, we find that RL concentrates the spectrum at every learning rate, while SFT...

    arxiv.org/abs/2611.08414 · PDF

  11. 11

    Conformal Prediction for Time-Dependent PDEs

    Joshua Stiller, Annika Schneider, Eyke Hüllermeier

    cs.LG · stat.ML

    Uncertainty quantification is crucial in scientific machine learning, where models inform safety-critical tasks such as flood forecasting, financial risk management, and thermal control in machines. Conformal prediction provides distribution-free coverage guarantees, but in time-dependent settings common to physics and engineering, these guarantees can break down, leading to systematic undercoverage. We study this problem in the context of...

    arxiv.org/abs/2611.08409 · PDF

  12. 12

    Synthesis of Ovarian Cancer Multi-Omics Datasets: A Biology-Informed Evaluation Framework

    Déborah Boyenval, Alina Malyutina, Jaana Oikkonen, Sammeli Liikkanen, Sampsa Hautaniemi

    cs.LG

    Paired multi-omic cancer cohorts are costly to assemble and difficult to share, motivating synthetic cohorts as a complementary resource. As generative model development accelerates, we shift the focus from proposing yet another architecture to defining evaluation criteria that determine whether synthetic data preserve biologically relevant structure. We introduce a biology-informed and patient-disjoint framework for evaluating the generation...

    arxiv.org/abs/2611.08394 · PDF

  13. 13

    Hybrid Machine Learning for Imbalanced Landing-Phase Safety Event Prediction from Flight Data

    Taha Asghar, Bilal Mufti, Syed Qasim Zaheer, Luqman Ahmed

    cs.LG · cs.AI

    Landing-phase safety-event prediction requires the detection of rare abnormal approaches within strongly imbalanced multivariate flight data. Capturing coupled temporal behavior while distinguishing event-specific signatures across airport operating contexts remains a central modeling challenge. This study proposes an interpretable hybrid feature-fusion framework for multiclass landing-phase safety-event prediction. A long short-term memory...

    arxiv.org/abs/2611.08386 · PDF

  14. 14

    Module-Aware Optimization for Graph Neural Networks

    Guy Hadad, Haggai Roitman, Moshe Eliasof, Bracha Shapira

    cs.LG

    Different optimizers impose different geometries on parameter updates. Adaptive methods such as Adam and AdamW use coordinate-wise updates, while matrix-aware optimizers such as Muon utilize weight-matrix structure through orthogonalized updates. For Graph Neural Networks (GNNs), however, the effective geometry of its weight matrices is shaped not only by its learned channel-mixing weights, but also by the graph-propagated representations it...

    arxiv.org/abs/2611.08379 · PDF

  15. 15

    Muir: Linear Attention with Sensitivity-Allocated Gating and Matched-Filter Readout

    Tushaar Gangavarapu, Gil Tabak, Clemens JS Schaefer

    cs.LG

    Linear attention compresses context into a fixed-size key-value state for linear-time training and constant-memory inference. Gated delta-rule models decay this state and erase its prediction for an incoming key before writing a new value; Gated DeltaNet-2 (GDN-2) generalizes this with channel-wise decay and decoupled, full-rank erase and write gates. However, we show that state updates are far less sensitive to erasure than to writes: key...

    arxiv.org/abs/2611.08376 · PDF

  16. 16

    StrataFlux: Memory-Hierarchy-Native Low-Bit LLM Inference

    Jake Hyun, Ziyu Zhao, Jae-sun Seo, Mohamed S. Abdelfattah

    cs.LG · cs.DC

    Quantization reduces the memory and bandwidth demands of large language model (LLM) inference but can degrade accuracy; offloading higher-precision data can instead stall generation. We introduce StrataFlux, a framework that challenges this tradeoff by recovering accuracy lost to low-bit weight and key-value (KV) cache quantization while retaining quantization's memory and speed advantages. StrataFlux co-designs refinement representation,...

    arxiv.org/abs/2611.08363 · PDF

  17. 17

    Discovering Learning Rules that Shape Others

    Hong Jun Lee, Ziyan Wang, Jun Wang, Stefanos Leonardos, Yali Du

    cs.LG

    In general-sum games, self-interested learners can reach outcomes that are poor for everyone, which has motivated methods that shape the opponent's learning. Methods like Learning with Opponent-Learning Awareness (LOLA), which differentiates through the opponent's learning process, and Model-Free Opponent Shaping (M-FOS), which uses model-free optimisation methods for long-horizon opponent shaping, have enabled mutual cooperation and...

    arxiv.org/abs/2611.08335 · PDF

  18. 18

    Interpretable many-body interaction laws for collective motion from few trajectories

    Timon S. Gutleb

    cs.LG · physics.bio-ph · q-bio.QM

    Models of collective motion usually reduce social interactions to pairwise forces, despite empirical evidence for many-body effects. We introduce the directional atomic cluster expansion (dACE), which adapts the atomic cluster expansion (ACE) of interatomic potentials to self-propelled agents. dACE provides a systematically improvable, exactly equivariant expansion of an agent's acceleration that splits uniquely into 1-, 2-, 3-body and higher...

    arxiv.org/abs/2611.08314 · PDF

  19. 19

    Dynamic Optimistic Constrained OCO with Memory via Delay Equivalence

    Mohammed Abdullah, George Iosifidis, Salah Eddine Elayoubi, Tijani Chahed

    cs.LG · math.OC

    Online Convex Optimization with memory and time-varying constraints (COCO-M) is a recently-introduced framework that captures the dynamics of stateful online learning and non-stochastic control with budget constraints. Despite its expressive power, COCO-M remains largely understudied. In this work, we tackle this problem in its full scale, providing an optimistic algorithm that incorporates untrusted predictions to tame universal dynamic...

    arxiv.org/abs/2611.08302 · PDF

  20. 20

    AutoCompress: Automated Hardware-Guided Model Compression

    Sheldon Liang, Ali Taghibakhshi, Saurav Muralidharan, Sharath Turuvekere Sreenivas, Ashwath Aithal, Pavlo Molchanov,...

    cs.LG · cs.AI

    LLM pruning must preserve quality while meeting hardware- and workload-specific constraints on latency, throughput, and memory. Existing pruning workflows often target parameter counts, requiring practitioners to manually select model sizes that meet latency, throughput, and GPU memory limits. However, parameter count alone does not predict these metrics, which depend on the model architecture, workload, hardware, and serving engine. We...

    arxiv.org/abs/2611.08290 · PDF

  21. 21

    Accelerating Safe Reinforcement Learning with Massive Parallelism

    Joonyoung Lim, Younghwan Yoo

    cs.LG · cs.AI

    We investigate the problem of scaling safe reinforcement learning (safe RL) to massively parallel training regimes, where a large number of environments are exe- cuted with short rollout horizons. Although this setting improves data throughput and wall-clock efficiency, it creates a mismatch with constrained Markov decision processes (CMDPs), whose safety constraints are defined over full-episode cost returns. We address this mismatch by...

    arxiv.org/abs/2611.08273 · PDF

  22. 22

    Scaling Neural Motor Decoding via Decoupled Behavioral Pretraining

    Divyansha Lachi, Vinam Arora, Shivashriganesh P. Mahato, Alexandre Andre, Eva L. Dyer

    cs.LG · cs.AI

    Recent neural decoding approaches achieve strong performance by training on large collections of paired neural-behavioral recordings, but such datasets are expensive, invasive, and difficult to scale. In contrast, behavioral data is abundant and easy to collect, raising the question of whether neural decoding can benefit from decoupling behavioral and neural representation learning. We introduce BEEMO, a flexible multi-session framework that...

    arxiv.org/abs/2611.08263 · PDF

  23. 23

    Nowcasting of Tropical Cyclone Wind Fields and Intensity from Geostationary Imagery

    Simon Donike, Pritthijit Nath, Cristina Radin, Arthur Avenas, Ruben Cartuyvels, Emiliano Diaz, William K. Jones,...

    cs.LG

    High-resolution tropical cyclone surface wind speeds derived from synthetic aperture radar (SAR) observations can support storm structure and intensity modelling, but remain spatially and temporally sparse and unavailable near-real-time. Geostationary satellites capture cloud tops continuously in space and time, but do not directly resolve surface winds. Here, we present simple neural networks to predict 2D wind fields and storm-scale...

    arxiv.org/abs/2611.08255 · PDF

  24. 24

    Geometric Relevance Is Not Sufficient: Evaluating Focused Post-Training Supervision in Diffusion Docking

    Michael Brown, Maryam Rahimimovassagh, Mehdi Yazdani-Jahromi, Sanjeda Sara Jennifer, Ivan Garibay

    cs.LG · q-bio.BM

    Diffusion-based molecular docking requires learning a corrective score field across a broad corruption distribution, yet individual pose errors can differ substantially in their structural consequences. This creates a post-training conflict between preserving broad score-field coverage and concentrating supervision on corrections that appear more relevant to docking geometry. Using DiffDock, a fixed-budget study compares broad continuation...

    arxiv.org/abs/2611.08249 · PDF

  25. 25

    FewStepRL: Fine-tuning Few-step Diffusion Models with Reinforcement Learning

    Haitao Lin, Minsi Ren, Yang Li, Xingtong Ge, Zheyuan Hu, Peiyan Hu, Tailin Wu

    cs.LG · cs.CV

    Reinforcement learning (RL) has become a standard post-training method for diffusion models, but most RL objectives roll out the current policy at every iteration, and with a many-step sampler these rollouts typically dominate the cost of training. Few-step models cut this cost, yet they model an average velocity over an interval, whereas existing RL objectives are built for an instantaneous velocity with a many-step sampler. We adapt RL...

    arxiv.org/abs/2611.08240 · PDF

  26. 26

    Beyond Quadratic Attention: Separable Gaussian Operators for ODE Vision Transformers

    Carlos Boned Riera, David Romero Sanchez, Oriol Ramos Terrades

    cs.LG · cs.CV

    Self-attention carries a quadratic cost in token count, which limits how far Vision Transformers can scale with input resolution. We introduce SSoG, a separable sum-of-Gaussians spatial operator that replaces dense self-attention with a learned mixture of axis-factorized Gaussian kernels, reducing the per-layer cost from $\mathcal{O}(N^2)$ to subquadratic in $N$. Unlike dot-product self-attention, whose Lipschitz constant is unbounded outside...

    arxiv.org/abs/2611.08236 · PDF

  27. 27

    Do sEMG Foundation Models Improve Cross-Subject Generalisation?

    William Raftery, Dario Farina

    cs.LG · q-bio.NC

    sEMG Foundation Models aim to learn general purpose representations over diverse subjects and recording conditions that boost performance on a variety of downstream tasks. However, existing evaluations make it difficult to discern whether pretraining alone is sufficient to combat the cross-subject overfitting which makes machine learning on sEMG so challenging. We conduct a systematic evaluation of three sEMG foundation models across three...

    arxiv.org/abs/2611.08199 · PDF

  28. 28

    BUMP: Brief Unexpected Mid-Interpolation Peak

    Melinda Flóra Kiss, Márton Muntag, Adrián Csiszárik, Dániel Varga

    cs.LG · stat.ML

    We identify an intriguing phenomenon when linearly interpolating between permutation-aligned neural networks: test accuracy briefly improves near the interpolation endpoints before dropping toward the midpoint, which we term BUMP (Brief Unexpected Mid-interpolation Peak). We observe that BUMP appears consistently across datasets, architectures, training regimes, and remains robust under substantial perturbations such as early checkpoints and...

    arxiv.org/abs/2611.08182 · PDF

  29. 29

    Interpretable Multimodal MRI-Clinical Fusion for Functional Outcome Prediction After Acute Ischemic Stroke

    Sina Raeisadigh, Henning Müller, Abderrahmane Hedjoudje

    cs.LG · cs.AI · cs.CV · eess.IV

    This study evaluates the added prognostic value of day-1 multimodal MRI for predicting three-month functional outcome after acute ischemic stroke (AIS). Sixty-six AIS patients with ADC, DWI, FLAIR, and clinical data were analyzed. Three-dimensional pretrained ResNet embeddings were extracted from each MRI modality, reduced using fold-isolated principal component analysis, and combined with structured clinical variables and ADC-derived lesion...

    arxiv.org/abs/2611.08154 · PDF

  30. 30

    Detecting Weather Fronts from Satellite Observations: What Does Reanalysis Add?

    Ioannis Dravilas, Miruna Stoicescu, Roope Tervo, Carlos Horn, Oriol Hinojo Comellas, Sina Montazeri, Andrea Meraner,...

    cs.LG · cs.CV · physics.ao-ph

    Weather fronts are transition regions between air masses with contrasting thermodynamic properties and are associated with mid-latitude precipitation and severe weather. We investigate machine learning (ML) methods for detecting weather fronts and quantify the contribution of reanalysis data to performance. We compare two configurations of the same ML model for pixel-wise frontal classification over the North Atlantic and Europe: (1) using...

    arxiv.org/abs/2611.08148 · PDF

  31. 31

    How Far Can Untrained Neural Networks Go in Anomaly Detection? From Tables to Pixels

    Hyojun Yi, Geunyoung Kim, Hogeon Seo, Seunghyoung Ryu

    cs.LG · cs.CV

    Anomaly detection (AD) is often framed as a representation-learning problem, where strong performance is expected to require learned features or AD-specific optimization. However, this view can overlook the intrinsic separability of anomalies from normal references, even in the absence of learned representations. We therefore investigate how far untrained neural networks can go in anomaly detection. To this end, we propose PRISM, a framework...

    arxiv.org/abs/2611.08136 · PDF

  32. 32

    PersonaAgent: Retrieval-Augmented Transaction Anomaly Scoring with Refreshable Multi-Entity Persona Memory

    Zhengyu Fang, Liang Wang, Yingtong Dou, Yuzhong Chen, Shubham Jain, Junpeng Wang, Xiran Fan, Menghai Pan, Jiarui...

    cs.LG

    In large-scale payment networks, entities (card, merchant, etc.) and their interactions play a key role in transaction anomaly detection. However, transaction patterns between different entities usually change more frequently than the production model refresh schedule. Meanwhile, entity embedding tables suffer from high-cardinality and sparse interaction issues. To resolve the above challenges, we present PersonaAgent, an agentic transaction...

    arxiv.org/abs/2611.08094 · PDF

  33. 33

    Gradient Boosting for Efficient Rank Discovery in Tensor Network Kernel Machines

    Albert Saiapin, Kim Batselier

    cs.LG · cs.AI

    Tensor network kernel machines (TNKMs) combine nonlinear features with low-rank tensor representations to model complex relationships efficiently. However, selecting an appropriate tensor rank remains challenging, as conventional rank-search methods require training multiple candidate models, increasing computational costs. In this work, we focus on a canonical polyadic (CP) tensor network parameterization and propose GB-CPKM, a gradient...

    arxiv.org/abs/2611.08093 · PDF

  34. 34

    Gabor Gabor: E-Gaborformer, a Strictly Causal Speech Recognizer with Gabor-Dictionary Convolutions and Delta-Rule Memory

    Nir Krakowski, Tamir Cohen, Ohad Zohar, Chen Zarfati

    cs.LG · cs.CL · cs.SD

    Real-time voice agents need a speech recognizer that never waits for future audio and whose memory per call does not grow with the length of the call. We present E-Gaborformer, a strictly causal encoder that combines Gated DeltaNet-2 recurrent layers with sliding-window attention in a 3:1 ratio, and builds every temporal convolution from a fixed dictionary of causal Gabor atoms. Its streaming state is constant, about 19 MB per call at...

    arxiv.org/abs/2611.08092 · PDF

  35. 35

    SAE Interprets MoE: Sparse AutoEncoders Reveal That MoE Experts Exhibit Human-Interpretable Fine-Grained Specialization

    Anyuan Zhuo, Shaohua Zhang, Yuan Lin, Pinyan Lu

    cs.LG

    Mixture-of-Experts (MoE) models route each token to a few of many experts, yet what an individual expert is responsible for remains unsettled. Existing analyses count how often an expert is selected on domain-labeled corpora, an instrument that fixes the taxonomy in advance, stays coarse, and records selection rather than contribution. We instead study specialization in feature space by training a sparse autoencoder (SAE) on the output of an...

    arxiv.org/abs/2611.08089 · PDF

  36. 36

    Value or Attention: Investigating Layer-wise Relevance Propagation for Graph Attention Networks

    Ken Schreiber, René P. Klausen, Sebastian Lapuschkin, Wojciech Samek, Jonas Naujoks

    cs.LG · cs.AI

    Graph Attention Networks (GATs) are increasingly used in domains where scientists need to read a model's reasoning, from medicine to materials science, yet the routing they learn remains opaque to most explainers. Layer-wise Relevance Propagation (LRP) follows the model's computation, but in an attention layer the relevance reaching a node is the sum of two mechanically different quantities: its content relevance through the value path and...

    arxiv.org/abs/2611.08088 · PDF

  37. 37

    Optimal Competitive Ratios for Chasing Convex Functions with Quadratic Growth

    Linzhe He, Sifan Yang, Wei Jiang, Lijun Zhang

    cs.LG

    Chasing convex function has attracted much attention in recent years, which can be formulated as a game between a learner and an adversary. In each round, the adversary reveals a convex function, and the learner selects a decision and incurs a hitting cost and a movement cost for changing its decision. The performance of the learner is measured by the \emph{competitive ratio}, which compares its cumulative cost with that of an optimal offline...

    arxiv.org/abs/2611.08078 · PDF

  38. 38

    Can an LLM Actually Reason and Program about Neural Network's Weights or is it just memorization on steroids? Exploring "Weight-Space Programming"

    Nazmus Sakib Ahmed

    cs.LG · cs.AI

    Large language models (LLMs) have recently been reported to read and write the raw weights of neural networks directly, a capability termed weight-space programming. We ask whether the reading side of this capability reflects reasoning about weight flows or a combination of local arithmetic at the network's edges and memorized priors over public datasets. We test Claude Opus 5.5 on small convolutional networks of about 125,000 parameters...

    arxiv.org/abs/2611.08070 · PDF

  39. 39

    Performance metrics in machine learning: a scenario-based guide from the confusion matrix to calibration, regression and uncertainty

    Mehmet Baygin, Sengul Dogan, Turker Tuncer

    cs.LG

    Performance metrics can yield different conclusions from the same predictions. This methodological tutorial explains the effects of class prevalence, decision thresholds, aggregation rules and software conventions through controlled scenarios. Twenty-two confusion-matrix metrics are organized in a prevalence-coded map. Fifteen scenarios cover binary and multi-class classification, score discrimination, probability calibration, regression,...

    arxiv.org/abs/2611.08048 · PDF

  40. 40

    Should We Pay This Much for Robustness? Efficient Proxy Certificates with Marginal Guarantees

    Soroush H. Zargarbashi, Mohammad Sadegh Akhondzadeh, Aleksandar Bojchevski

    cs.LG

    Modern classifiers remain sensitive to small input perturbations, while strong model-agnostic robustness certificates such as randomized smoothing are often too expensive for deployment, requiring an extensive number (e.g. 2000) of model forward passes per input. We propose a framework that delegates certification to a cheap proxy function while statistically controlling its deviation from the original sound certificate. Given an exchangeable...

    arxiv.org/abs/2611.08035 · PDF

  41. 41

    Reproducing Kernel (Quasi-)Banach Spaces with the $\ell^p$ Coefficient Norms

    Tianfang Wang, Guohui Song, Haizhang Zhang

    cs.LG

    This paper advances the theory of reproducing kernel Banach spaces (RKBSs) by investigating the general $\ell^p$ regularization framework, with a particular focus on the non-convex regime $0<p<1$. While our preceding work established the foundation for RKBSs with the $\ell^1$ norm, the pursuit of stronger sparsity---closer to the $\ell^0$ pseudo-norm---naturally motivates this exploration into $\ell^p$ quasi-norms. For $0<p<1$, we construct...

    arxiv.org/abs/2611.08032 · PDF

  42. 42

    GTAFL: Addressing Test-Agnostic Long-Tailed Federated Learning in Medical Image

    Guangyu Chen, Jingyun Zeng, Hui Jiang

    cs.LG

    In healthcare, protecting patient privacy is crucial due to the sensitivity of medical data and its extensive accessibility. Federated Learning (FL) offers a decentralized and privacy-preserving training paradigm, making it an ideal solution for healthcare applications. A critical challenge in healthcare is that real-world medical data often exhibits long-tailed distributions in local and global views. Existing methods addressing long-tailed...

    arxiv.org/abs/2611.08021 · PDF

  43. 43

    Learning Dynamics of On-Policy Distillation

    Guan Zhe Hong, Sumeet Ramesh Motwani, Liang Chen, Yangchen Pan, Junchi Yu, Jialin Yu, Philip Torr, Yi Ren

    cs.LG · cs.CL

    On-policy distillation (OPD) is emerging as an important paradigm for language-model post-training, combining dense teacher supervision with training on student-generated trajectories. This creates a feedback loop where each update changes the student and thereby the training distribution that generates subsequent updates. Yet, how this feedback shapes learning and generalization remains poorly understood. We conduct a mechanistic analysis of...

    arxiv.org/abs/2611.08016 · PDF

  44. 44

    Towards Multiscale Molecular Representation Learning through Geometry-Based Fragmentation for Polymer Property Prediction

    Makoto Takamoto, Chaitanya Sampara, Viktor Zaverkin, Daniel Oñoro-Rubio, Mathias Niepert

    cs.LG · cond-mat.mtrl-sci · physics.chem-ph

    It is well established that molecular substructures can contribute unequally to chemical and physical properties, with specific motifs or side chains often playing dominant roles. Fragment-based representations have therefore emerged as an effective strategy for incorporating chemically meaningful inductive biases, as exemplified by molecular fingerprints and fragment-based attention mechanisms. However, many existing fragmentation-based...

    arxiv.org/abs/2611.07998 · PDF

  45. 45

    QRAE: Query-Relevant Attention Entropy for Context Compression in DeepSearch Agents

    Yingjie Tan, Haiquan Qiu, Quanming Yao, Manyi Zhong, Xianzhi Yu, Yaqing Wang

    cs.LG

    DeepSearch LLM agents repeatedly search and carry retrieved sources into subsequent reasoning steps. This expanding context increases token consumption and latency, although many results do not contribute to later reasoning. We propose Query-Relevant Attention Entropy (QRAE), an online context compression method that uses sparse evidence-assessment heads to decide which intact retrieved chunks should persist across reasoning steps. It scores...

    arxiv.org/abs/2611.07976 · PDF

  46. 46

    Tight Asymptotic Bounds on the Maximum Number of Linear Regions of Piecewise-Linear Kolmogorov--Arnold networks

    Ye Su, Yong Liu

    cs.LG

    Schoots et al.~\citep{schoots2025relating} (AISTATS 2025) established a generally loose upper bound on the number of linear regions of continuous piecewise-linear (CPWL) Kolmogorov--Arnold networks (KANs), without providing a lower bound. We consider a $T$-layer network with width vector $\mathbf n=(n_0,\ldots,n_T)$, input dimension $n_0=d$, output width $n_T=1$, and at most $k\geq2$ affine segments per edge. Our matching upper and lower...

    arxiv.org/abs/2611.07973 · PDF

  47. 47

    CoRD: Coupled Rollouts Fill the Blind Spot of On-Policy Distillation

    Guhan Chen, Bohan Li, Jun Rao, Hejin Wang, Songtao Tian, Yexin Xie, Xiaojun Meng, Peng Yuan, Jiansheng Wei, Xuebo...

    cs.LG · cs.CL

    On-policy distillation trains a student on its own rollouts, so states the teacher would reach but the student does not are seldom supervised. We introduce CoRD (coupled-rollout distillation), which keeps the on-policy loss and moves only the distribution of training states toward the teacher, drawing rollouts from a geometric interpolation of student and teacher through the maximal coupling of speculative decoding. The coupling keeps the...

    arxiv.org/abs/2611.07959 · PDF

  48. 48

    Invariance before structure: two modes of emergent compression in representation learning

    Hardik Rajpal, Dan Goodman

    cs.LG · cs.NE · q-bio.NC

    Learned representations show striking invariance across biological and artificial neural networks, with different substrates and variations driven by drift, heterogeneity and initialisation. To understand this invariance, we propose Emergent Adaptive Representation Learning (EARL), a framework that distinguishes two modes of emergence in learned representations: state-space compression, where a learned representation gains predictive power...

    arxiv.org/abs/2611.07952 · PDF

  49. 49

    Leakage- and Label-Aware Benchmarking of Class-Imbalance Strategies for Multimodal Chest X-Ray and EHR Classification

    Dang-Khoa Bui, Quang-Huy Tran, Minh-Khoi Nguyen-Bui, Duc-Tuan Ngo, Thanh-Trong Tran, Tuan-Khoi Nguyen, Hoang-Anh Ngo

    cs.LG · cs.CV

    Benchmarks of class-imbalance strategies for multimodal chest X-ray (CXR) and electronic health record (EHR) prediction are often evaluated on record-level splits of cohorts in which one patient contributes several radiographs. We audit such a benchmark on 29,083 MIMIC-CXR radiographs of 10,479 patients with eight diagnosis targets. First, the record-split benchmark was misleading: a lookup table that copies the label of a development-set...

    arxiv.org/abs/2611.07950 · PDF

  50. 50

    The Blindfolded Controller: Decision Model-Guided Training for Reinforcement Learning

    Muhammad Ibrahim Khan, Aniket Dixit, Faizan Ahmed, James Brusey

    cs.LG

    Reinforcement learning (RL) provides a framework for continuous improvement through interaction with an environment. However, RL agents struggle against tasks that offer exploration challenges and sparse rewards, because learning efficient policies from scratch is slow and sample-inefficient. In contrast, decision models (DMs), given a description of the environment and task, can perform well in sparse-reward tasks, but, as they do not learn...

    arxiv.org/abs/2611.07934 · PDF

  51. 51

    AttnSR: Attention Step Reward for Long-Horizon Agentic RL

    Mengsong Wu, Tianchen Zhou, Zhaochen Su, Zhenhua Liu, Zhe Cui, Aimin Zhou

    cs.LG · cs.AI

    Long-horizon LLM agents make multi-step decisions, yet outcome-reward-only reinforcement learning provides little direct supervision for individual steps. We introduce AttnSR, a step-level reward augmentation method that derives reward allocations from policy attention extracted during training replay, without a separately trained reward model or additional rollouts. AttnSR traces backward through the policy model attention graph to estimate...

    arxiv.org/abs/2611.07932 · PDF

  52. 52

    Flux as a Codec: Zero-Shot Lossy Compression with Flux

    Fuma Kimishima, Jinjia Zhou

    cs.LG · cs.CV

    We propose FluxC, a zero-shot codec that uses Flux as a compression prior. Although DiffC has shown that pretrained diffusion models can serve as zero-shot codecs, we ask how the choice of coding path and reconstruction method affects rate--distortion--perception behavior. We analyze FluxC along three axes: coding cost and bit allocation, reconstruction method, and low rate distortion. At matched signal-to-noise ratio (SNR), we show that the...

    arxiv.org/abs/2611.07929 · PDF

  53. 53

    Same Answer, Different Future: Matched Predictions Do Not Imply Matched Belief Updating in Language Models

    Weihan Li, Junhao Wu, Xinlei Chen, Tianshi Zheng

    cs.LG

    Language models can produce similar probabilistic predictions after observing outcomes, reading summaries, receiving instructions, or undergoing activation interventions. Whether these routes induce equivalent belief states remains unclear: agreement in current predictions does not establish how a model will respond to subsequent evidence. We investigate this distinction in a Bernoulli inference task with an analytic Bayesian reference....

    arxiv.org/abs/2611.07913 · PDF

  54. 54

    DISCERN: Distributional Synthetic Replay for Retain-Free Incremental Machine Unlearning

    Nicolás Segura-Kunsagi, Juan Luis Suárez-Díaz, Iván Sevillano-García, Francisco Herrera

    cs.LG · cs.AI

    Machine unlearning aims to remove the influence of specific training data from a trained model without retraining from scratch. In real deployments, however, deletion requests arrive over time, calling for incremental class unlearning (ICU): classes are removed sequentially, previously forgotten knowledge must remain forgotten, and the remaining classes must be preserved. We present DIstributional SynthetiC rEplay for Retain-free iNcremental...

    arxiv.org/abs/2611.07892 · PDF

  55. 55

    When, Where, How: Active In-Context Learning for Tabular Foundation Models

    Christian M. M. Frey

    cs.LG · cs.AI

    Tabular Foundation Models (TFMs) adapt to a new task only through the labeled rows in their context. However, labels are expensive to acquire in real-world scenarios due to the need for highly specific domain knowledge to ensure high-quality data. Active in-context learning transfers classical acquisition rules to this setting, in which the model that guides acquisition changes with every label added. We ask how the value of diversity,...

    arxiv.org/abs/2611.07886 · PDF

  56. 56

    Computation, not weight access: large language models as auditors of clinical prediction networks

    Zhiyu Liu, Xinyu Yang

    cs.LG · cs.AI

    Frontier large language models (LLMs) have been reported to read, edit and write the weights of small neural networks, raising the prospect of auditing clinical prediction models from their files alone. Public demonstrations, however, mixed reading raw weights with running code. We separated these conditions in internally pre-specified simulation experiments. Tanh networks of 105 to 38,401 weights were trained on simulated cohorts of patients...

    arxiv.org/abs/2611.07847 · PDF

  57. 57

    EnergyBench: a live, gate-aligned benchmark of day-ahead forecasts for European power markets

    Evgenii Genov, Ander Martinez Alonso, Christoph Bergmeir

    cs.LG · eess.SY

    Energy forecasting models are usually compared on backtests and fixed public datasets. Backtests can use data that did not yet exist when the forecast would have been made, and a fixed test set, reused across studies and possibly contained in the pretraining data of foundation models, invites overfitting to the benchmark. EnergyBench evaluates every entry live instead. Each day at 11:30 CET/CEST, thirty minutes before the day-ahead (SDAC)...

    arxiv.org/abs/2611.07784 · PDF

  58. 58

    PBPO: Prefix-Block Policy Optimization for Diffusion Language Models

    Katherine Tieu, Dongqi Fu, Si Zhang, Hanqing Zeng, Tianxin Wei, Chenglin Wei, Yinglong Xia, Hong Li, Jingrui He

    cs.LG

    Diffusion Large Language Models (dLLMs) commonly decode responses by proceeding autoregressively across blocks while iteratively denoising multiple tokens within each block. This structure makes block-level policy optimization a natural adaptation of autoregressive policy optimization. However, the exact likelihood of a completed block is intractable because it marginalizes over the intermediate states and reveal orders of the denoising...

    arxiv.org/abs/2611.07783 · PDF

  59. 59

    Adapters as Data: The Latent Space of LoRAs

    Moritz Grävinghoff, Konstantin Wehmeyer, Aron Asefaw, Boris Knyazev, Damian Borth

    cs.LG · cs.CV

    LoRA adapters are usually treated as finished artifacts rather than as a data modality. Prior work has shown that LoRA adapters populate a low-dimensional manifold that a linear PCA basis already organizes well enough to support different operations such as sampling, editing, and inversion. We ask what a learned, nonlinear latent adds beyond this. We train a transformer autoencoder on the same population of LoRA identity adapters, mapping...

    arxiv.org/abs/2611.07775 · PDF

  60. 60

    Optimal Learning in Games with Arbitrary Delays: Best-of-Both-Worlds for Self-Play and Adversarial Opponents

    Emmanuel Esposito, Andrew Jacobsen, Hao Qiu, Alexander Ryabchenko, Mengxiao Zhang

    cs.LG · stat.ML

    We study learning in zero-sum games with delayed feedback. We introduce Multi-Anchor Optimistic Online Mirror Descent (MAO-OMD), a single algorithm which obtains optimal fast rates in self-play on smooth games, while simultaneously maintaining minimax optimal guarantees against arbitrary opponents. In particular, in smooth zero-sum games with arbitrary and unknown feedback delays, MAO-OMD obtains $O(σ_{\max}+1)$ social and individual regrets...

    arxiv.org/abs/2611.07773 · PDF

  61. 61

    Scaling Laws for SAE Training Data

    Nikita Koriagin, Nikita Balagansky, Daniil Gavrilov

    cs.LG · cs.AI · cs.CL

    Activation storage is often a bottleneck in sparse autoencoder (SAE) training. We study how many unique activations an SAE needs when its total training budget is fixed. Quality improves rapidly as the buffer grows, then reaches a plateau where replaying the buffer is nearly as effective as using fresh activations throughout training. Beyond this point, additional unique activations bring little benefit to reconstruction or interpretability,...

    arxiv.org/abs/2611.07756 · PDF

  62. 62

    Neural Collapse on Imbalanced Datasets and Open-Set Scenario

    Qiufu Li, Huibing Xiao, Linlin Shen, Yongdong Huang

    cs.LG

    Neural collapse (NC) describes the elegant geometric structure that sample features and classifier vectors exhibit when a classification model's loss reaches its minimum. Class imbalance, however, generally distorts standard NC. In this paper, we theoretically reformulate NC on imbalanced datasets and in open-set settings. Our analysis is based on an optimization problem over class-wise averaged losses, rather than the global averaged losses...

    arxiv.org/abs/2611.07753 · PDF

  63. 63

    Foresee to Trade: Forecast-Augmented Policy Optimization for Portfolio Management

    Zeyu Liu, Xuanzhi Feng, Haoran Xu, Lai Sing Kwong, Yang Li, Hualei Zhang, Hao Chen, Jie Zhang, Xiaoyi Pang, Song Guo

    cs.LG

    Online investment services must translate asset opportunities into investment outcomes for users' accounts, yet the same market opportunity can warrant different account actions. We study how supervision from future asset opportunities can strengthen account-level trading decisions and propose Forecast-Augmented Relative Policy Optimization (FRPO). FRPO separately supervises asset opportunities and account decisions through a jointly learned...

    arxiv.org/abs/2611.07750 · PDF

  64. 64

    ConvPE: Convolutional Position Embedding Improves Transformers

    Zehao Wang

    cs.LG

    ConvPE (Convolutional Position Embedding) improves both the performance and efficiency of Transformer models in language modeling, long-context extension, and vision. It injects a windowed, content-dependent bias into attention scores through a lightweight causal convolution over a low-dimensional projection of the hidden states. As a lightweight plug-in, ConvPE is compatible with RoPE, attention masks, and KV concatenation. It also supports...

    arxiv.org/abs/2611.07749 · PDF

  65. 65

    Ordering Multivariate Time Series by Volatility Memory: A Temporal Convolutional Approach to Volatility Heredity

    Kaoutar El Azaar, Mohamed Hamlich, Moncef Garouani, Franck Ravat, Asmae Chakir, Mohamed Tabaa

    cs.LG

    Volatility dynamics can carry information about temporal relationships that is not apparent in the conditional mean. Extracting this information is difficult because the persistence of each variable can obscure its dependence on other variables. This paper presents QS-TCN, a method for ordering multivariate time series that combines statistical estimates of volatility memory with temporal convolutional learning. Persistence estimates obtained...

    arxiv.org/abs/2611.07746 · PDF

  66. 66

    Comparing Classification and Feature Learning by Analyzing Their Loss Functions in Normalized Spaces with Different Scale Factors

    Qiufu Li, Huibin Xiao, Linlin Shen, Weijia Fan, Xi Jia, Yongdong Huang

    cs.LG · cs.CV

    Classification and feature learning are two closely interrelated fundamental tasks. Feature learning in supervised settings is typically achieved by training classification models, under the common assumption that higher classification accuracy implies better feature quality. In this paper, we first point out that classification accuracy reflects the sample-wise local properties of a model on a dataset, whereas feature learning aims to learn...

    arxiv.org/abs/2611.07741 · PDF

  67. 67

    Static Recovery Is Not Dynamic Stability: Dynamics-Aware Benchmarking of Protein Motif Scaffolding

    Emile de Bruyn, Moritz Schäffler, Jannik Schneider, Sebastian R. Schmidt, Chanwoong Hwang, Aleena Siji, Utkarsh...

    cs.LG · q-bio.BM

    Protein motif scaffolding benchmarks assess whether generative models can build scaffolds around functionally important residues. Their conclusions depend on which motifs are tested and which criteria define success. Existing benchmarks typically use small hand-curated motif sets and static refolding criteria, but protein function often depends on conformational ensembles and transitions. Here, we address both benchmark scope and success...

    arxiv.org/abs/2611.07723 · PDF

  68. 68

    Reinforcing What the Student Has Learned: Teacher-Guided On-Policy Distillation

    Bozhen Fang, Linhao Zhong, Xuehao Wang, Huanyi Zheng, Hao Chen, Chunhua Shen

    cs.LG

    On-policy distillation (OPD) improves language-model reasoning through dense teacher feedback on student-generated responses.We first observe an amplitude disparity in this feedback: on math reasoning responses, style-related positions exhibit 3.5 times the mean absolute teacher--student log-probability difference of mathematical positions.Scaling OPD updates by these discrepancies thus directs much of the update strength toward matching the...

    arxiv.org/abs/2611.07722 · PDF

  69. 69

    Evaluating Language Model Features for Zero-Shot Time-Series Anomaly Detection Under Isolation Assumption

    Romain Hermary, Nesryne Mejri, Djamila Aouada

    cs.LG · cs.AI

    Time-Series Anomaly Detection (TSAD) is a fundamentally challenging task: it is unsupervised, requires high local precision capabilities, and suffers from limited labelled data. Recent work has explored adapting Large Language Models (LLMs) to time-series via cross-modal transfer. However, existing approaches are poorly suited to Multivariate TSAD (MTSAD), as tokenisation often obscures fine-grained temporal and variables structures, and most...

    arxiv.org/abs/2611.07720 · PDF

  70. 70

    Equivariant Deep Learning in Direct-Product Space for Electronic-Structure Hamiltonian Prediction

    Shi Yin, Zujian Dai, Xinyang Pan, Lixin He

    cs.LG · cond-mat.mtrl-sci · cs.AI · physics.comp-ph

    Most existing equivariant architectures for Hamiltonian prediction propagate features in direct-sum spaces of irreducible spherical tensors, whereas the target Hamiltonian is naturally an operator block in a direct-product representation over local orbital and spin bases. This representation mismatch fragments contiguous Hamiltonian blocks, obscures direct manipulation of orbital couplings, and organizes features into irregular...

    arxiv.org/abs/2611.07695 · PDF

  71. 71

    Non-Parametric Mutual Information Estimation with Vine Copulas

    Sigurd Holmsen, Berit Omli Øksnes, Ingrid Hobæk Haff, Sylvia Richardson, Ali Ramezani-Kebrya

    cs.LG

    We introduce VINE: a non-parametric mutual information (MI) estimator based on simplified D-vine copulas. Approximating the joint density with a simplified D-vine allows the density ratio to simplify to a product of pair copulas, which we can estimate non-parametrically. This structural simplification yields VINE, a flexible and computationally efficient MI estimator. We compare VINE against SotA neural MI estimators on a range of datasets,...

    arxiv.org/abs/2611.07693 · PDF

  72. 72

    $O(1/n)$ Convergence Rate for RVI $Q$-Learning

    Avik Kar, Siddharth Chandak, Rahul Singh, Shalabh Bhatnagar

    cs.LG

    We establish finite-time convergence guarantee for synchronous relative value iteration (RVI) $Q$-learning for finite weakly communicating Markov decision processes (MDPs). For stochastic Krasnosel'skiĭ--Mann iterations driven by a non-expansive, linearly regular map on a finite-dimensional normed space, we establish last-iterate mean-squared residual bounds under martingale-difference noise whose conditional second moment grows at most...

    arxiv.org/abs/2611.07676 · PDF

  73. 73

    Instrumental Variables in Continuous-Time Stochastic Processes

    Lars Pennig, Søren Wengel Mogensen, Niki Kilbertus

    cs.LG

    Causal questions often arise in systems that evolve in continuous time. While instrumental variable (IV) methods provide a principled approach to identify causal effects in static or discrete-time settings, even in the presence of unobserved confounding, they do not yet extend to continuous-time dynamics. In this work, we develop an IV framework for stationary continuous-time stochastic processes parameterized by stochastic differential...

    arxiv.org/abs/2611.07674 · PDF

  74. 74

    Calibrated Target Noise Recovers Curvature from the Gradients

    Arash Jamshidi, Katsiaryna Haitsiukevich, Aristides Gionis, Kai Puolamäki

    cs.LG

    Estimating curvature information from gradients alone is a fundamental problem in machine learning. The challenge becomes particularly acute in settings where only aggregate mini-batch gradients are available, yet curvature information is still required for applications such as preconditioning in optimisation, sampling, and early stopping. Perhaps surprisingly, we show that a simple target-perturbation scheme suffices to recover such...

    arxiv.org/abs/2611.07671 · PDF

  75. 75

    An Information Bottleneck for Post-hoc Memorization Mitigation in LLMs

    Dina EL Zein, Shashi Kumar, James Henderson

    cs.LG

    Recent studies have shown that large language models (LLMs) can memorize and reproduce copyrighted training data, raising serious privacy and intellectual property concerns. In this work, we investigate whether memorization can be mitigated without modifying or retraining the underlying model through an information bottleneck. We propose a modular plug-and-play framework that inserts a nonparametric variational information bottleneck (NVIB)...

    arxiv.org/abs/2611.07669 · PDF

  76. 76

    Simple Meta-Learning for HPO and CASH Using Performance Representations

    Emiel Witting, Cheng Yan, Tom Julian Viering

    cs.LG

    Bayesian optimization for Hyperparameter Optimization (HPO) and Combined Algorithm Selection and Hyperparameter Optimization (CASH) often relies on explicit hyperparameter encodings as surrogate inputs. In high-dimensional search spaces, however, such representations can make it difficult for standard surrogates to learn useful performance models. We instead study an alternative meta-learning representation that describes each configuration...

    arxiv.org/abs/2611.07659 · PDF

This edition is part of The Daily Abstract — cs.LG archive. Subscribe to receive these in your inbox each morning, automatically translated to Spanish, with reply-to-PDF: arxivdaily.ignorelist.com.

Colophon Set in Georgia, with system sans for interface chrome and a monospaced stack for code and paper identifiers. Sole accent: amber #D99C5E. Built and served on an always-free VM. The masthead is set 14% letterspaced because newspapers do that and it works.