cs.LG · 2026-10-08 · No. 137

Machine Learning, 2026-10-08.

79 new papers in cs.LG. Titles, authors, abstracts. Links to arXiv. Want this in your inbox every morning? Subscribe →

01 — The papers

79 entries
  1. 01

    Decoupling Exploration from Optimization in RLVR

    Saif Punjwani, Micah Goldblum

    cs.LG · cs.AI · cs.CL

    Modern language models undergo reinforcement learning with verifiable rewards (RLVR) on top of already-trained checkpoints. A key promise of RLVR is the discovery of new reasoning strategies. In principle, a model can sample novel ideas absent from its prior training data. In practice, however, augmenting RLVR with strong novelty incentives has seen limited success and can degrade model quality. Because verifiable rewards supervise only a...

    arxiv.org/abs/2610.10536 · PDF

  2. 02

    Distilling Graph Geometry: Knowledge Gap from GNNs to MLPs

    Zhewei Chen, Hao Zhu, Jiaojiao Jiang, Ahad N. Zehmakan

    cs.LG

    GNN-to-MLP distillation aims to retain the predictive accuracy of a message-passing teacher while deploying a graph-free MLP at inference. Existing methods mainly transfer node-wise predictions or use confidence-based reweighting, but they do not specify where the student should preserve the teacher's graph-induced geometry. We show that this omission leads to two spectral failure modes in the student's representation space. On sparse graphs,...

    arxiv.org/abs/2610.10520 · PDF

  3. 03

    Why Forget-Only Unlearning Needs Memorization

    Luka Radić, Vikrant Singhal, Amartya Sanyal

    cs.LG · cs.IT · stat.ML

    Machine unlearning asks for a deletion algorithm whose output is close to retraining from scratch without the selected forget examples. In this work, we study forget-only unlearning, where the deletion algorithm receives only the trained model and the examples to forget, with no retained data or extra training information. We ask whether forget-only unlearning is always possible. We first show that this depends on the learning method:...

    arxiv.org/abs/2610.10519 · PDF

  4. 04

    Oracle-Efficient and Parameter-Free Agnostic Smoothed Online Learning

    Sasha Voitovych, Adam Block, Alexander Rakhlin, Abhishek Shetty

    cs.LG · stat.ML

    Online learning is an attractive framework in many domains because it permits well-defined learning even when data are dependent or chosen adversarially. This generality, however, comes at a steep price, introducing significant statistical and computational barriers. Recently, smoothed online learning has emerged as a promising framework that interpolates between the fully adversarial and fully stochastic settings by assuming that the...

    arxiv.org/abs/2610.10499 · PDF

  5. 05

    Two-Level Softmax Sampling Done Right: Correcting Bias from Size Imbalance and Dispersion

    Walid Bendada, Guillaume Salha-Galvan

    cs.LG · cs.IR · stat.ML

    Sampling from a softmax distribution is a fundamental operation in machine learning, but its linear complexity in the number of items makes exact sampling impractical at scale. Two-level softmax (2LS) sampling is a popular alternative enabling sublinear-time sampling. Assuming items are partitioned into clusters, 2LS first samples a cluster and then an item within it. In this paper, we show that, despite its advantages, 2LS introduces...

    arxiv.org/abs/2610.10483 · PDF

  6. 06

    Composing What Each Teacher Learned: Multi-Teacher On-Policy Distillation through Teacher-Relative Shifts

    Hejian Sang, Zhengze Zhou, Shayan Mohajer Hamidi, Xiaomin Li, Rohit Jain, Alborz Geramifard

    cs.LG · cs.AI

    Multi-teacher on-policy distillation (MOPD) is used in two settings. In common-domain composition, several teachers score each student rollout from one prompt domain and their signals form a single target; in routed-domain distillation, prompts from different domains are assigned to the corresponding specialist. Both settings usually transfer each teacher's endpoint policy, which mixes what post-training changed with preferences inherited...

    arxiv.org/abs/2610.10460 · PDF

  7. 07

    NeuralBES: A Differentiable, Control-Aware Emulator for Scalable Building Energy Modeling

    Ting-Yu Dai, Takuya Kurihana, Wing Yee Au, Hon Yung Wong

    cs.LG

    Demand-side flexibility i.e. forecasting, shifting, and curtailing residential energy loads, depends on thermal models trusted across millions of heterogeneous buildings. Existing tools force a hard tradeoff: high-fidelity physics simulators such as EnergyPlus are accurate but sequential and require per-building calibration, while purely data-driven sequence models scale but abandon the physical structure that makes their predictions...

    arxiv.org/abs/2610.10459 · PDF

  8. 08

    A Good Self-Teacher Meets the Student Where They Are: Joint On-Policy Learning and Teaching

    Randy Ardywibowo, Arnav Dalal, Jiantao Jiao

    cs.LG · cs.AI

    Reinforcement Learning (RL) from outcome rewards suffers from sparse supervision, particularly on difficult, long-horizon tasks where successful trajectories are rare and costly to generate. On-Policy Distillation (OPD) offers an attractive alternative by providing dense token-level supervision from a stronger teacher along the student's own generations. Self-distillation methods further remove the need for a separate teacher model by...

    arxiv.org/abs/2610.10447 · PDF

  9. 09

    Seq-Flow: Efficient Probabilistic Forecasting with Self-Rollout Error Control

    Yinan Huang, Shitij Govil, Bo Dai, Pan Li

    cs.LG

    Many scientific forecasting tasks require updating a distribution over future trajectories as new observations arrive. Conventional diffusion and flow models generate each forecast from Gaussian noise, often at the cost of many sampling steps. Warm-start methods reuse earlier predictions to reduce this cost, but their models are not trained to perform the forecast update itself, which can compromise quality under few-step sampling. In this...

    arxiv.org/abs/2610.10440 · PDF

  10. 10

    Q-Learning with Scalar Adjoint Matching

    Yonghoon Dong, Minsung Yoon, Jaehyuk Kim, Jungwoo Park, Changyeon Kim, Jinwoo Shin

    cs.LG · cs.AI · cs.RO

    Flow policies capture rich and diverse action distributions, and fine-tuning them with off-policy RL to improve beyond the demonstrations has drawn growing interest. However, fine-tuning a flow policy against a learned value function is not trivial, because the policy generates its action over many flow steps. Adjoint matching offers a principled way to update the flow model itself by propagating value information from the final action back...

    arxiv.org/abs/2610.10437 · PDF

  11. 11

    Which Rollout Taught It That? BehaviorTrace and the Limits of Training-Data Attribution in Online RL

    Amit Nautiyal

    cs.LG · cs.CL

    When reinforcement learning teaches a language model a new behavior, can we find the training rollouts that taught it? And when an attribution method says it can, how do we know the answer is real? We study both questions on online RL fine-tuning with GRPO, using a planted behavior with a known cause. We release BehaviorTrace, an open evaluation harness that combines full-gradient sketching, the planted-behavior setup, and controls for...

    arxiv.org/abs/2610.10422 · PDF

  12. 12

    Training Parallel Speculative Draft Models by Directly Minimizing Expected Decoding Rounds

    Yunxiao Zhao, Changxiao Cai

    cs.LG · cs.AI · cs.CL · stat.ML

    Speculative decoding accelerates large language model inference by using a low-cost draft model to propose tokens that the full-size target model verifies in parallel. Parallel and semi-autoregressive (semi- AR) drafters improve drafting efficiency by proposing an entire block in a single forward pass, but training them raises a new difficulty: the draft distribution for a given position depends on where the decoding round starts, and where...

    arxiv.org/abs/2610.10411 · PDF

  13. 13

    Cross-Domain Pretraining for Steady-State Neural CFD Surrogates

    Anthony Zhou, Amir Barati Farimani, Shirley Ho, Rudy Morel

    cs.LG

    Neural surrogates for computational fluid dynamics (CFD) have the potential to greatly enhance engineering innovation through accelerating simulation. However, the primary limitation for neural surrogates is the lack of generalization to geometries and applications beyond the training set, which is significant given the diversity of engineering scenarios. Currently, this is addressed by generating a new dataset for a specific application;...

    arxiv.org/abs/2610.10398 · PDF

  14. 14

    Executing Causal Structure Learning with Linear-Attention Transformers

    Amartya Roy, Sayar Karmakar

    cs.LG

    Transformers can execute algorithms on data given in their input. We ask whether they can do the same for causal discovery. We study a standard continuous method that repeatedly updates a candidate causal graph while enforcing acyclicity. We explicitly construct a fixed-weight transformer whose forward pass exactly reproduces one update of this method, so repeated blocks reproduce its optimization trajectory. The transformer carries the...

    arxiv.org/abs/2610.10395 · PDF

  15. 15

    Kernel Autoresearch for Open-Ended Model Discovery

    Richard Cornelius Suwandi, Feng Yin, Kevin Murphy

    cs.LG

    Kernels encode the inductive bias of a wide range of machine learning models, yet automated kernel design faces a fundamental dilemma. A fixed grammar of base kernels and operators guarantees validity but limits the search to structures expressible by those building blocks. Conversely, unrestricted programs remove this limitation but no longer guarantee validity. In our stress tests, 22-58% of LLM-generated kernels that pass numerical checks...

    arxiv.org/abs/2610.10394 · PDF

  16. 16

    OrBIT: Structure-Guided Embedding Compression

    Yunied Puig, Amit Kumar Jaiswal

    cs.LG

    Embedding tables are among the largest components of modern language models. Most compression methods fix a coding geometry such as coordinate blocks, low-rank subspaces, or unrestricted codebooks, and optimize within it. We instead ask whether the coding geometry can itself be discovered. We introduce \emph{OrBIT}, a structure-guided embedding compression framework that learns reusable local geometry from orbit dynamics and uses it to...

    arxiv.org/abs/2610.10385 · PDF

  17. 17

    Boosting and the Expressive Power of Simple Weak Learners via the $γ$-VC Dimension

    Arthur da Cunha, Kasper Green Larsen, Liang-Yu Zou

    cs.LG

    Boosting converts weak hypotheses with a small edge over random guessing into highly accurate predictors, but the expressive power of the resulting classifier can depend strongly on the structure of the base class. We study this phenomenon through the $γ$-VC dimension introduced by Alon et al. (STOC 2021). Our first result shows that this parameter characterizes the sample complexity for weak-to-strong learning up to a constant factor scaling...

    arxiv.org/abs/2610.10383 · PDF

  18. 18

    ResidualQuant: KV Cache Quantization for Looped Transformers with 2-Bit Residuals

    Heejun Kim, Junyoung Lee, SangLyul Cho, Dongsu Han, Insu Han, Sehoon Kim

    cs.LG

    Looped Transformers improve parameter efficiency by repeatedly applying shared Transformer blocks over multiple recurrent loops, increasing computational depth without increasing the parameter count. However, KV cache memory still scales with the number of loops, becoming a key memory bottleneck that limits batch size and inference throughput. KV cache quantization can alleviate this bottleneck, but existing methods often suffer substantial...

    arxiv.org/abs/2610.10381 · PDF

  19. 19

    Continual Learning without Continual Training

    Nikita Narayanan, Ritham Majumdarr, Sonali Parbhoo

    cs.LG

    Continual learning requires models to adapt to new domains and new classes while retaining prior knowledge. Many existing methods rely on continued optimization, using regularization, replay, or parameter expansion to prevent new updates from overwriting previously learned knowledge. Instead, we propose replacing continual training with continual inference: a PFN-based model that is meta-trained, and then frozen, adapting to new classes only...

    arxiv.org/abs/2610.10379 · PDF

  20. 20

    Input-Blind Controls Produce Substantial Oracle Headroom for Layer Programs in Multiple-Choice Evaluation

    Yibei Guo, Rui Liu

    cs.LG · cs.CL · cs.SE

    Adaptive computation aims to improve language-model inference by tailoring execution to each input. For layer programs, oracle evaluations use known answers to estimate the potential gain from this flexibility, before a practical selector is available. However, a gain from selection does not by itself explain why the chosen programs help. This study examines this distinction using 32 layer-skipping and repetition programs on two models and...

    arxiv.org/abs/2610.10368 · PDF

  21. 21

    Temporally Interpretable Differentiable Decision Trees

    Eisuke Hirota, Aarav Sane, Rohan Paleja

    cs.LG · cs.RO

    Interpretability offers a solution to safe autonomy by providing transparency into an agent's underlying decision-making model. Within sequential-decision making tasks, differentiable decision trees (DDTs) are one approach to such interpretability, maintaining automatic-differentiable policies while providing humans with a discrete tree-based visualization. Nonetheless, current implementations of DDTs are not well-suited for...

    arxiv.org/abs/2610.10367 · PDF

  22. 22

    Koopman Observers for Diffusion Acceleration: Correcting Feature Forecasts with Shallow Measurements

    Hanru Bai, Yuanchao Xu, Fengyi Li

    cs.LG · math.DS · stat.ML

    Feature caching accelerates diffusion sampling by replacing expensive network evaluations with predictions from previously computed activations. However, forecasts based only on past features cannot directly incorporate changes in the current denoising state. We investigate whether inexpensive, freshly computed features can serve as observations for correcting these predictions. We introduce an observation-corrected Koopman framework for...

    arxiv.org/abs/2610.10366 · PDF

  23. 23

    ORDERS: An Empirical Study of Norm-Rank Aggregation for Personalized Federated Learning

    Koffka Khan

    cs.LG

    Personalized federated learning combines shared representations with client-specific predictors, but the contribution of a server weighting rule can be obscured by local training and evaluation choices. We study ORDERS, a configuration that combines a shared backbone, a private residual adapter and classifier, geometric weights assigned by descending update norm, feature alignment, and private-parameter perturbations. The server computes a...

    arxiv.org/abs/2610.10361 · PDF

  24. 24

    AutoAdapt: Automatic Domain Discovery Enables Low-Cost Extensibility

    Josh McGiff, Salma Mekaoui, Robert Shanahan, Nikola S. Nikolov

    cs.LG

    Instruction-tuned models are deployed into environments where domains are heterogeneous and evolve, yet adding new domains or data typically requires costly retraining. We present AutoAdapt, a modular framework that incorporates new domains and data via targeted single-adapter training without modifying other adapters. The framework automatically discovers latent domains, uses them to train per-domain Low-Rank Adaptation (LoRA) adapters...

    arxiv.org/abs/2610.10349 · PDF

  25. 25

    Average-Reward Reinforcement Learning for Multichain MDPs: A Hierarchical Decomposition Approach

    Huizhen Yu, Isaiah Heidt

    cs.LG · math.OC

    We study learning optimal policies in average-reward multichain Markov decision processes (MDPs), where the optimal gain may depend on the initial state and recurrence structures vary across policies, creating challenges for reinforcement learning (RL) methods. We propose an asynchronous value-iteration-based RL algorithm that requires no model knowledge beyond the MDP's transition graph and leverages Bather's decomposition to hierarchically...

    arxiv.org/abs/2610.10326 · PDF

  26. 26

    HAN-Mamba: Hierarchical Selective State Space Networks for Multi-Scale Financial Volatility Forecasting

    Mihai Bogdan Deaconu, Ioan Daniel Pop

    cs.LG · cs.CE

    Short-horizon realized volatility forecasting requires the integration of market information that evolves at incompatible temporal resolutions, from second-level order book dynamics to weekly regime drift. Our conference work introduced HAN-T, a hierarchical architecture in which scale-specific Transformer encoders process short, mid, and long-horizon streams and a learned attention fuser weighs their contributions. This article replaces the...

    arxiv.org/abs/2610.10323 · PDF

  27. 27

    Thinking in Depth: Retrospective Inference for Tabular Foundation Models

    Hao-Run Cai, Si-Yang Liu, Zi-Jian Cheng, Kun-Yang Yu, Jin-Hao Sheng, Guo Yu, Chonghan Liu, Zhi Zhou, Jun-Peng Jiang,...

    cs.LG

    Tabular foundation models (TFMs) are pretrained across diverse tabular tasks and make predictions on a new table at inference time using its labeled examples as context. Most recent TFMs perform such in-context prediction with stacked Transformer layers, repeatedly transforming how examples are represented and compared. By tracing individual queries through several strong TFMs, we find that predictive refinement is highly uneven across depth...

    arxiv.org/abs/2610.10317 · PDF

  28. 28

    PoreML: A Data-Driven Framework for Learning Multiphase Flow in Porous Media

    Chunyang Wang, Mingrui Zhang, Yuyan Zhang, Linqi Zhu, Xin Ju, Edo Sicco Boek, Martin J. Blunt, Gege Wen

    cs.LG · physics.flu-dyn

    Multiphase flow in porous microstructures is central to CO$_2$ storage, fuel-cell operation, and flip-chip packaging. Predicting these flows remains challenging because wettability and complex pore geometry govern the nonlinear evolution of fluid interfaces. Machine learning holds substantial promise for advancing the field, but progress is constrained by scarce time-resolved 3D datasets and a lack of a unified workflow for training and...

    arxiv.org/abs/2610.10314 · PDF

  29. 29

    Fault-tolerant foundation models

    Trevor McCourt, Ila R. Fiete, Isaac L. Chuang

    cs.LG · cs.AI · cs.AR

    Emerging computer hardware often trades reliability for energy efficiency; here we show that large-language models (LLMs) can be trained to tolerate this unreliability, and that rather than degrading, their error resilience actually increases as they grow. Modified neural scaling laws inferred from 40,000 GPU-hours of training runs on simulated faulty digital hardware quantify this trend and suggest that models learn to compute within "good"...

    arxiv.org/abs/2610.10311 · PDF

  30. 30

    RSIGym: A Flexible Environment for Recursive Self-Improvement

    Fanqing Meng, Lingxiao Du, Haocheng Lu, Qiguang Chen, Ziqi Zhao, Zijian Wu, Jiayuan Zhuo, Mengkang Hu, Michael Qizhe Shieh

    cs.LG

    Recursive self-improvement requires carrying accepted changes into later improvement cycles, while studying agent-proposed changes also requires substantial research infrastructure. Existing settings often leave agents to rebuild routine infrastructure or restrict exploration to individual components. We introduce RSIGym, an agent-native research environment based on Everything as a Service (EaaS). RSIGym exposes training, inference, rollout,...

    arxiv.org/abs/2610.10310 · PDF

  31. 31

    How Do Transformers Learn to Represent Symmetries?

    Eduardo Santos-Escriche, Valerie Engelmayer, Ya-Wei Eileen Lin, Stefanie Jegelka

    cs.LG

    Training Transformer-based architectures with finite data augmentation has become an increasingly popular approach in geometric machine learning. Despite its empirical success, the interplay between the Transformer architecture, invariance to different symmetries, and augmentation budgets remains underexplored. In this paper, we study the ability of a vanilla Transformer to learn various symmetries through finite data augmentation for point...

    arxiv.org/abs/2610.10305 · PDF

  32. 32

    SemanticFold: Latent Sequence Compression SeparatesLanguage Modeling, Decodability, and Reasoning

    Mingyan Liu, Min Huang

    cs.LG · cs.AI · cs.CL

    We study whether latent sequence compression of prompt prefixes preserves the capabilities that large language models rely on during inference. We introduce SemanticFold, a compression scheme that folds prefix hidden states at learned boundaries, and evaluate it across five model scales: Qwen3-1.7B, Qwen3-8B, SmolLM2-1.7B, Pythia-1.4B, and Pythia-6.9B. We use a fixed-target protocol: a frozen prefix is executed natively or compressed, and...

    arxiv.org/abs/2610.10304 · PDF

  33. 33

    Continual Graph Multi-Agent Reinforcement Learning

    Tommaso Marzi, Ahmed Hendawy, Jan Peters, Carlo D'Eramo, Andrea Cini, Cesare Alippi

    cs.LG

    In Continual Multi-Agent Reinforcement Learning (CMARL), agents learn cooperative policies across sequences of tasks, aiming to adapt effectively to new tasks while preserving the ability to solve previously encountered ones. In many applications, tasks differ in their underlying structure, which can represent, for example, distinct operational conditions or target configurations (e.g., different network topologies in power grids or...

    arxiv.org/abs/2610.10302 · PDF

  34. 34

    Revisiting Explainable AI through Model-Independent Concept Dictionaries

    Thomas Schnake, Doreen Schöppenthau, Alexander Meyer, Jacques Corbeil, Klaus-Robert Müller, Grégoire Montavon

    cs.LG · stat.ML

    Modern applications of AI rely on increasingly complex models. Explainable AI (XAI) has emerged as a set of techniques aimed at improving model transparency. However, existing XAI methods typically assume input features to be inherently interpretable, or they rely on intermediate internal abstractions that are difficult to characterize and highly architecture-specific, hindering consistent use across models. To address these limitations, we...

    arxiv.org/abs/2610.10301 · PDF

  35. 35

    Shared Gaussianization: What Gaussian Regularizers Certify About Contrastive Learning, and What They Miss

    Ruoyu Zhao, Yuting Chen, Jinheng Zhang, Zhehao Zou, Tong Che

    cs.LG · stat.ML

    What can a distribution-matching regularizer such as SIGReg in LeJEPA certify about contrastive learning? We study shared Gaussianization (SG), a characteristic-function Gaussianity test on the average of two normalized views, scaled by an independent $χ_d$ radius. Because disagreeing views shorten the average, one test detects both misalignment and non-uniformity. SG vanishes exactly at the aligned, uniform minimizers of population InfoNCE,...

    arxiv.org/abs/2610.10299 · PDF

  36. 36

    Physics-Aligned Electronic Ground-State Learning Improves Generalization

    Eike S. Eberhard, Xaver Kainz, Viktor Kotsev, Abdulrahman Aldossary, Stephan Günnemann

    cs.LG · physics.chem-ph · quant-ph

    Machine-learned interatomic potentials (MLIPs) excel at in-distribution tasks, accelerating drug and material development, yet they struggle to generalize out-of-distribution. We propose to push the cost-accuracy Pareto frontier by designing observable-agnostic electronic ground-state descriptor models (GSMs) with computational costs situated between MLIPs and Kohn-Sham density functional theory (KS-DFT). We align the learning objectives and...

    arxiv.org/abs/2610.10298 · PDF

  37. 37

    PatchBench: Measuring Collateral Damage in Activation Patching

    Alexi Canesse, Mathis Le Bail, Maël Jenny, Clément Elliker, Mahammed El Sharkawy, Sonia Vanier

    cs.LG · cs.CL · cs.CR

    An LLM safety patch can pass a benchmark while still being a poor repair. This risk is especially acute for jailbreak repairs, where the goal is to correct a specific unsafe behaviour without changing unrelated behaviours. A patch may block exact evaluation prompts yet fail on close harmful variants, or suppress harmful behaviour by over-refusing benign prompts that share its wording or structure. Existing protocols primarily test whether...

    arxiv.org/abs/2610.10276 · PDF

  38. 38

    Sparse Planning in Visual World Models via Cost Gradients

    Yingchen Xu, Edward Grefenstette

    cs.LG

    Token-based world models enable fine-grained latent planning, but repeatedly processing large spatial token grids makes action search expensive. We introduce COSTGRAD, a training-free, goal-conditioned selector that ranks spatial tokens by the gradient norm of the planning cost with respect to each input token. By deriving importance from the downstream control objective, COSTGRAD targets tokens that matter for planning rather than merely for...

    arxiv.org/abs/2610.10274 · PDF

  39. 39

    A Closed-Loop Non-Asymptotic Convergence Analysis of PPO with Learned Critics and Clipping

    Junwei Su, Mengfan Liu, Yanyong Zhang, Chuan Wu

    cs.LG

    Despite its widespread use, Proximal Policy Optimization with clipping (PPO-Clip) remains difficult to tune, and the interactions among critic learning, clipping, and rollout reuse remain incompletely understood. We develop a \emph{non-asymptotic} analysis of PPO-Clip as a \emph{closed-loop actor--critic} system. It captures actor--critic coupling, nonsmooth probability-ratio clipping, finite-batch reuse, and predictable early stopping under...

    arxiv.org/abs/2610.10273 · PDF

  40. 40

    PairAudit: Guiding Human Review with Graph Tokens under Distribution Shift

    Jiran Tao, Binyan Jiang

    cs.LG · cs.CR

    Intrusion detectors can confidently misclassify attacks that were not seen during training. Human review can correct these errors, but only a limited number of cases can be checked. Uncertainty-based review may overlook confident errors, while anomaly scores alone do not show whether changing the review plan will correct more errors. We introduce PairAudit to find overlooked errors and improve review under a fixed budget. Its graph tokens...

    arxiv.org/abs/2610.10260 · PDF

  41. 41

    Logarithmic Regret via Passive Change Detection in Piecewise-Stationary Self-Tuning Regulation

    A. Ch. Madhusudanarao, Rahul Singh

    cs.LG · cs.AI

    We study minimum-variance control of an unknown autoregressive system with exogenous inputs and coefficients that change at unknown times. Under bounded independent disturbances, fixed detection gaps, stability and feasibility conditions, and sufficient time between changes, we prove \(O((C+1)\log((T+1)/δ))\) regret with probability at least \(1-δ\), where \(T\) is the horizon and \(C\) the number of changes. Unlike switching bandits, where...

    arxiv.org/abs/2610.10250 · PDF

  42. 42

    Stationary Bias and Extrapolation in Nonlinear Two-Timescale Stochastic Approximation

    A. Ch. Madhusudanarao, Rahul Singh

    cs.LG · cs.AI

    Constant-step stochastic approximation generally has a nonzero stationary mean error that persists under time averaging. This paper studies that error for nonlinear two-timescale recursions driven by an exogenous finite-state Markov chain. Under stated smoothness assumptions and conditions on the stationary distribution, we derive a first-order bias expansion whose error bound remains uniform as the slow step size becomes much smaller than...

    arxiv.org/abs/2610.10246 · PDF

  43. 43

    MorphCL: Morphological Contrastive Learning for Inertial-based Human Activity Recognition

    Marius Bock, Yuwei Zhang, Juergen Gall, Michael Moeller, Kristof Van Laerhoven, Cecilia Mascolo

    cs.LG · cs.HC

    Despite the ubiquity of sensors in wearable and mobile devices and the abundance of human movement data they generate, translating unlabeled recordings into foundational motion models remains an open challenge. Self-supervised learning (SSL) has alleviated the need for costly annotations, yet existing approaches leave the global structure of large-scale motion data largely untapped, relying on randomly sampled batches and local comparisons...

    arxiv.org/abs/2610.10245 · PDF

  44. 44

    ProtocolMatch: Protocol-Dependent Model Selection for Scientific Dynamics Forecasting

    Lu Wei, Yufeng Wang, Haibin Ling

    cs.LG · quant-ph

    Scientific dynamics forecasting is often framed as an architecture choice, although deployment is also determined by observed history, rollout feedback, compute budget, physical objective, and test distribution. We formulate protocol-dependent model selection and introduce ProtocolMatch, a compute-matched, validation-selected, and failure-preserving evaluation framework. On driven quantum-spin dynamics, we compare recurrent,...

    arxiv.org/abs/2610.10239 · PDF

  45. 45

    From Prompts to Trees: Effective LLM-Guided Tree Generation for Few-Shot Tabular Classification

    Yue Qiu, Zekang Du, Yiqun Diao, Bingsheng He, Qinbin Li

    cs.LG · cs.AI · cs.CL

    While Large Language Models (LLMs) possess rich world knowledge and impressive generalization capabilities, their direct application to tabular data classification is hindered by high inference costs and limited interpretability. In contrast, decision trees are fast and transparent but often underperform in low-data regimes. In this work, we propose a novel framework that bridges these paradigms by distilling LLM knowledge into interpretable...

    arxiv.org/abs/2610.10227 · PDF

  46. 46

    Evaluating Sequence Assembly Strategies for Differentially Private Synthetic Time-Series Forecasting

    Guoxiong Long, Huizhen Huang, Qikun Cai, Tao Huang, Chen Hou

    cs.LG

    Differentially private time-series generators commonly produce fixed-length synthetic windows, whereas downstream forecasting models often require long continuous training sequences. How these windows are assembled after generation can therefore alter the effective synthetic data presented to a forecaster, even when the trained generator remains unchanged. We study this post-generation sequence assembly process by systematically varying...

    arxiv.org/abs/2610.10222 · PDF

  47. 47

    Finite-Sample Approximation of Hessian-Guided Perturbed Wasserstein Gradient Flows

    Ryotaro Kawata, Atsushi Nitanda, Taiji Suzuki

    cs.LG · stat.ML

    Wasserstein gradient flow extends gradient descent to probability measures. Its Hessian-guided perturbed variant (PWGF) adds Gaussian perturbations to escape saddle points in nonconvex problems. We investigate when its approximation by finitely many interacting particles remains accurate over growing time horizons. Our analysis retains the curvature accumulated along the population-driven reference path: negative curvature can amplify...

    arxiv.org/abs/2610.10218 · PDF

  48. 48

    Edge Accuracy Is Not Enough: Why Dynamics-Learned Structure Fails to Transfer to Inverse Problems

    Nicholas Tan Jerome, Fangnian Wang

    cs.LG

    A natural strategy for inverse problems with scarce labelled data is to transfer relational structure learned from abundant forward-simulation data. We show this strategy fails systematically, even when it satisfies the standard theoretical justification for why structure should help. We prove that approximate structure provides estimation-error benefits whenever the edge error satisfies $Δ< n^2 - kn$, reducing sample complexity from $O(n^2)$...

    arxiv.org/abs/2610.10213 · PDF

  49. 49

    OrthoGen: A Generative Orthogonal Learner for Time-Varying Treatments

    Tomàs Garriga, Valentyn Melnychuk, Konstantin Hess, Eduard Serrahima de Cambra, Axel Brando, Gerard Sanz, Stefan Feuerriegel

    cs.LG

    Estimating conditional distributional potential outcomes (CDPOs) over time is important in medicine (e.g., to estimate patient-specific risks under different treatment sequences). However, this task is challenging because of time-varying confounding, yet existing adjustment strategies for this task are limited. In this paper, we aim to learn CDPOs under time-varying treatments using flexible generative models. Our contributions are two-fold....

    arxiv.org/abs/2610.10210 · PDF

  50. 50

    How to train your model organism

    Xilin Wang, David Bau, Byron C. Wallace

    cs.LG

    Model organisms of alignment-relevant behaviors (e.g., backdoors, sycophancy, spurious correlations) have emerged as a key tool for evaluating whitebox interpretability techniques. We argue that the prevailing practice of training model organisms to a single objective of installing the target behavior is insufficient and propose validating model organisms with respect to three objectives with associated metrics: target-behavior installation,...

    arxiv.org/abs/2610.10203 · PDF

  51. 51

    Robust Decentralized Fairness Auditing

    Sayan Biswas, Jade Garcia Bourrée, Anne-Marie Kermarrec, Palak, Martijn de Vos

    cs.LG

    Emerging legislation requires large language models (LLMs) to be audited for compliance with regulatory standards, particularly fairness. Such black-box audits typically assume a single auditor with access to a large, representative set of queries. In practice, it can be difficult for an auditor to obtain such a query set, but multiple auditors can together cover the relevant demographic groups by auditing the LLM collaboratively with their...

    arxiv.org/abs/2610.10199 · PDF

  52. 52

    Pre-training of Bayesian Optimization Algorithm through Bayesian Optimization

    Satoshi Katayama, Shoyo Hunt, Shintaro Masuda, Masayuki Karasuyama

    cs.LG · stat.ML

    Bayesian optimization (BO) is widely used as a standard approach for expensive black-box optimization. However, BO algorithms often involve parameters that must be specified in advance, and their performance can strongly depend on these choices. We propose a framework for optimizing such parameters using sample paths drawn from a Gaussian process (GP) inferred from the information available at the start of BO. We use cumulative regret as the...

    arxiv.org/abs/2610.10186 · PDF

  53. 53

    A Unified Information-Theoretic Approach to Constrained Multi-Fidelity Multi-Objective Bayesian Optimization

    Rikuto Matsumoto, Masanori Ishikura, Masayuki Karasuyama

    cs.LG

    Bayesian optimization often involves multiple objectives, constraints, and fidelity levels. We address the challenge of jointly selecting where and at which fidelity to evaluate to identify the highest-fidelity feasible Pareto frontier in this combined setting. From a unified information-theoretic perspective, we measure query utility by the information gain about this frontier, provided by an observation. Since this mutual information is...

    arxiv.org/abs/2610.10174 · PDF

  54. 54

    m-Set Adversarial Bandits with Winner Feedback

    Nicolò Cesa-Bianchi, Matteo Papini

    cs.LG · stat.ML

    We show upper and lower bounds on the regret of $m$-set adversarial bandits for different utilities (winner reward or sum of rewards) and feedback models (winner index, winner reward, sum of rewards, and their combinations). By comparing to standard bounds for combinatorial and MNL bandits, our results reveal how subtle changes in the setting can have a dramatic impact on the learning rates. Our main technical contributions are the...

    arxiv.org/abs/2610.10128 · PDF

  55. 55

    What Can a Gaussian Process Design Test

    Ivan De Boi, Marnix Van Soom

    cs.LG

    A Gaussian process (GP) model can agree with the data for two reasons: its assumptions are right, or the chosen inputs could never have shown that they are wrong. The distinction can be checked from the design before any responses are observed. Every model implies relations that its noiseless responses must satisfy at the chosen inputs, such as the middle value lies on the line through its two neighbours. For GPs built from finitely many...

    arxiv.org/abs/2610.10122 · PDF

  56. 56

    YANchor-4B: Effective Long-Horizon Reasoning in O(N) Time with O(1) Memory

    Huishan Ji, Hua Xu, Weiming Zhang, Qirui Ye

    cs.LG · cs.CL

    Long-horizon reasoning demands access to earlier information at a manageable generation cost. Full-history attention incurs growing storage and computation, while recurrent compression can lose precise details. Therefore, we present YANchor-4B, a general-purpose recurrent model that preserves crucial memory as ANchors for retrieval during subsequent reasoning. Beyond $O(N)$-time generation and $O(1)$ memory, YANchor enables effective...

    arxiv.org/abs/2610.10118 · PDF

  57. 57

    CAFE+FNO: Fourier Kernel Generation via Multiplicative Feature Composition

    Hyungjoon Juen, Minwoo Shin

    cs.LG · math.NA

    The Fourier Neural Operator (FNO) learns solution operators of partial differential equations (PDEs) through Fourier-space kernel parameterization, but frequency truncation can limit the learning of high-frequency variations. AM-FNO and SirenFNO generate kernels for all grid modes from spectral coordinates using shared networks, making coordinate encoding and generator design important. Recent work on implicit neural representations (INRs)...

    arxiv.org/abs/2610.10105 · PDF

  58. 58

    Multi-Agent Coordination via Support-Preserving Distillation

    Sangmin Lee, Youngju Na, Chanmi Lee, Sung-eui Yoon

    cs.LG · cs.MA · cs.RO

    Offline MARL increasingly relies on generative policies to model multimodal joint behavior, typically by distilling a centralized teacher into decentralized one-step actors under the CTDE. We identify a failure mode at the teacher training stage: standard flow-based teachers pair noise with replay targets independently, so nearby noise samples can be routed toward conflicting coordination modes. The teacher then produces samples between valid...

    arxiv.org/abs/2610.10087 · PDF

  59. 59

    Activation-Aware Weight Tensorization: A Calibration-Time Preconditioner for Tensor-Network LLM Compression

    Alessandro Beatini, Marco Maronese, Emanuele Rodolà

    cs.LG · cs.AI

    Post-training tensor-network compression replaces Transformer linear layers with Tensor Train (TT) or Tree Tensor Network (TTN) operators, but standard decompositions minimize weight-space Frobenius error rather than functional error under the layer's activation distribution. We propose Activation-aware Weight Tensorization (AWT), a training-free calibration wrapper that preconditions each weight matrix with a diagonal activation-derived...

    arxiv.org/abs/2610.10085 · PDF

  60. 60

    Efficient Provably Private Classification with a Tabular Foundation Model

    Talal Alrawajfeh, Cristiana Diaconu, Ossi Räisä, Sebastian Rodriguez Beltran, Yuan He, John Bronskill, Richard E....

    cs.LG · cs.CR

    Tabular data underpin prediction and decision-making in medicine, finance, government and science, but often contain sensitive individual-level information, creating a need for accurate prediction while preserving privacy. Traditional private learning provides formal privacy guarantees, but requires slow dataset-specific optimisation, suffers substantial utility loss under strong privacy, and is often difficult to apply correctly. Tabular...

    arxiv.org/abs/2610.10068 · PDF

  61. 61

    WxFM-XL: Adapting Univariate Foundation Models to Multi-Station Weather Forecasting

    Xiao Wang, Changjian Chen, Zhuo Tang, Rongwen Li, Hongwu Liu, Kenli Li

    cs.LG

    With the rise of univariate time series foundation models (e.g., Sundial, Timer), initial efforts have been made to extend them to multivariate settings. However, these models mainly focus on modeling correlations among variables. When they are applied to multi-station weather forecasting, two important factors are often overlooked: (1) the spatial information of stations, and (2) different error priors of different stations relative to the...

    arxiv.org/abs/2610.10057 · PDF

  62. 62

    Matching of signal, noise and hardware timescales for filtering and forecasting of correlated noise signals

    Joshua Donald, Alex Gabbitas, Arthur G. T. Coveney, Sergey Savel'ev, Pavel Borisov

    cs.LG · cond-mat.dis-nn · physics.app-ph · physics.data-an

    Physical reservoir computing exploits the nonlinear dynamics of physical systems to process time-dependent data with greater energy efficiency than conventional machine learning approaches. However, physical reservoirs have fixed intrinsic response timescales, whereas real-world signals combine deterministic and stochastic components across multiple timescales. Here we show, using a nanoporous niobium oxide reservoir, synthetic noisy signals...

    arxiv.org/abs/2610.10037 · PDF

  63. 63

    Oscillatory Neural Dynamics over Sheaves

    Jan-Willem Van Looy, Alessandro Trenta, Alessio Gravina, Alessio Borgi, Ferdinando Zanchetta, Pietro Liò, Davide...

    cs.LG

    Effective long-range propagation remains a central challenge in graph neural networks, as increasing a model's propagation depth does not guarantee that distant nodes effectively influence each other. Sheaf neural networks enrich graph propagation through matrix-valued transport between stalks; still, this expressivity alone does not automatically imply effective long-range communication. We introduce ONDA, a long-range graph learning...

    arxiv.org/abs/2610.10018 · PDF

  64. 64

    A Drosophila Whole-Connectome Network Can Learn Human-Designed Cognitive Tasks

    Joonghui Cho, Minchan Kang, Daeshik Kim

    cs.LG

    Can a biological wiring diagram serve as a useful computational substrate beyond the behaviors for which it evolved? We use the publicly released MaleCNS v1.0 connectome, reconstructed from a single adult male Drosophila specimen, as the fixed recurrent topology of an artificial network. We train separate models for bounded addition and for a controlled grounded relational language task built from a fixed 100-word lexicon. In both models, one...

    arxiv.org/abs/2610.10014 · PDF

  65. 65

    Beyond Reward Suppression: Near-Optimal Offline Attacks on Warm-Start Bandits with Bounded Rewards

    Qirun Zeng, Manhin Poon, Xiangxiang Dai, Qixin Zhang, Jinhang Zuo

    cs.LG · cs.AI

    Adversarial attacks on bandits aim to mislead a learner toward a target arm while keeping the attack cost small. Existing attacks typically achieve this by suppressing non-target arms. In practice, however, manipulation such as fake reviews often directly promotes the target item. We study this gap through bounded offline attacks on warm-start bandits, where an attacker can inject only valid action-reward pairs into the warm-start history...

    arxiv.org/abs/2610.10000 · PDF

  66. 66

    Temporal Predictive Multiplicity: Equally Accurate Time Series Models Yield Different Forecast Trajectories

    Emanuele Albini, Francesca Toni, Saumitra Mishra, Francesco Leofante

    cs.LG · cs.AI

    Models with near-identical predictive performance can yield substantially different predictions, a phenomenon known as predictive multiplicity. Prior work has mostly studied this at the level of individual scalar outputs. In time-series forecasting, however, predictions across horizons jointly define a trajectory, and horizon-wise comparisons can hide important differences in predictive behavior. To address this problem, we introduce temporal...

    arxiv.org/abs/2610.09994 · PDF

  67. 67

    Efficient Patch-Based Anomaly Detection Fused with Diffusion Driven Generative Modeling for Semiconductor Wafer Bin Map Open Set Anomaly Detection

    Limon Bin Hossain, Md Sadib Rahman Ananta

    cs.LG · cs.AI

    Spatial defect signatures on wafer bin maps (WBMs) trace yield loss to specific process faults, yet supervised classifiers recognize only the defect types seen during training, and one-class detectors built on a single mechanism tend to capture either local structural deviations or global distributional violations, but rarely both. This work proposes a hybrid one-class framework that couples a patch-based student-teacher detector...

    arxiv.org/abs/2610.09993 · PDF

  68. 68

    TR-PTQ: High-Accuracy Integer-Only Transformer Post Training Quantization via Taylor Region Reformulation

    Eliyahu Levy, Adam Teman, Yoni Pugachov

    cs.LG

    Post-training quantization (PTQ) enables efficient deployment, yet transformer architectures remain challenging to quantize due to nonlinear layers. While existing methods attribute accuracy loss to insufficient numerical precision, often necessitating floating-point fallbacks, we demonstrate that degradation is actually driven by specific structural error sources. We find that learned scale parameters in normalization layers and compounded...

    arxiv.org/abs/2610.09969 · PDF

  69. 69

    Force without transmission: a depth-induced rank collapse that no loss on the representation reopens

    Martin Hofmann, Patrick Mäder

    cs.LG

    Training can drive a transformer into a rank collapse: all token representations point in one direction, and learning stops. In a related collapse of attention, a loss term with a bounded corrective force repairs the network during the run. We ask whether such a term repairs rank collapse. We collapse small transformers by weakening their skip connection and treat copies of the collapsed network. No added loss term repaired the collapse,...

    arxiv.org/abs/2610.09958 · PDF

  70. 70

    Learning Traffic Flow Dynamics with Stochastic Physics-Informed Neural Cellular Automata

    Federica Bragone, Matthieu Barreau

    cs.LG

    Traffic flow modeling is essential for understanding and predicting the collective dynamics of vehicles on road networks. Cellular automata provide a simple, interpretable yet powerful framework for representing these dynamics via local interaction rules, while retaining the ability to reproduce complex macroscopic traffic phenomena. However, learning local transition rules from data while preserving physically meaningful constraints remains...

    arxiv.org/abs/2610.09946 · PDF

  71. 71

    Expected Sample Complexity in Multi-Armed Bandits

    Nadav Sukenik, Nadav Merlis

    cs.LG · stat.ML

    Sample complexity is a widely used metric in sequential decision-making problems, defined as the number of suboptimal decisions during the interaction between the agent and an environment. We study the sample complexity of stochastic multi-armed bandit problems and introduce the expected sample complexity performance measure, analyzing it in a novel framework called approximately correct in expectation (ACE). We show that ACE guarantees imply...

    arxiv.org/abs/2610.09929 · PDF

  72. 72

    KGATE : a Knowledge Graph Embedding Training Environment

    Benjamin Loire, Galadriel Brière, Célia Brahimi, Antoine Toffano, Anaïs Baudot

    cs.LG · cs.AI

    Knowledge graph embedding (KGE) models encode the entities and relations of a knowledge graph into a low-dimensional latent space, enabling tasks such as classification or link prediction. Most KGE models follow an autoencoder architecture, in which an encoder projects the knowledge graph into the latent space and a decoder reconstruct it. Combining both encoder and decoder components is increasingly needed, yet existing libraries rarely...

    arxiv.org/abs/2610.09927 · PDF

  73. 73

    Eigenvalues of the Hessian in Deep Learning: The Origin of Symmetry and Its Breaking

    Yossi Arjevani

    cs.LG · math.OC · stat.ML

    Hessian spectra at trained models in deep learning exhibit a persistent pattern: eigenvalues organize into distinct clusters, including a large bulk near zero and a few isolated outliers. This paper shows that a natural account of these spectral phenomena emerges when the original setting is understood as a departure from a nearby, otherwise hidden, highly symmetric reference. Modifications, including changes to the architecture, data...

    arxiv.org/abs/2610.09919 · PDF

  74. 74

    NeuralZip: Reusable Setup for Fast Lossless Compression

    Martín Bravo, Samuel Horváth, Gonzalo Navarro, Andrés Abeliuk

    cs.LG

    Lossless compression can reduce the storage and movement of model weights without changing their floating-point values, but repeated statistical analysis and code construction add computational overhead. We study whether the statistical structure of exponents can be prepared once and reused. For this, we introduce NeuralZip, which groups chunks with similar exponent distributions, shares Huffman codes, and selectively represents recurring...

    arxiv.org/abs/2610.09916 · PDF

  75. 75

    RollVerify: Bridging Efficiency and Accuracy in Long-Tail Rollout Reinforcement Learning

    Yongqiang Yao, Jinru Tan, Kaihuan Liang, Zixin Yin, Yazhe Niu, Ruihao Gong, Dahua Lin, Ningyi Xu

    cs.LG · cs.AI

    Reinforcement learning is crucial for improving large language models' reasoning and generalization. It relies on massive rollouts whose lengths become increasingly long-tailed as context windows grow. In on-policy training, these long-tail rollouts can result in GPU bubbles, reducing system utilization and limiting RL scalability. Asynchronous or partial-rollout methods improve throughput by relaxing synchronization, but inevitably introduce...

    arxiv.org/abs/2610.09914 · PDF

  76. 76

    Identifiability of a dissipative knowledge-dynamics model: exact recovery under designed excitation, degeneration on observational data

    Arman Kostanian, Armen Beklaryan

    cs.LG · math.OC · stat.ML

    Human learning is a dissipative dynamical process: mastery accumulates through practice, decays through forgetting, and propagates across interdependent concepts. We model it as a nonlinear dissipative system of ordinary differential equations whose parameters are mechanistically meaningful (a concept-transfer matrix encoding prerequisite coupling, per-concept forgetting rates, and a saturating practice-response gain), and we study when those...

    arxiv.org/abs/2610.09889 · PDF

  77. 77

    Layerwise Error Attribution for Fast and Robust Mixed-Precision Post-Training Quantization

    Samy Houache, Yann Traonmilin, Jean-François Aujol

    cs.LG

    Mixed-precision post-training quantization is a network compression method that assigns bits layer by layer, under a global memory budget using a small calibration set. The main difficulties are to overcome the combinatorial nature of the allocation problem and to manage the sensitivity to small, potentially corrupted databases. Hence, an efficient allocation method should be fast to compute and preserve model quality when calibration data...

    arxiv.org/abs/2610.09877 · PDF

  78. 78

    MUNITE: Unified Multimodal Latent Inference for Any-to-Any Multimodal Generation

    Kyeongmin Yeo, Minhyuk Sung

    cs.LG

    We introduce MUNITE, a latent-variable framework for flexible any-to-any multimodal generation that treats encoding and latent generation as the same inference problem under different amounts of observed evidence. Given any subset of modalities, MUNITE models the conditional distribution over the latent representation associated with the complete observation. Full observation recovers deterministic encoding, no observation recovers the latent...

    arxiv.org/abs/2610.09866 · PDF

  79. 79

    Stream-Based Active Learning with Cooperative Neural Networks for Data-Efficient Partial Inverse Design: An Automotive Glass Run Channel Case Study

    Agung Nugraha, Hyerin Kwon, Heungjun Im, Gian Antariksa, Jihwan Lee

    cs.LG · cs.AI

    Inverse design in engineering often runs into a simple problem. Each labeled training sample must be produced through expensive simulation, so building a large dataset is slow and costly. This study addresses that problem for partial inverse design, where only some design variables are specified and the rest must be inferred to reach a target performance value. We propose CoNN-AL, a framework for data-efficient partial inverse design that...

    arxiv.org/abs/2610.09848 · PDF

This edition is part of The Daily Abstract — cs.LG archive. Subscribe to receive these in your inbox each morning, automatically translated to Spanish, with reply-to-PDF: arxivdaily.ignorelist.com.

Colophon Set in Georgia, with system sans for interface chrome and a monospaced stack for code and paper identifiers. Sole accent: amber #D99C5E. Built and served on an always-free VM. The masthead is set 14% letterspaced because newspapers do that and it works.