cs.LG · 2026-10-15 · No. 144

Machine Learning, 2026-10-15.

85 new papers in cs.LG. Titles, authors, abstracts. Links to arXiv. Want this in your inbox every morning? Subscribe →

01 — The papers

85 entries
  1. 01

    Your Transformer Is Secretly Elastic

    Walter Simoncini, Dheeraj Varghese, Gertjan J. Burghouts, Cees G. M. Snoek

    cs.LG · cs.AI

    Scaling up transformers has produced models with broad capabilities, deployed across heterogeneous hardware and fluctuating workloads. Yet, model families offer only a few fixed sizes (e.g., S, B, L), forcing users to commit to a model based on hardware and latency constraints. This rigidity is often suboptimal under varying query loads. We introduce a retraining-free method that elastifies pre-trained transformers into a single model that...

    arxiv.org/abs/2610.20588 · PDF

  2. 02

    Continuum Attention

    Afshin Khadangi

    cs.LG · cs.AI

    We introduce Continuum Attention (CA), a causal Transformer attention operator that replaces discrete key and value lookup with integration over a continuous, piecewise linear memory curve. Each query induces an exponential relevance density along the curve. Integrating the density and interpolated values yields exact normalized attention from closed form exponential moments, including half token boundary contributions. The gradients follow...

    arxiv.org/abs/2610.20586 · PDF

  3. 03

    Learnable Hodge-Laplacian Driven Mixture of Experts for Steady-PDE Surrogates on Coarse Unstructured Meshes

    Stavros Nousias, Akis Nousias, Konstantinos Gkrispanis

    cs.LG · cs.CV

    Current approaches for steady partial differential equations surrogate models on unstructured meshes apply a single mixing mechanism to the whole solution. This paper proposes an architecture that partitions the solution among vanilla transformer experts, each focused on a different spectral scale. The partition is built from a discrete Hodge Laplacian $L_0=\star_0^{-1} d_0^{\top}\star_1 d_0$, that serves as a fitting layer measuring...

    arxiv.org/abs/2610.20585 · PDF

  4. 04

    One Task, Many Interfaces: Simple Cross-Interface Consistency Training Improves LLM Agent Robustness

    Artin Tajdini, Manpreet Kaur, Peng Qi

    cs.LG · cs.AI · cs.CL

    Post-training binds the competence of an LLM agent to the interface it was trained on. Semantics-preserving perturbations of that interface, such as renaming an action or changing the notation, leave the underlying task unchanged yet make the agent worse at it. Scoring each sample independently means interface randomization only broadens this dependence from a single interface to the training interface distribution, so out-of-distribution...

    arxiv.org/abs/2610.20576 · PDF

  5. 05

    Recovering Reliable Supervision via Critique-Guided On-Policy Distillation

    Kaixiang Zhao, Tianrun Yu, Shangzhe Li, Yuxiao Yang, Amanda Hughes, Weitong Zhang, Taylor W. Killian

    cs.LG · cs.AI

    On-policy distillation (OPD) trains a student model on its own rollouts with token-level supervision from a teacher, assuming that the teacher can reach a correct answer from any prefix the student writes. We call this property the recoverability of the teacher and show it fails on incorrect rollouts: the teacher follows student errors, although it recognizes them. We propose CriticOPD, using error recognition to restore recoverability. The...

    arxiv.org/abs/2610.20572 · PDF

  6. 06

    Detection of Potato Plant Leaf Diseases using Convolutional Neural Networks

    Haimanot Bitew Alemayehu, Simachew Melaku Ayalew, Chilotie Desalew Mengesha, Ayisheshim Almaw Bogale, Anteneh Birhan...

    cs.LG · cs.AI

    Potato is an important food-security crop in Ethiopia, but its production is affected by foliar diseases, particularly early blight and late blight. This study develops a lightweight convolutional neural network (CNN) to classify potato leaves into three categories: healthy, early blight, and late blight. The proposed model was trained using 2,215 images from the PlantVillage dataset and evaluated on an independent field dataset containing...

    arxiv.org/abs/2610.20544 · PDF

  7. 07

    Revisiting Simple Baselines for Relational Deep Learning

    Viet The Nguyen, Vijay Prakash Dwivedi, Gunther Gust

    cs.LG

    Relational deep learning has emerged as a powerful paradigm for learning end-to-end predictive models over relational databases by representing them as temporal heterogeneous graphs. Recent advances based on graph neural networks and Transformers have achieved strong performance across diverse domains. However, the effectiveness of simpler relational baselines remains underexplored, making it difficult to assess how much architectural...

    arxiv.org/abs/2610.20534 · PDF

  8. 08

    Generative Representation Learning for Scalable Latent Tokenization

    Alan Baade, Kyle Sargent, Vincent Sitzmann, Eric Ryan Chan, Justin Johnson, Li Fei-Fei, Gordon Wetzstein

    cs.LG · cs.CV

    We introduce an approach for self-supervised representation learning designed to be used as a tokenizer for generative modeling. Our objective supervises a representation loss by factorizing the pointwise mutual information of the data, which we estimate by taking information differences between conditional predictions from a learned generative model. We justify our algorithm theoretically and empirically, demonstrating that our generative...

    arxiv.org/abs/2610.20533 · PDF

  9. 09

    The Physics Is the Network: Fast, Training-Free Neural Emulation

    Luka Todorovic, Kaushal Gianchandani, Sai Ravela

    cs.LG · eess.SY · nlin.CD · stat.AP

    Neural emulation need not require learning a substitute for known physics. We investigate PolyNets that couple a continuous-time neural representation of polynomial dynamics with a simple recurrent network for discrete-time integration. When the governing coefficients are known, the resulting tensor graph requires no training and preserves the prescribed numerical scheme while enabling efficient hardware execution and parallel simulation....

    arxiv.org/abs/2610.20525 · PDF

  10. 10

    Bayesian Learning to Select Fair Classifiers

    Saatvik Kher, Hyungrok Do, Judy Zhong, Padhraic Smyth

    cs.LG

    Evaluations of classifiers (AI or human or agents) often assume a batch setting in which every agent produces labels on every example in a test set, which can be expensive or impractical in many contexts. This motivates online policies that adaptively select agents to query as examples arrive and update policies as outcomes are observed. We study the problem of selecting, in an online stream, the agent that best balances (i) predictive...

    arxiv.org/abs/2610.20514 · PDF

  11. 11

    A Two-Level Clinical Novelty Prioritization Framework for Disease Trajectories Discovered from Longitudinal Electronic Health Records

    Jin-Jia Zheng, Vincent S. Tseng

    cs.LG

    Graph-based analysis of electronic health records (EHRs) yields disease-trajectory catalogs, but structural support does not establish semantic novelty. We present a two-level framework to prioritize 99,527 branching motifs previously discovered by an upstream patient-disjoint, train-only MIMIC-IV v3.1 pipeline and frozen after validation-based catalog selection. The framework separates path-level and node-to-endpoint semantic sparsity from...

    arxiv.org/abs/2610.20512 · PDF

  12. 12

    Graph-Enhanced Disease Trajectory Discovery and Temporal Endpoint Forecasting from Longitudinal Electronic Health Records

    Jin-Jia Zheng, Vincent S. Tseng

    cs.LG

    Longitudinal electronic health records (EHRs) can reveal disease-progression structure, while useful forecasting requires leakage-controlled discovery, censor-aware outcomes, and inspectable trajectory evidence. We developed a graph-enhanced framework in MIMIC-IV v3.1 that splits patients before graph construction, learns train-only consecutive-admission transitions, and freezes endpoint-specific branching-motif policies selected on...

    arxiv.org/abs/2610.20502 · PDF

  13. 13

    Hindsight-Privileged RLVR: Single-Rollout Token Credit Without Group Sampling

    Wei Chen, Yubing Wu, Zhou Xu, Haoyue Zhang, Delu Zeng, Jiahao Wang, Junmei Yang, Pan tiewen, Jingfeng Liu, Zhiheng...

    cs.LG · cs.AI

    Reinforcement learning with verifiable rewards (RLVR) trains language models from checked task outcomes, but assigning a terminal reward to individual reasoning tokens or actions remains difficult. Group-relative methods such as Group Relative Policy Optimization (GRPO) sample several attempts per prompt, but lose their reward-based learning signal when all attempts receive the same outcome. We propose Hindsight-Privileged RLVR (HP-RLVR), a...

    arxiv.org/abs/2610.20497 · PDF

  14. 14

    Average-Reward Policy Identification from a Single Continuing Trajectory

    Tongxi Wang

    cs.LG

    In reinforcement learning, an agent must explore an unknown environment to learn how to make good decisions. Unlike methods that can freely sample any state--action pair, an agent learning from a single trajectory must spend additional time navigating between states. It is therefore natural to expect that learning from a single trajectory requires more experience. In this paper, we show that this additional cost is not always necessary. For...

    arxiv.org/abs/2610.20486 · PDF

  15. 15

    Decoupled Marginal Sharpening for Training-Free Inference-Time Scaling

    Weixin Wang, Wei Deng, Anderson Schneider, Yuriy Nevmyvaka, Pan Xu, Andrew Bennett

    cs.LG

    We study how to improve LLM reasoning at inference time without updating model weights or using external reward models. Recent reward-free power-sampling methods define the sequence-level power target $π_α(y\mid x)\propto p_0(y\mid x)^α$, where $α>1$, which increases the relative probability of completions that are already likely under the base model. Existing methods use Metropolis--Hastings or Sequential Monte Carlo (SMC) to approximately...

    arxiv.org/abs/2610.20483 · PDF

  16. 16

    Selective Growing Q-Networks (SGQN): An Analysis of Adaptive Action Grids in Decoupled Q-Learning

    Parham Nooranbakht, Seyed Mahdi Basiri Azad, Joschka Bödecker

    cs.LG · cs.AI

    Value-based Q-learning for continuous control can discretise the action space, but the resolution of the discretisation trades early exploration against final precision. Growing Q-Networks (GQN) grow the resolution uniformly from coarse to fine. We introduce Selective Growing Q-Networks (SGQN), which keeps decoupled Q-learning and replaces uniform growth with an adaptive action grid: bins are added next to bins with high estimated Q-value,...

    arxiv.org/abs/2610.20463 · PDF

  17. 17

    Learning by Examples: Power Grid Steady State Analysis with a Tabular Foundation Model

    Olayiwola Arowolo, Oluwadara Adedeji, Peter Palensky, Jochen L. Cremer

    cs.LG

    Foundation models have shown strong generalization across scientific domains, but their use in power systems remains largely task-specific. We investigate whether a pretrained tabular foundation model (TFM), TabPFN, can serve as a general-purpose neural solver for power grid steady-state analysis. We evaluate the TFM on three tasks: power flow (PF) prediction, N-k contingency screening (NCS), and voltage stability margin (VSM) prediction. We...

    arxiv.org/abs/2610.20462 · PDF

  18. 18

    The Accuracy Well: Nonmonotone Accuracy Dynamics at Vanishing Initialization

    Yaroslav Gusev, Eugene Golikov, Dmitry Yarotsky

    cs.LG

    We study classification dynamics under small initialization and identify a pronounced non-monotone behavior of accuracy that we call the accuracy well: accuracy rises rapidly early in training, subsequently drops, and only much later recovers. We observe this phenomenon in deep linear networks, fully connected ReLU networks on MNIST, deep convolutional networks on CIFAR-10, and controlled synthetic tasks, across different losses and...

    arxiv.org/abs/2610.20439 · PDF

  19. 19

    Continual Learning for Reliable In-Sequence Molecular Communication

    Siddhant Setia, Junichi Suzuki, Tadashi Nakano

    cs.LG

    This paper considers reliable in-sequence molecular communication, where bio-nanomachines exchange molecule-encoded information in a specified order, and investigates a new simulation-based method to incrementally evaluate communication protocols by gradually increasing the complexity of experimental settings. The proposed method, called Affine Memory Replay (AMR), builds a continual learning (CL) model to estimate communication performance...

    arxiv.org/abs/2610.20426 · PDF

  20. 20

    OrionMoE-v1.0: A Mixture-of-Experts Foundation Model for Tabular In-Context Learning

    Mohamed Bouadi, Nassim Bouarour, Aditya Tanna, Vinay Kumar Sankarapu

    cs.LG · cs.AI

    Tabular foundation models enable in-context learning by predicting unseen observations from a labeled support table without task-specific parameter updates. Despite recent advances, most existing approaches rely on dense architectures that apply the same transformations across features and datasets. We introduce OrionMoE-Small, a compact tabular foundation model that combines distribution-aware representations with dataset-conditioned sparse...

    arxiv.org/abs/2610.20418 · PDF

  21. 21

    Tracking a Small Set of Experts in Online Convex Optimization

    Angelos Assos, Anand Brahmbhatt, Elad Hazan

    cs.LG · math.OC · stat.ML

    For the fundamental problem of prediction from expert advice, Freund asked whether one could compete with a comparator that switches $k$ times among only $m$ experts. For a discrete set of experts, this problem was resolved by Bousquet and Warmuth (2020), who gave an efficient algorithm with optimal regret matching the information-theoretic limit. We study the analogous problem in online convex optimization, where the action space is...

    arxiv.org/abs/2610.20416 · PDF

  22. 22

    One Size Does Not Fit All: Curvature-Aware Adaptive Perturbation for SAM

    Alessia Rossi, Felipe Campelo, Gabriel L. Oliveira

    cs.LG · cs.AI

    Poor generalisation in deep neural networks is frequently associated with convergence to sharp minima. Sharpness-Aware Minimisation (SAM) improves generalisation of deep neural networks by minimising a combination of the training loss and the local sharpness of the loss landscape, biasing convergence towards flat minima. However, SAM relies on a fixed perturbation radius $ρ$ that ignores local curvature, leading to inefficient behaviour: the...

    arxiv.org/abs/2610.20415 · PDF

  23. 23

    When Does Momentum Help in Reinforcement Learning with Verifiable Rewards?

    Kaifei Wang, Zikang Shan, Liwei Wang, Yinyu Ye, Han Zhong

    cs.LG · cs.AI · math.OC · stat.ML

    Momentum is a classical tool for accelerating convex optimization and a standard component of modern large language model training. Yet recent empirical work has questioned the usefulness of first-moment momentum in reinforcement learning with verifiable rewards (RLVR). We study this question through a bandit formulation of RLVR with bias-corrected exponential moving average (EMA) momentum, where the EMA coefficient $β$ controls how quickly...

    arxiv.org/abs/2610.20413 · PDF

  24. 24

    Hyperspherical Contrastive Representations Unlock Planning and Exploration

    Jonghae Park, Sungil Son, Jonghun Shin, Junseo Lee, H. Jin Kim, Benjamin Eysenbach

    cs.LG

    We develop a self-supervised reinforcement learning algorithm that learns to build towers of up to eight cubes from scratch, without prior data, rewards, demonstrations, or pretrained models. Our method, Contrastive Planning via Spherical Linear Interpolation (C-Slerp), uses contrastive representations as reusable building blocks and repeatedly composes them to infer a sequence of intermediate waypoints. Under our hyperspherical...

    arxiv.org/abs/2610.20409 · PDF

  25. 25

    Training Language Models to Fail Gracefully

    Daman Arora, Xuecheng Liu, Himanshu Gaurav Singh, Fahim Tajwar, Abitha Thankaraj, Ruslan Salakhutdinov, Haiwen Feng,...

    cs.LG

    Reinforcement learning makes language models better at solving tasks, but the same mechanism also rewards guessing. When success is the only way to earn reward, opting for a safe fallback is always worse than a long-shot attempt. When a wrong action can corrupt a database or ship broken code, that incentive can be costly. Therefore, we propose training models to fail gracefully: to preserve successful behavior whenever possible while learning...

    arxiv.org/abs/2610.20401 · PDF

  26. 26

    From Human Corrections to Learned Segmentations: A Critical Mathematical Review of Information, Learning, and Human Effort

    Jon Sporring

    cs.LG

    When does a human correction become useful learning, and what establishes better segmentation at matched human effort? This critical narrative review follows feedback through its selection, learner update, and evaluation. It compares active learning, machine teaching, interactive segmentation, and sequential predictive evaluation using a common mathematical account of target information, predictive error, response predictability, and human...

    arxiv.org/abs/2610.20396 · PDF

  27. 27

    ReAl: Refine neural operators via aligning dynamical flow

    Jing Xu, Zenglin Xu, Li Yi, Ruoxi Jiang, Yuan Qi

    cs.LG

    Neural partial differential equation (PDE) surrogates trained with mean squared error (MSE) can capture dominant dynamics but often lose high-frequency information and fine-scale structures. Generative models (e.g., Flow Matching) can capture the distribution of fine-scale details, yet stochastic errors may accumulate during autoregressive rollouts and compromise long-term stability. To address these limitations, we introduce Refine and...

    arxiv.org/abs/2610.20369 · PDF

  28. 28

    Fractional is Better: Learnable Derivative Orders in Neural Operator Learning

    Fares B. Mehouachi, Saif Eddin Jabari

    cs.LG · stat.ML

    Neural operators learn mappings between function spaces, enabling fast surrogate solutions to partial differential equations. Despite remarkable architectural diversity, these methods often share a common input representation: raw coordinate-value pairs $(x, u(x))$. We ask whether inputs aligned with PDE differential structure can improve learning. Through Picard iteration on mild solutions, we show that derivatives of the input appear...

    arxiv.org/abs/2610.20363 · PDF

  29. 29

    Beyond Expert Rankings: Contrast Calibration for Mixed-Precision MoE Quantization

    Renjie Xie, Juncheng Yang, Aoting Hu, Yan Liang, Mingxi Zhang, Zheheng Hong, Wei Xu, Yanzhi Chen

    cs.LG · stat.ML

    Mixture-of-experts (MoE) language models activate only a few experts per token, yet all expert weights must be stored. Mixed-precision quantization reduces this storage cost by assigning higher bit-widths to sensitive experts and lower bit-widths to resilient ones, but expert rankings alone do not determine how strongly to weight their errors. To address this gap, we propose NatScale, a contrast-calibrated allocation framework for...

    arxiv.org/abs/2610.20353 · PDF

  30. 30

    NeuroDETR: Multiscale Discrete Spatiotemporal Tokenization for Asynchronous EEG Event Detection

    Yaozhi Wen, Kun Zhao, Wei Wei, Haoyang Chen, Kangning Wang, Yi Sun, Shaowei Gu, Dingkun Liu, Shuang Qiu, Huiguang He

    cs.LG

    Asynchronous EEG event detection supports practical brain-computer interfaces by identifying event classes and temporal boundaries directly from continuous recordings without predefined event boundaries at inference time. The primary challenge lies in extracting event-related patterns embedded in ongoing background activity. To address this, we propose \textbf{NeuroDETR}, a discrete spatiotemporal multiscale framework that combines structured...

    arxiv.org/abs/2610.20349 · PDF

  31. 31

    Non-asymptotic Convergence of Average-reward Q-learning with Options

    Kintan Saha, Ahana Deb, Anders Jonsson, Debabrota Basu

    cs.LG · cs.AI

    We study the finite-sample convergence of Q-learning algorithms for the average-reward formulation of Reinforcement Learning (RL) with options. While options allow temporarily extended and hierarchical actions, the average-reward formulation incorporates continual tasks beyond discounted and episodic RL. We establish the first $\tilde{O}(1/\text{iterations})$ convergence guarantees, in the mean-square sense, for average-reward option...

    arxiv.org/abs/2610.20347 · PDF

  32. 32

    When Does Task-Aware Imputation Matter? An Analysis of Robustness Under Train-Test Mismatch

    Maria V. Salvador, Ana F. Sequeira, Jaime S. Cardoso

    cs.LG

    Missing data is a pervasive challenge in machine learning, often degrading predictive performance and complicating model deployment. Most existing approaches treat imputation as a preprocessing step independent of the downstream task, a separation that becomes problematic under train-test mismatches in missingness mechanisms or rates. We introduce a unified framework that treats missing entries as variables in a task-conditioned optimization...

    arxiv.org/abs/2610.20342 · PDF

  33. 33

    Training Dynamics of Transformers on Unbounded Inputs: Attention Learning Can Lag Output Layer Learning

    Yaomengxi Han, Debarghya Ghoshdastidar

    cs.LG

    The success of transformers is attributed to several factors, including their capacity to express formal languages over variable-length inputs, and ability of the attention mechanism to focus on relevant input features. However, there is limited understanding of the training dynamics of shallow transformers, specifically how to decouple the learning of the attention and the output (feedforward) layer. A natural extension of the theory of...

    arxiv.org/abs/2610.20337 · PDF

  34. 34

    Are We Overthinking One-Step Generation?

    Woobin Park, Jaeyeon Kim, Yilun Du, Sitan Chen, Bohyung Han

    cs.LG

    Diffusion and flow matching are now dominant frameworks for generative modeling in continuous domains, but they typically require multiple network evaluations at inference. This has motivated extensive research on one-step generative modeling primarily built on diffusion and flow, though existing approaches often rely on complex training machinery. In this work, we introduce Priming Models, a simple two-stage framework for training one-step...

    arxiv.org/abs/2610.20331 · PDF

  35. 35

    Comparative Evaluation of Deep-Learning based DoA Estimators for Real-Time ISAC Beamforming

    Anjali K. S., Manoj Kumar Panda, Chandramani Singh

    cs.LG

    Integrated Sensing and Communication (ISAC) systems rely on accurate Direction-of-Arrival (DoA) estimation for effective beamforming and spectral-efficiency enhancement. While numerous classical and deep-learning-based DoA estimation methods have been reported in the literature, their suitability for real-time ISAC operation remains insufficiently understood because practical deployment requires consideration of both estimation accuracy and...

    arxiv.org/abs/2610.20326 · PDF

  36. 36

    SpectrumQuant: Spectrum Alignment for LLM Quantization

    Dwaipayan Haldar, Indranil Patra, Azhar Yousuf, Chandra Sekhar Seelamantula

    cs.LG · stat.ML

    Orthogonal transforms have emerged as an effective tool for low-bit quantization of large language models (LLMs), as they reshape activation distributions without changing the underlying model function. Existing approaches typically rely either on fixed data-agnostic transforms, such as Hadamard rotations, or on calibration-time optimization of learned rotations. We study rotation selection from first principles and show that maximizing...

    arxiv.org/abs/2610.20303 · PDF

  37. 37

    Predicting Unseen Scientific Interactions: Conditional Flow Models Hallucinate Non-Additive Effects

    Sam Gijsen, Moritz Seiler, Kerstin Ritter

    cs.LG · q-bio.QM

    Conditional generative models are increasingly used for scientific prediction beyond the direct training support. Yet scientific questions often concern whether one intervention changes the effect of another. Such interactions can be of primary interest while explaining little of the total variation, allowing models to appear accurate overall without recovering the relevant scientific effect. It remains unclear how these models learn...

    arxiv.org/abs/2610.20288 · PDF

  38. 38

    From Plato to Aristotle: Toward a General Theory of Representation Hypotheses

    Dai Shi, Andi Han, Xiaoyu Li, Luke Thompson, José Miguel Hernández-Lobato

    cs.LG · stat.ML

    The Platonic Representation Hypothesis (PRH) holds that increasingly capable models converge toward a common representation. However, recent studies have shown different views on aligning representations via their neighborhoods, spanning trees, and distances. We show that these views can differ because the alignment criteria behind them sit at different points of one lattice, and we formalize this explanation as the Lattice Representation...

    arxiv.org/abs/2610.20286 · PDF

  39. 39

    TANGO: RNA Topology and Geometry Co-Design

    Tianmeng Hu, Biao Luo, Ke Li

    cs.LG · cs.MA

    Designing RNA sequences that satisfy target tertiary-structure objectives is an important problem in RNA engineering, with broad applications in synthetic biology and therapeutics. Existing 3D RNA design methods have made substantial progress, but they largely treat the target fold as a geometry-only objective. This is incomplete for RNA, whose functional folds are specified not only by tertiary geometry but also by secondary base-pairing...

    arxiv.org/abs/2610.20277 · PDF

  40. 40

    Resolution Allocation with Learnable Resizing and Spectral Filter Parameterization

    Wei Wu, Eric C. Larson

    cs.LG · cs.CV

    Learnable downsampling and spectral-domain filter parameterizations have each made convolutional networks more flexible, but their interaction has not been systematically examined. We bridge this gap by examining convolutional kernels learned in the spatial and spectral domains, together with learned downsampling ratios under a cropped resizing implementation. We find that spectral and spatial kernels respond differently to learned resolution...

    arxiv.org/abs/2610.20274 · PDF

  41. 41

    FUSE: Fairness-aware Update Screening against Byzantine Exploitation in $q$-Fair Federated Learning

    Tiantong Sun, Jinlin Liu, Xingru Wei, Zequan Fang, Wei Chen, Qiang Yang, Xiaojin Zhang

    cs.LG · cs.AI

    Client-level fair federated learning, in the form of $q$-fair federated learning ($q$-FFL), agnostic federated learning, or tilted empirical risk minimization, has emerged as an attractive solution to equalize utility across heterogeneous clients. This paradigm grants high-loss clients greater relative influence, while a central server aggregates their loss-weighted updates into a single shared model. Yet, the Byzantine robustness of $q$-fair...

    arxiv.org/abs/2610.20264 · PDF

  42. 42

    Harness Policy Distillation: Aligning Agent Policies with Runtime Semantics

    Junze Zhu, Weihao Chen, Zhiyang Zhou, Zhen Wu, Xinyu Dai

    cs.LG · cs.AI

    Deploying a general-purpose model as an agent means running it inside a product-specific runtime harness that decides which actions are executable, how state advances, and when the loop may terminate. Ordinary trajectory supervision shows which action was taken but leaves the runtime semantics behind that action implicit. We introduce Harness Policy Distillation (HPD), which makes these runtime control semantics an explicit training target...

    arxiv.org/abs/2610.20261 · PDF

  43. 43

    Long-Horizon RL Needs Muon

    Kai Ruan, Jinghao Lin, Tao Chen, Ziqi Zhou, Qianshan Wei, Zihe Huang

    cs.LG · cs.AI

    Long-horizon agents must coordinate reasoning, tool use, and environment feedback over extended interactions. Learning from sparse outcome rewards requires weighting trajectories of different lengths and updating the policy while rollouts are still being collected. We examine a single-rollout asynchronous recipe combining Muon, token-averaged loss, truncated importance sampling, and auxiliary length and tool-error penalties. Single-rollout...

    arxiv.org/abs/2610.20256 · PDF

  44. 44

    Adversarial Reinforcement Learning for Robust Planetary Landing

    Miguel Leiva-Vélez, Nicolas Gaston-Rozado, Andrea Scorsoglio, Hodei Urrutxua, Víctor Rodríguez-Fernández

    cs.LG · eess.SY

    Deep reinforcement learning (DRL) policies for planetary landing and other aerospace guidance, navigation, and control tasks are commonly trained under randomized disturbances to improve robustness. However, typical stochastic perturbation sampling assumes independent variations, whereas real mission disturbances often exhibit spatial and temporal correlation and cannot be reproduced through independent random draws. This work investigates...

    arxiv.org/abs/2610.20252 · PDF

  45. 45

    Retrieval Over Training: Similarity search-based Model Selection for Time Series Anomaly Detection

    Christos Panourgias, Roberto Stanzione, Adrien Petralia, Themis Palpanas, Paul Boniol

    cs.LG · cs.AI

    Anomaly detection is a fundamental task for time-series analytics, with important implications for the downstream performance of many applications. Despite the large number of anomaly detection methods proposed in the literature, recent benchmark studies have shown that no single detector performs best across highly heterogeneous time series. Therefore, a practical and scalable solution is to develop a model-selection method that, for a given...

    arxiv.org/abs/2610.20243 · PDF

  46. 46

    MAEM: Mitigating Recency Bias via a Replay-Free, Capacity-Controlled Dynamic Mixture of Experts for Class-Incremental Graph Learning

    Junlin Chen, Wenhao Wang, Junchao Xu, Qingsong Yao, Fei Xue, Li Sun, Xiao Wang, Ruijie Wang

    cs.LG · stat.ML

    Graph Neural Networks (GNNs) excel on graph-structured data but suffer catastrophic forgetting when classes arrive sequentially. In class-incremental graph learning, predictions are often dominated by recently learned classes, causing severe recency bias. Dynamic expansion can isolate knowledge across sessions, but many methods assume task-incremental settings and require task identities at test time to select branches or heads---information...

    arxiv.org/abs/2610.20238 · PDF

  47. 47

    HGRP: Heterogeneous Graph Neural Network for Routability Prediction

    Yule Sheng, Yue Yuan, Shan Shen, Daying Sun

    cs.LG

    Routability prediction enables rapid evaluation of placement quality without routing, providing effective feedback for routability-driven placement optimization. Recent machine learning methods have achieved efficient routability prediction through data-driven approach. However, their prediction performance highly depends on the capability of input representations to capture essential circuit information. To address the limitations of...

    arxiv.org/abs/2610.20209 · PDF

  48. 48

    Dimension-Optimal Dynamic Regret in Exp-Concave Online Learning

    Yanyi Li, Wendao Wu, Haihan Zhang, Qinshi Wang, Chenheng Zhang, Chunyuan Zheng, Cong Fang, Haoxuan Li, Zhouchen Lin

    cs.LG · math.OC

    Tracking changing comparators is a central problem in online learning. For exp-concave losses, previous work established the $T^{1/3}P_T^{2/3}$ dependence on the horizon $T$ and Euclidean path length $P_T$ for fixed dimension, while the optimal cost of dimension remained unresolved. We resolve this question by establishing a sharp $d^{1/3}$ curved dimension factor, improving the $d^{2/3}$ factor of recent switching and interval reductions [1,...

    arxiv.org/abs/2610.20187 · PDF

  49. 49

    RM-OPD: On-Policy Distillation for Generative Reward Models

    Qinglin Su, Licheng Pan, Zhen Fang, Jiakang Wang, Qingnan Ren, Yunsheng Lu, Yan Gao, Zhixuan Chu

    cs.LG · cs.CL

    Reinforcement Learning on Mixed-domain preference data (Mix-RL) has become the mainstream approach for training general-purpose Generative Reward Models (GRMs). However, Mix-RL suffers from two critical bottlenecks: the reward sparsity induced by outcome rewards, and the cross-domain interference arising from jointly optimizing heterogeneous capabilities, which together give rise to imbalanced capability acquisition and pervasive reward...

    arxiv.org/abs/2610.20174 · PDF

  50. 50

    Optimizing Sparse Boolean Rule Networks using Mixed Integer Linear Programming

    Thomas Pammer, Johannes Fürnkranz, Markus Sinnl

    cs.LG · stat.ML

    We study the problem of learning accurate and cognitively simple binary classifiers that are directly interpretable as Boolean rules. We propose mixed integer linear programming (MILP) formulations for training sparse disjunctive normal form (DNF) networks, viewed as small networks with a conjunctive hidden layer and a disjunctive output layer. Contrary to previous work that uses MILP for inductive rule learning, we start with a fixed network...

    arxiv.org/abs/2610.20170 · PDF

  51. 51

    In-Context Batch Black-box Optimization via Flow Matching

    Quoc-Anh Hoang Nguyen, Xinyu Zhang, Daolang Huang, Julien Martinelli, Samuel Kaski

    cs.LG

    Batch black-box optimization is important for expensive systems where many candidates can be evaluated in parallel, yet only a few rounds of evaluation are affordable. In this regime, effective batch construction must identify multiple promising regions while avoiding redundant evaluations, and conventional Bayesian optimization can require increasingly difficult batch-selection procedures as the batch size grows. We introduce Flow Matching...

    arxiv.org/abs/2610.20138 · PDF

  52. 52

    Seeing at the Edge: Energy Characterization of Vision-Language Model Inference on Edge Platforms

    Junlin Liao, Harshit Kondle, Di Liu, Zonghua Gu

    cs.LG

    Vision-language models (VLMs) are moving onto edge devices, but little is known about how much energy they draw on single-board computers with NPU or GPU accelerators, or where that energy goes. We measure total-system energy, latency, and thermal behavior of Qwen2-VL-2B-Instruct and its text-only sibling on a Raspberry Pi 5 with a Hailo-10H NPU and on a Jetson Orin Nano Super, on both CPU and accelerator, and separate the fixed energy of...

    arxiv.org/abs/2610.20136 · PDF

  53. 53

    A Bregman Proximal Viewpoint on Neural Operators

    Abdel-Rahim Mezidi, Jordan Patracone, Saverio Salzo, Amaury Habrard, Massimiliano Pontil, Remi Emonet, Marc Sebban

    cs.LG

    We present several advances on neural operators by viewing the action of operator layers as the minimizers of Bregman regularized optimization problems over Banach function spaces. The proposed framework allows interpreting the activation operators as Bregman proximity operators from dual to primal space. This novel viewpoint is general enough to recover classical neural operators as well as a new variant, coined Bregman neural operators,...

    arxiv.org/abs/2610.20120 · PDF

  54. 54

    KANLab: TensorFlow Playground for Kolmogorov-Arnold Networks

    Lukas Niehaus, Robin Rawiel, Ulf Krumnack

    cs.LG

    KANLab is a browser-based visualization tool designed to interactively teach the core concepts of Kolmogorov-Arnold Networks (KAN), forked from the widely successful TensorFlow Playground. To maximize educational clarity, KANLab omits residual connections and learnable scaling parameters found in modern KAN implementations, ensuring that displayed B-spline activation curves directly correspond to their underlying trainable control points....

    arxiv.org/abs/2610.20117 · PDF

  55. 55

    From Spectral to Frobenius: When Muon Loses Its Spectral Implicit Bias

    Yongtao Wu, Mahdi Mansouri, Zhenyu Zhu

    cs.LG · cs.AI

    Prior work on implicit bias shows that gradient descent selects the Frobenius max-margin direction on linearly separable problems, whereas Fan et al. (2025) established spectral max-margin behavior for spectral-norm steepest descent methods, including idealized Muon with exact orthogonalization. In practice, however, Muon approximates orthogonalization using finite Newton-Schulz iterations and a positive stabilization constant. We prove that...

    arxiv.org/abs/2610.20112 · PDF

  56. 56

    Exhaustive Canonical Skeletons Generation: A Formal Dataset for Decomposable Symbolic Regression

    Giorgio Morales, Anikó Ekárt

    cs.LG

    Neural symbolic regression (SR) models require diverse datasets of mathematical expressions for robust training. However, existing datasets rely predominantly on stochastic sampling methods, which suffer from high structural redundancy and incomplete functional coverage. To address this, we introduce an exhaustive dataset of univariate canonical skeletons generated via a deterministic constrained context-free grammar. A canonical skeleton...

    arxiv.org/abs/2610.20108 · PDF

  57. 57

    What Do Foundation Models for Market Microstructure Learn? A Controlled Study with Synthetic Limit Order Books

    Fernando Acero

    cs.LG

    Autoregressive transformers trained on limit order book (LOB) message streams are increasingly proposed as foundation models for market microstructure. Because the generative process of real markets is unknown, these models can be judged only by the realism of their output, leaving open what they actually learn. We make the question answerable by inverting the setup: we train GPT-style models on order flow from simulators whose latent...

    arxiv.org/abs/2610.20104 · PDF

  58. 58

    A Large-Dimensional Analysis of Least-Squares Federated Learning via Generalized Total Variation Minimization

    Ekkehard Schnoor, Malik Tiomoko, Alexander Jung

    cs.LG · stat.ML

    Federated learning over decentralized networks inherently trades off the suppression of local finite-sample noise through collaboration against the bias that collaboration introduces between heterogeneous nodes. We study this tension by performing a large-dimensional analysis of Generalized Total Variation Minimization with least-squares models. Using tools from random matrix theory, we show that Lipschitz observations of the global GTVMin...

    arxiv.org/abs/2610.20075 · PDF

  59. 59

    Rethinking Prediction Horizons: Decoupling Model Queries from State Updates in Few-Step MeanFlow

    Tinglan Wang, Haixu Liu, Shaobo Chen, Nuo Qian, Qi Gao, Kai Wen, Yu Yao

    cs.LG · cs.AI

    MeanFlow predicts interval-average velocities, but conventional few-step sampling ties the interval queried from the model to the duration of the state update. We separate these two quantities at inference, fixing the model, update grid, and number of queries while varying the prediction horizon. Our central finding is that the horizon response depends on both query position and the states entering that query. Across seven independently...

    arxiv.org/abs/2610.20071 · PDF

  60. 60

    Trainable at Depth, Solvable at Small Size: What the Loss Surface of a Polynomial Activation Dictates for Designing, Initializing and Training Networks

    Xiuyi Yang

    cs.LG · cs.AI

    What makes a deep network train, and what shape is its loss surface? We answer both with one construction at two scales: a polynomial activation f (z) = z + λz 3 , clamped and projected onto the unit sphere, plus an unnormalized identity path. As a deep stack it trains to one hundred layers on MNIST with no normalization machinery; as a small model its square loss is a polynomial system we solve exactly. The small scale explains the large...

    arxiv.org/abs/2610.20069 · PDF

  61. 61

    GBTSVR: Twin Support Vector Regression via Granular Ball Computing

    A. Quadir, M. Tanveer

    cs.LG

    Regression models used in real-world applications are often challenged by noise, outliers, data heterogeneity, and asymmetric error characteristics, which severely limit the effectiveness of conventional point-wise learning models. While twin support vector regression (TSVR) partially addresses these issues through the use of nonparallel regression bounds, it still relies on individual samples as basic learning units. As a result, TSVR...

    arxiv.org/abs/2610.20053 · PDF

  62. 62

    Sequential Learning-Based Physics-Informed Neural Network Modeling for Parameter Estimation on a Power Converter

    Samuela Rokocakau, Panagiotis Kakosimos, Giulia Tresca, Pericle Zanchetta

    cs.LG · eess.SY

    A staged physics-informed neural network (PINN) is proposed for identifying the inductance, capacitance and parasitic resistances of a DC buck converter from time-domain data. Gradients are released in the order of L to C to rL to rC, a sequence derived from sensitivity analysis. The five-stage schedule recovers L and C to within 5% and the series ESR to 10%, while a three-stage variant attains the same accuracy 11% faster. An 84 factorial...

    arxiv.org/abs/2610.20041 · PDF

  63. 63

    On the Convergence of Semi-Gradient TD: A Mean Field Perspective

    Haruki Settai, Tadashi Kozuno, Shinji Ito

    cs.LG · cs.AI

    Semi-gradient temporal-difference learning is a core algorithmic component of reinforcement learning, but bootstrapping with function approximation generally induces a non-conservative vector field and can lead to instability. We analyze this effect for two-layer neural networks in the mean-field regime and identify a simple stability mechanism in which entropy-induced relaxation dominates the sensitivity of the moving bootstrap target. Under...

    arxiv.org/abs/2610.20014 · PDF

  64. 64

    ChemArgus: Automated Scientific Diagnostics for Lab-Grade Multimodal Chemistry Reasoning: A Frontier Benchmark

    Hanyao Guo, Mo Cui, Zihan Tan, Leixin Sun, Yichun Wang, Yinan Shen, Guojun Zhu, Jia Jiang, Erhao Chen, Zui Chen,...

    cs.LG

    Chemistry evaluation has moved past multiple-choice probes toward open-ended expert problems, yet the benchmarks that define the current frontier still score answer endpoints alone. Such scoring conflates memorized facts with stepwise derivation, treats visual and textual inputs as interchangeable, and confounds long-horizon planning with stepwise execution. These blind spots prevent any diagnosis of where the reasoning chain breaks in a...

    arxiv.org/abs/2610.20012 · PDF

  65. 65

    Outperforming 1000 Euclidean Layers With Hyperbolic Goal-Conditioned RL

    Timo Klein, Thomas Lang, Andrii Shkabrii, Lukas Miklautz, Yllka Velaj, Sebastian Tschiatschek

    cs.LG · stat.ML

    Scaling self-supervised contrastive reinforcement learning (CRL) to 1000-layer Euclidean neural networks has enabled state-of-the-art goal-reaching performance, yet wall-clock time scales linearly with depth. We find that changing the geometry achieves stronger performance with far shallower networks. This matters in complex navigation environments, where walls can make route distance deviate substantially from Euclidean distance. In a...

    arxiv.org/abs/2610.20009 · PDF

  66. 66

    Provable Transformer Context Compression for In-Context Reinforcement Learning

    Yajie Zhao, Yingjun Mo, Zhiyong Wang, Bo Tang, Chao Xu, Fang Kong

    cs.LG · cs.AI

    Transformers can perform in-context reinforcement learning without parameter updates. However, existing theoretical constructions often require context sizes that grow with the number of sampled transitions, leading to increasing memory costs in large-scale problems. This raises a fundamental scalability question: how much context is intrinsically necessary to carry out such learning? We answer this question for fixed-policy TD evaluation. We...

    arxiv.org/abs/2610.20004 · PDF

  67. 67

    Empirical Boundaries of Reservoir Computing for Few-Shot Sequence Pattern Tasks

    Mario Mallea, Angela Nebot, Franscisco Mugica

    cs.LG · cs.AI

    Efficient sequence modeling remains a fundamental challenge in machine learning. Despite the success of Transformers, recurrent neural networks (RNNs) have recently re-emerged as efficient alternatives. Particularly, Echo State Networks (ESNs) offer a compelling framework of randomized RNNs that have been extensively studied for time-series forecasting and dynamical systems. This paper presents an empirical analysis of distributional,...

    arxiv.org/abs/2610.19997 · PDF

  68. 68

    LatentLoop: Scaling Looped MoE Models with Asymmetric Compute Allocation

    Zefeng He, Yunkai Chai, Xiaoyang Chen, Tong Zhu, Zongliang Wu, Yun Luo, Yafu Li, Yu Cheng

    cs.LG

    Looped Mixture-of-Experts (MoE) models combine parameter reuse with sparse computation to improve performance under matched budgets. Prior work demonstrates that different loop visits can play different roles even when they share parameters. To probe the contributions of different loop visits, we learn separate residual coefficients in a model that applies a shared block twice, observing a smaller scale in the first visit. Fixed coefficient...

    arxiv.org/abs/2610.19975 · PDF

  69. 69

    Safe-FCL: Continual Risk Awareness and Fine-Grained Safety Aggregation for Federated Continual Learning of LLMs

    Shuaijie Ma, Yongkang Hu, Yu Cheng, Jiuan Zhou, Yi Xie, Yushuo Zhang, Zhaoxia Yin

    cs.LG · cs.AI

    Federated fine-tuning enables large language models (LLMs) to learn collaboratively from distributed client data, but malicious client updates can compromise global model safety. As federated tasks evolve, changing malicious update patterns challenge the ability of existing defenses to maintain reliable risk detection over time. To address this challenge, we propose Safe-FCL, a framework integrating continual risk awareness with fine-grained...

    arxiv.org/abs/2610.19968 · PDF

  70. 70

    Learning Graph Energies for Finite-Time Generation

    Amartya Banerjee, Xingyu Xu, Caroline Moosmüller, Harlin Lee

    cs.LG · cs.AI

    Energy-based models can represent distributions over discrete structures such as molecular graphs, but an energy that is accurate at equilibrium need not yield accurate samples when the sampler stops after a fixed number of steps. We introduce finite-horizon energy learning, which adapts the acceptance energy of a Metropolis--Hastings sampler to the distribution it returns after a prescribed number of steps, with initialization, proposals,...

    arxiv.org/abs/2610.19946 · PDF

  71. 71

    MorphingNet: Symbolic Regression Where the Network Is the Formula

    Jincheng Gao, Jiahe Yi, Shuo Sun

    cs.LG

    We introduce MorphingNet, a network for symbolic regression (SR) whose units change their function continuously. Instead of searching over expressions built from a fixed operator library, MorphingNet trains the form of each function as a parameter, so SR becomes one continuous optimization problem. Its units are either pole-free rational functions or complex exponential--logarithm functions, which move between exponentials, sinusoids, damped...

    arxiv.org/abs/2610.19942 · PDF

  72. 72

    MolHop: Scaffold-Hopping Molecular Design with Agentic Reinforcement Learning

    Yanyi Su, Chenyan Wang, Yulei Gong, Yeyun Chen, Xiaohui Yang, Jun Cheng

    cs.LG · cs.CL

    Molecular design is often formulated as an optimization problem over existing molecular spaces. However, molecular optimization and the discovery of structurally distinct replacement molecules represent two fundamentally different search problems. LLM-based molecular design agents typically perform the former, conducting local refinement within the neighborhood of existing structures to improve target properties through structural...

    arxiv.org/abs/2610.19920 · PDF

  73. 73

    Beyond DFT electronic band structures via dual-space orbital learning

    Minhyuk Kang, Beom Seok Kang, Nikolai Argatoff, Miguel Nouman, Danish Khan, William A. Goddard, Anima Anandkumar

    cs.LG · cond-mat.mtrl-sci

    Accurate electronic band structures are essential for predicting material properties, but high-fidelity calculations are computationally demanding. We introduce O-kRystal, a dual-space orbital-learning framework that uses a lower-cost Perdew-Burke-Ernzerhof (PBE) calculation as both an energy baseline and a source of structured orbital information. Combining PBE band energies with real- and reciprocal-space orbital matrices, O-kRystal...

    arxiv.org/abs/2610.19919 · PDF

  74. 74

    CARD: Closing the Teacher-Student Loop for Adversarial Robustness Distillation

    Zhi Li, Haowei Liu, Hongchen Yang, Xiaoxuan Wang, Song Gao, Zhenli He, Wei Zhou, Shaowen Yao

    cs.LG · cs.CV

    Adversarial robustness distillation typically treats a robust teacher as a fixed knowledge source and transfers robustness to a compact student in a one-way manner. This paradigm terminates teacher-student interaction once the student is distilled, thereby discarding potentially useful feedback from the distilled student. However, exploiting such feedback is nontrivial because indiscriminate weak-to-strong transfer may propagate student...

    arxiv.org/abs/2610.19911 · PDF

  75. 75

    When Non-Adversarial Noise Hurts Learning: Benchmarking Skill Self-Evolution in Agents

    Tao Yu, Zhongtian Luo, Yuchen Long, Yiheng Lu, Yuxuan Zhou, Yanpei Gong, Siyi Cao, Yuqing Wu, Haopeng Jin, Zhenghao...

    cs.LG

    Large language model agents increasingly improve by distilling experience into reusable skills, but errors in that experience can persist in learned rules and affect later tasks. Existing evaluations largely examine clean learning or perturbations encountered after skills are frozen, leaving robustness during self-evolution insufficiently understood. We introduce RSE-Bench, to our knowledge the first benchmark for evaluating skill...

    arxiv.org/abs/2610.19910 · PDF

  76. 76

    Universal Sample-Based Embeddings of Higher-Order Objects

    Radovan Haluška, Christopher Solinas, Nathan R. Sturtevant, Martin Schmid, Michael Bowling

    cs.LG · cs.AI

    Neural networks operate naturally on finite-dimensional vectors, yet many problems require reasoning about higher-order objects such as distributions, functions, and graphs. Such objects capture relationships within a ground set and rarely admit an explicit compact representation. Moreover, we typically want a compact representation not of a single higher-order object but of an entire family of such objects, including members unseen during...

    arxiv.org/abs/2610.19899 · PDF

  77. 77

    On the Limits of Representation-Based Test-Time Adaptation for Multimodal LLMs

    Mikołaj Piórczyński, Wojciech Masarczyk, Damian Sójka, Sebastian Cygert, Bartłomiej Twardowski

    cs.LG · cs.CV

    Vision-language models suffer significant accuracy drops under common corruptions. While test-time adaptation (TTA) successfully mitigates these shifts in CLIP-like contrastive models, its application to generative multimodal large language models (MLLMs) remains underexplored and uniquely challenging due to their sequential token generation. In this work, we demonstrate that representation-based TTA fails to recover performance in MLLMs. Our...

    arxiv.org/abs/2610.19892 · PDF

  78. 78

    Occupancy Modeling via Drifting: One-Step Generation Enables Direct Policy Extraction

    Robin Uhrich, Marco Bagatella, Núria Armengol Urpí, Pavel Kolev, Georg Martius

    cs.LG

    The growing interest in behavior foundation models, combined with recent advancements in generative modeling, has led to an appealing recipe for reinforcement learning from unlabeled data: pre-train a generative model to model occupancies, i.e., discounted state visitations, on reward-free trajectories, such that a policy can be easily extracted once a small reward-labeled dataset becomes available. We decompose this recipe along three axes:...

    arxiv.org/abs/2610.19883 · PDF

  79. 79

    Neural Wasserstein Splines: Representation-Invariant Second-Order Priors for Trajectory Inference

    Amartya Banerjee, Harlin Lee, Caroline Moosmüller

    cs.LG · cs.AI · stat.ML

    Population dynamics are often observed only as unpaired snapshots, which leave the evolution between measurements underdetermined. Second-order regularization can constrain it, but a penalty on the accelerations of sampled particles is a prior on joint particle paths, and different particle representations of the same marginal curve can have different acceleration costs. We therefore introduce Neural Wasserstein Splines (NWSP), a sample-based...

    arxiv.org/abs/2610.19878 · PDF

  80. 80

    SPARS: A Stability-Plasticity-Aware Region Selection Framework for Continual Learning

    Teif Aldaajani, Huda Hakami, Maha A. Thafar

    cs.LG

    Continual learning aims to acquire new knowledge while retaining previously learned knowledge. However, updating shared parameters when learning new tasks often overwrites existing knowledge, causing catastrophic forgetting. Memory replay methods mitigate catastrophic forgetting but rely on previous samples, raising privacy, memory, and scalability concerns. On the other hand, parameter isolation-based methods preserve learned knowledge but...

    arxiv.org/abs/2610.19876 · PDF

  81. 81

    Energy Consumption Analysis of Discrete Diffusion and Autoregressive Language Models

    Boris Ruf, Marcin Detyniecki

    cs.LG

    As Large Language Models (LLMs) continue to scale, their energy consumption during inference has become a critical concern. While Autoregressive (AR) models are the industry standard, Diffusion Language Models (DLLMs) have emerged as a promising alternative capable of parallel token generation. In this paper, we perform a comparative energy benchmark of diffusion models (LLaDA-8B, Dream-7B) against AR counterparts (Llama-3.1-8B, Qwen-2.5-7B)....

    arxiv.org/abs/2610.19855 · PDF

  82. 82

    Memory-Efficient Subspace Optimization with Residual Correction: From Momentum SGD to Muon

    Sifan Yang, Hongyi Tao, Dingzhi Yu, Haomin Bai, Wei Jiang, Silin Gao, Pi Bu, Lijun Zhang

    cs.LG · cs.AI

    Memory-efficient subspace optimization has emerged as a practical approach to reducing the memory cost of LLM training while preserving optimization performance. Previous algorithms typically rely on the leading singular subspace of stochastic gradients, yet the gradient noise may dominate the subspace in later training stages and can prevent convergence. To overcome this limitation, we adopt a residual correction framework, which maintains...

    arxiv.org/abs/2610.19844 · PDF

  83. 83

    Rent-Seeking Language Models: Reward Models Set the Prices, Frequent Audits Curb the Rent

    Zixiao Wang, Xingbo Du, Duzhen Zhang, Le Song

    cs.LG

    Reward model overoptimization in RLHF is usually described as reward hacking. We recast it as rent-seeking: the policy, a firm, earns reward from a regulator, the proxy reward model, that true quality does not back. The lens turns overoptimization into measurable quantities, namely rent, a capture index and deadweight loss, and into testable predictions from the economics of regulation. In a fixed, version-controlled testbed with three model...

    arxiv.org/abs/2610.19832 · PDF

  84. 84

    CTIA-ODE: Continuous-Time Inter-Variable Attention for Long-Term Time Series Forecasting

    Chengyi Pan, Huangqimei Zheng, Hao Xue

    cs.LG

    Although multivariate time series forecasting has broad application prospects, effectively incorporating exogenous variables remains challenging. Prevailing discrete-time architectures often impose rigid temporal alignment across heterogeneous variables, which may limit their ability to capture continuously evolving external effects. To address this limitation, we introduce CTIA-ODE, a continuous-time forecasting framework centered on...

    arxiv.org/abs/2610.19831 · PDF

  85. 85

    MAPLE: A Multi-Axis Parametric Knowledge Extraction Lens for LLM Evaluation

    Minsuh Joo, Youna Kim, Choonghyun Park, Soo Kyung Kim, Hyunsoo Cho

    cs.LG · cs.CL

    LLMs absorb vast knowledge from training data, yet their inherently probabilistic nature offers no guarantee that this knowledge will surface in their outputs. To bridge this gap, Knowledge Extraction Enhancement (KEE) techniques such as activation steering, decoding adjustments, and prompting modify inference, potentially reinforcing existing predictions or surfacing less accessible answers. Yet their evaluations centered on aggregate...

    arxiv.org/abs/2610.19827 · PDF

This edition is part of The Daily Abstract — cs.LG archive. Subscribe to receive these in your inbox each morning, automatically translated to Spanish, with reply-to-PDF: arxivdaily.ignorelist.com.

Colophon Set in Georgia, with system sans for interface chrome and a monospaced stack for code and paper identifiers. Sole accent: amber #D99C5E. Built and served on an always-free VM. The masthead is set 14% letterspaced because newspapers do that and it works.