cs.LG · 2026-10-13 · No. 142
Machine Learning, 2026-10-13.
81 new papers in cs.LG. Titles, authors,
abstracts. Links to arXiv. Want this in your inbox every morning? Subscribe →
01 — The papers
81 entries-
01
AI Models Can Provably Hide Arbitrary Capabilities
Andis Draguns, Sumeet Ramesh Motwani, Raymond Douglas, Christian Schroeder de Witt
cs.LG · cs.CR
Capability evaluations aim to surface dangerous model behaviors before deployment, but their reliability depends on the assumption that hidden capabilities can be elicited. We show this assumption does not hold under adversarial conditions by constructing backdoor attacks that embed encrypted neural networks within host model weights, decrypting and executing them only upon receiving a secret trigger. Using digital lockers, these hidden...
-
02
KL Before and After Mixing: A Round-Complexity Separation in Parallel Sampling
Tianyi Wu
cs.LG · cs.IT
A parallel sampler can choose at random which coordinates to generate together. For a target $p$ and the output law $q_U$ of a schedule $U$ drawn afresh for each output, we compare the schedule-averaged divergence $\mathbb{E}_U D_{\mathrm{KL}}(p\|q_U)$ with the output-law divergence $D_{\mathrm{KL}}(p\|\mathbb{E}_U q_U)$, in nats. Rounds commit independent draws from exact conditional marginals frozen at the round's start, each output's...
-
03
Designing Protein Structures from Natural Language
Joseph Lee, Yanjing Li, Bo Qiang, Gregory Lau, Allyson Ettinger, Noah A. Smith, Rory Donovan-Maiye, David Baker,...
cs.LG · q-bio.BM
A central challenge in de novo protein design is generating proteins with desired structural and functional properties. Existing design models typically require a dedicated input representation for each property, limiting the design objectives they support. We develop Prompteina, the first all-atom structure generation model that supports new and broader design objectives specified in natural language, analogous to text-to-image generation....
-
04
Revisiting Tabular Deep Regression: Objectives and the Limits of Global Early Stopping
Danil Rudenko, Georgii Bulgakov, Nikolay Kartashev, Ivan Rubachev, Daniil Krasylnikov, Artem Babenko
cs.LG · cs.AI
While deep learning models for tabular data have recently received significant research attention, the choice of objective functions for regression tasks remains an understudied degree of freedom. Several lines of research offer alternatives to the standard Mean Squared Error (MSE), but differences in datasets, backbones, and evaluation protocols make their practical benefits difficult to compare. In this work, we address this gap by...
-
05
Prediction Is Not a Sufficient Memory State
Robert Jenkinson Alvarez
cs.LG
A memory that reproduces every current prediction may still have forgotten how to learn from the next observation. We characterize this distinction for exact cumulative learning and attention. For smooth convex residual losses with unrestricted real features and labels, a continuous recurrent state of fixed dimension exists exactly when the loss is polynomial. With ordinary, unweighted observations, degree $D$ requires precisely...
-
06
Partially Informed Stochastic Optimization via Zeroth-Order Residual Tracking
El Mahdi Chayti, Alireza Mirrokni, Martin Jaggi
cs.LG
Many optimization problems provide direct access to only part of the gradient, while the remaining component must be inferred from function evaluations. This structure arises in performative prediction, performative reinforcement learning, and bilevel optimization with a black-box lower level. We introduce Partially Informed Stochastic Optimization (PISO), which combines the available gradient component with a running estimate of the...
-
07
Inference-Time Steering Across Scaffolds and Epitopes for Sample-Efficient De Novo Nanobody Design
Mason Minot, Simon Friedensohn, Cédric R. Weber
cs.LG
Generative molecular design typically relies on over-generation and post-hoc filtering. Inference-time steering methods like Feynman-Kac (FK) steering improve efficiency by biasing trajectories toward high-reward candidates, yet they optimize within a single fixed configuration. In de novo nanobody design, upfront commitment to conditioning choices like scaffold identity or target epitope creates a bottleneck. Since yield varies across these...
-
08
Returning to Development States: When Full Sandbox Snapshots Help Coding Agents
Daniel Thi Graviet, Tanmay Shah
cs.LG · cs.SE
Coding agents often make useful progress before they fail, but retry-from-scratch discards the machine state created along the way. This makes failure recovery an interesting search problem: if intermediate environment states preserve meaningful progress, restoring and branching from them could outperform repeatedly restarting from the initial state. We implement Go-Explore-style search over full Daytona snapshots and find that higher and...
-
09
Specialists Learn Faster: Knowledge Organization in Pre-training Shapes Knowledge Update Efficiency
Sheng Zhao, Yuhao Zhu
cs.LG
An ideal (language) model must continually update its knowledge. Existing work largely focuses on making these updates efficient after training. We investigate how the organization of time-varying knowledge during pre-training shapes the efficiency of subsequent updates. Models trained on chronologically ordered knowledge specialize in the most recently encountered knowledge; we call them specialists. In contrast, models trained on the same...
-
10
On the Design Space of Continuous Diffusion Language Models
Oscar Davis, Daan Roos, Alice Bizeul, Amitis Shidani, Victor Turrisi, Louis Béthune, Pierre Ablin, Marco Cuturi
cs.LG
Continuous diffusion language models (CDLMs) promise to bring the benefits of continuous flow modelling to language, including accelerated sampling and controllable generation. The field has grown rapidly, and the capabilities of CDLMs have begun to rival those of discrete diffusion. Yet, unlike for continuous modalities such as images or videos, our grasp of the levers that matter when training CDLMs remains limited. In this work, we focus...
-
11
Shared Canonical Component Removal Beyond Expert Views
Jose Roberto Tello Ayala, Akl C. Fahed, Roy Perlis, Finale Doshi-Velez
cs.LG
In clinical practice, experts often begin by converting raw inputs into a set of expert features before making a decision. While black-box models trained on raw input can outperform models trained on expert views, it remains unclear what predictive signal the raw model exploits beyond what the expert view captures. We introduce an audit framework to quantify this gap using Shared Canonical Component Removal (SCCR). Unlike standard...
-
12
TabMF: Learning Dataset Representations with Tabular Foundation Models
Rafiq Kamel, Nicolas Atienza, Aziz Fouché, Nicola Cancedda, Josif Grabocka
cs.LG
Representations of supervised datasets support higher-level machine-learning tasks such as dataset retrieval and distribution-shift monitoring. However, existing approaches typically rely either on hand-crafted meta-features or on dataset encoders trained specifically for representation learning. We introduce TabMF, a framework that repurposes pretrained tabular foundation models (TFMs) as encoders of supervised datasets. TabMF equips a...
-
13
Spanning Diffusion and Flow-based Posterior Sampling via Exact Gaussian Conditioning
Teo Deveney, Lea Bogensperger, Hok Shing Wong
cs.LG · cs.CV
Flow-matching and diffusion models are increasingly used as priors for linear inverse problems, however, most methods are bespoke to one class of trajectory prior. Plug-and-play methods have been proposed, however these implicitly decorrelate the guided trajectory, resulting in broad priors that can struggle when data is sparse. This work proposes $β$-Flow: a general plug-and-play method indexed by a correlation parameter $β$ that generalises...
-
14
QTestBench: A Test Framework for Evaluating LLM-Generated Quantum Code
Meriem Terki, Nouhaila Innan, Minghao Shao, Muhammad Kashif, Alberto Marchisio, Muhammad Shafique
cs.LG · cs.CL · quant-ph
Large language models (LLMs) have shown strong results in classical code generation and are increasingly used to write the unit tests that safeguard that code. Whether they can do the same for quantum software is less clear, because quantum programs are probabilistic, depend on numerical tolerances, and are judged by domain-specific checkers, so a well-formed test can still disagree with the benchmark's verdict. Existing quantum benchmarks...
-
15
FORTE: A Production Framework for Feature Evaluation and Deprecation in Industrial Recommender Systems
Vishnu Arun, Adam Obeng, Tianyou Wang, Zilin Ren, Mukuntha Narayanan Sundararaman, Han Wang, Shivin Thukral,...
cs.LG · cs.AI
Production recommender systems have feature-lifecycle asymmetry: adding new features must produce gains on some metric, while feature removal must not regress any metric. In practice, feature deprecation is hard to prioritize as a feature can fail to be removed if it interacts with other features, its importance changes daily, it cannot be evaluated before production models change or data drift occurs, it is in contention for A/B test space...
-
16
Randomized Momentum Transport with Cheap Predictors
El Mahdi Chayti, Martin Jaggi
cs.LG · math.OC
Momentum methods such as STORM and second-order momentum track the gradient by adding a transport correction to momentum, at the cost of an extra oracle call per iteration. We introduce randomized momentum transport, which evaluates the correction with probability $q$ and otherwise uses a cheap prediction, with importance weighting that preserves the conditional mean and adds $(q^{-1}-1)$ times the prediction loss to the variance. Under...
-
17
Your Discrete Graph Diffusion Model is Secretly Memorizing
Adarsh Jamadandi, Nicolas Keriven
cs.LG · cs.AI
Discrete graph diffusion models (GDMs) corrupt the graph structure by adding noise independently to nodes and edges under a Markovian noise model. In the unconditional setting without node features, an equivariant denoiser trained on a single graph is able to generate novel graphs and sample-level metrics such as edge overlap and WL kernel similarity find no evidence of memorization. In this work, we challenge this view and show for the first...
-
18
ReplaySSM: Cache SSM Inputs, Not State
Ze-Wei Liou, Harris Nover, Amey Naik, Andrii Skliar, Igor Shovkun, Ido Shahaf, Tri Dao
cs.LG · cs.AI
State-space models (SSMs) maintain a fixed-size recurrent state, but reading and writing it at every decoding step can bottleneck hybrid models. This eager state management causes large state I/O with low arithmetic intensity, leaves speculative decoding no cheap rollback, and feeds state quantization error into subsequent recurrent updates. We introduce ReplaySSM, which unrolls the recurrence to represent the state as a checkpoint and a...
-
19
DualQuant: Dual-scale Calibration-free Weight-and-Activation Quantization for LLMs
Mahsa Salmani, Sayeh Sharify, Hesham Mostafa
cs.LG · cs.AI
Low-bit inference on accelerators with native 4-bit support requires quantizing both weights and activations to realize the full throughput gains these formats provide. While post-training weight quantization has been extensively studied, activation quantization remains considerably more challenging and typically requires calibration data. In this work, we introduce DUALQUANT, a dual-scale, calibration-free post-training quantization...
-
20
When Edge Independence Fails: Joint Graph Diffusion with Latent Sociability Priors
Adarsh Jamadandi, Nicolas Keriven, Aline Roumy
cs.LG · cs.AI
Discrete graph diffusion models corrupt graph structure through independent edge updates with a Markovian noise model. Combined with the exchangeability requirement on the output distribution, the terminal prior is necessarily an Erdős--Rényi (E-R) random graph. This has a fundamental consequence for unconditional graph generation without node features: in the large graph limit, any GNN denoiser applied to an E-R graph approximately outputs...
-
21
Matched-Context Group Policy Optimization for LLM Agent Training
Chenxu Zhang, Yaorui Shi, Wenlong Zhang, Jianchao Tan, Zihan Wang, Yiming Cheng
cs.LG · cs.AI
Training long-horizon language model agents requires assigning credit to intermediate decisions from sparse terminal feedback. Within a finite group of sibling rollouts, useful action comparisons depend on their context, evidence for both alternatives, and how that evidence enters learning. We propose Matched-Context Group Policy Optimization (MCGPO), a procedure for constructing weighted action preferences from already collected...
-
22
Studying the Effects of Task Diversity Using Small Mathematical Worlds
Emily Xia Meng, Xuanyu Yang, Junaid Hasan, Michael Ruofan Zeng, Henry Kvinge
cs.LG
Basic mathematical tasks like modular addition have been a valuable tool for illuminating aspects of neural network behavior that are challenging to see in noisy, unverifiable, large-scale, real-world settings. These studies have typically focused on a single task. However, modern language models are exposed to a vast number of tasks despite being trained with a next-token prediction objective. In this work, we introduce two mathematical task...
-
23
SafeDiSTL: Decomposing Joint Constraints for Safe Multi-Agent Reinforcement Learning
Hongrui Zheng, Aditya Singh, Rahul Mangharam
cs.LG · cs.AI
Cooperative multi-agent tasks often impose constraints on a shared objective or collective safety. A team level violation cost function can measure this constraint outcome, but does not distinguish which agents' actions most strongly influence that violation. We present SafeDiSTL, a constrained multi-agent reinforcement learning framework that allows for credit assignment even with joint constraint measurements. We use Signal Temporal Logic...
-
24
Orthros: Phase-Aware Heterogeneous Attention for Efficient Transformers
Yuhong Chou, Zehao Liu, Yuqi Pan, Qian Liu, Xianwei Chen, Zhongzhu Zhou, Jibin Wu
cs.LG · cs.AI
Efficient attention mechanisms, such as linear and sliding window attention, aim to reduce the computational complexity of Transformers from quadratic to linear. However, their degraded in-context recall often necessitates interleaving full attention layers, leaving the quadratic bottleneck unresolved in long-context processing. To address this, we introduce Orthros attention, which shifts the efficiency paradigm from structural allocation to...
-
25
SeoulMMOD: A Large-Scale Multimodal Origin-Destination Flow Benchmark
Taeyoung Yu, Seonbin Jo, Jiwon Kim, Junyoung Byun
cs.LG
Origin-destination (OD) flow forecasting supports urban mobility decisions by predicting the volume and patterns of human movement across a city. Most existing public OD benchmarks provide only a partial view of citywide mobility, covering a single transport service or at most two modes. With partial mode coverage, observed demand changes are difficult to separate into shifts across transport modes and changes in total travel. We introduce...
-
26
Amortized Counterfactual Prediction with Graph Neural Networks
Hendrik Stumpf, Francesco Quinzan
cs.LG · cs.AI
Counterfactual inference asks how an observed outcome might have differed under different circumstances, such as whether a patient would have recovered had they received a different treatment. Such questions arise whenever decisions depend on the outcomes of alternatives that were not observed. Answering counterfactual queries generally requires knowledge of the underlying causal structure, which is often unavailable in practice. A common...
-
27
Machine Unlearning for Federated Socially Fair k-Means Clustering
Aravind Reddy, Deeraj S K, Mohd Wasif Raza, Saurav Prakash
cs.LG · cs.DS
Federated clustering can support applications such as patient stratification across hospitals and customer segmentation across distributed financial datasets. Responsible AI calls for fair representation of demographic groups and support for data deletion, motivated in part by rights such as the GDPR's right to erasure. These goals interact: deleting records can change the balance between groups and alter the clustering of the remaining...
-
28
Towards Understanding the Training of GFlowNets: A Theoretical Perspective
Rui Hu, Yu Chen, Zhuoran Li, Longbo Huang
cs.LG
Generative Flow Networks (GFlowNets) learn policies that sample discrete objects in proportion to a positive reward. They are trained by minimizing balance objectives, whose exact satisfaction guarantees correct sampling. However, a comprehensive theory linking nonzero balance loss and approximate stationarity to sampling accuracy, especially under evolving sampling distributions and backward policies, is still lacking. To bridge this gap, we...
-
29
Calibration in One Go: Jointly Learning Accurate and Calibrated Neural Networks
Nektarios Kalampalikis, Pablo M. Olmos, Isabel Valera
cs.LG · cs.AI · stat.ML
Neural networks (NNs), while highly flexible and versatile, are often poorly calibrated. Post-hoc calibration methods adjust the outputs of an already trained model using a separate calibration dataset and additional calibration parameters. This process adds overhead that requires separate optimization of the calibrator and subsequent model selection. In this paper, we show that standard (post-hoc) calibrators can be trained end-to-end with...
-
30
Variance-Reduced Policy Gradient is Nearly Minimax-Optimal
Leonardo Cesani, Matteo Papini, Marcello Restelli
cs.LG · cs.AI
Standard Policy Gradient (PG) algorithms suffer from high estimation variance because they rely on Monte Carlo updates. While off-policy variance reduction mitigates this problem, establishing optimal sample complexity currently requires either restrictive assumptions, such as uniformly bounded importance weights, or impractical second-order operations. In this work, we introduce Loopless Variance-Reduced PG (LVR-PG), a unified theoretical...
-
31
Transformer-based In-Context Score Operator Learning for Diffusion Models
Wuzhe Xu, Zhaiming Shen, Yulong Lu, Wenjing Liao, Rongjie Lai
cs.LG
Diffusion models typically learn the generative dynamics of a single target distribution or a family of distributions indexed by conditioning variables fixed during training. This limits their ability to adapt to a previously unseen distribution from only a small set of examples. We introduce an in-context solution operator learning framework for diffusion models, in which each target distribution is treated as a task drawn from a...
-
32
VQIL: Visible Quantum Interference Learning for Perturbation Response Ranking on Graphs
Seokjun Lee, Sergio A. Ortega, Daniel K. Park, Namshik Han
cs.LG
Learning how local perturbations affect a network is important for identifying which components drive or absorb a system's response. We formulate this as seed-conditioned response ranking: given a perturbation at one seed node, ranking all other nodes by response strength. This is challenging when the model is given only the interaction topology, while the mechanism governing how perturbations propagate must be learned from data. We present...
-
33
On the Encoding of Calendar Features in Covariate-Informed Time Series Foundation Models
Alexander Kreusel, Maximilian Beichter, Alexandra Nikoltchovska, Benjamin Schäfer, Ralf Mikut, Frederik Zahn, Veit Hagenmeyer
cs.LG
Time series foundation models (TSFMs) increasingly offer covariate support, with calendar information being the most common type of categorical covariate in practice, as it is available for every point in time. However, how these features are encoded is largely treated as an implementation detail rather than as an object of study, despite encoding being one of the few design choices left for pretrained models at inference time. With the...
-
34
Pretraining and Test-Time Training Budgets Matter for ARC: A Controlled Study of VARC and TRM
Adrien Deliege, Marc Van Droogenbroeck
cs.LG · cs.AI
The Abstraction and Reasoning Corpus (ARC) is increasingly used to evaluate whether neural networks can infer task-specific transformations from few demonstrations. Recent ARC models differ in architecture, but also in broader training and evaluation protocols, including pretraining data, data augmentation, test-time setup, and optimization budgets, making conclusions about architecture benefits debatable. In this work, we study how the...
-
35
TabReliBench: Evaluating Tabular Foundation Models Beyond Predictive Accuracy
Tianqi Zhao, Jia Guo, Yan Shuo Tan, Qiong Zhang
cs.LG
Tabular foundation models (TFMs) are increasingly popular for their strong predictive performance. Many also provide posterior predictive distributions (PPDs), whose reliability remains less studied. We assess reliability through point prediction, distributional quality, marginal and subgroup calibration, and interval efficiency. We then introduce TabReliBench, a large classification and regression benchmark for these comparisons and...
-
36
SelectArena: A Modern Benchmark for Feature Selection on Real-world Tabular Data
Bastian Schäfer, Dominika Matus, Lennart Purucker, Ricardo Knauer, Erik Rodner, Frank Hutter, Anton Sokolov, Arnaud Liefooghe
cs.LG · cs.AI
Selecting a subset of features (columns) from a tabular dataset is a ubiquitous challenge in real-world applications. Industry experts seek to minimize model training costs, climate scientists require an interpretable subset of factors to understand extreme weather events, and clinicians need to choose among diagnostic tests due to financial and time constraints. When looking at the scientific literature to determine which feature selection...
-
37
PANDA: Prior-guided Attentional Dual-path Architecture
Rishabh Jain, Federico Belotti, Stefano Coniglio, Pietro Lio, Stefano Fiorini
cs.LG
Hypergraph Neural Networks (HNNs), designed to model higher-order relations beyond pairwise interactions, typically rely either on fixed structural weights or on normalised attention. Fixed weights can propagate misleading information in heterophilic settings, while normalisation makes attention coefficients competitive, in the sense that, in a message-passing framework, increasing the weight of one sender must reduce the weight of others in...
-
38
Private and Distributionally Robust Federated Multiclass Classification
Andrea P. Hernandez, Michael Ibrahim, Nagi Gebraeel
cs.LG · math.OC
Fault diagnosis in asset-intensive industries requires multiclass models trained on noisy, sensitive data collected across geographically distributed client sites. Federated learning enables collaborative training without centralizing data, but most approaches do not provide formal privacy guarantees for individual records stored at client sites and can degrade under label noise. To address these limitations, we develop a differentially...
-
39
A small cognitive core explains human behavior at scale
Marcel Binz, Eric Schulz
cs.LG
Psychology has proposed many principles of cognition. Each, however, was developed in isolation, leaving their combined explanatory power unknown. We assembled a small set of established principles into one model, which we call COntextual Retrieval and Encoding (CORE). CORE consists of a hierarchy of episodic memories into which prediction errors are written and whose encoding, retrieval, and forgetting depend on context. Fitted jointly to...
-
40
EchoTab: Learning Measurement-Grounded Representations for Echocardiographic Diagnosis through Tabular Reconstruction
Michelle Espranita Liman, Ioana Ionac, Alexander Müller, Eimo Martens, Daniel Rueckert, Philip Müller
cs.LG
Diagnostic information in multi-view medical imaging studies, like echocardiography, is distributed across views, each capturing different aspects of cardiac structure and function. In clinical practice, these views are first interpreted through quantitative measurements, which then support specific diagnoses. However, existing echo representation learning methods do not explicitly model measurements as an intermediate representation between...
-
41
$L_0$ Feature Selection via Monte Carlo Tree Search in High-Dimensional Financial Regression
Cameron Mouangue, Tristan Cazenave
cs.LG · q-fin.CP · q-fin.ST
Feature selection is a central problem in high-dimensional financial regression, where large numbers of engineered variables and strong correlations among predictors reduce model stability and interpretability. $L_1$ regularization (Lasso) is widely used for sparse regression, but its shrinkage can be restrictive when informative signals are spread across correlated variables. Exact sparse regression avoids this shrinkage but requires solving...
-
42
Simultaneous Competitive and Regret Guarantees for TCP Acknowledgment
Tsubasa Harada, Yasushi Kawase, Hanna Sumita
cs.LG · cs.DS
We study simultaneous competitive and regret guarantees for TCP acknowledgment. Each acknowledgment serves all pending packets, and the objective is to minimize the total acknowledgment and waiting costs. It is well known that no deterministic online algorithm achieves a competitive ratio better than $2$. Given $k$ randomized online experts and $0<\varepsilon<1$, we construct a randomized online algorithm whose realized total cost is always...
-
43
Rationally Linearized MLPs
Arijit Sehanobish, Byeongchan Kim, Avinava Dubey, Min-hwan Oh, Krzysztof Choromanski
cs.LG
We present Rationally Linearized Multilayer Perceptrons(RaLiMPs), a new approach to efficiently linearizing MLPs using low-degree rational approximations of standard activation functions. Inspired by polynomial-based MLP linearization, we show that rational approximations can provide accurate representations at substantially lower degree. RaLiMPs exploit these approximations to construct kernel representations with narrow bottlenecks,...
-
44
Probabilistic Ship Trajectory Forecasting using Transformer Models on AIS Data
Frederik E. T. Schöller
cs.LG · cs.RO
Anticipation of vessel trajectories is an essential task when navigating at sea. This applies to both human navigators and autonomous surface vessels. Trajectory forecasting enables sub-routines such as collision avoidance to assess the situation and react accordingly in time. This paper investigates Transformer and Performer models for trajectory prediction of surface vessels in confined waters. Using widely available position data from the...
-
45
Simulating Training Spikes of Deep Neural Networks by Layer-wise Random Feature Models
Ran Tian, Courtney Paquette
cs.LG
Intermittent ``spikes'' in loss, during the pre-training of Large Language Models (LLMs), remains a mysterious phenomenon both theoretically and empirically. In this paper, we show that simple quadratic models with heavy-tailed, power-law distributed inputs can replicate these spikes. We introduce a theoretical framework and simulation algorithm to model per-layer training dynamics of Deep Neural Networks (DNNs), using such quadratic models....
-
46
FedPCD: Priority-Constrained Federated Aggregation
Yousef Irshaid, Dara Varam, Mohamed I. AlHajri
cs.LG
Federated learning (FL) seeks to optimize a shared global model from decentralized client data. While effective for average performance, this process can produce unfair performance across clients, particularly under statistically heterogeneous data distributions. Existing fairness-aware and multi-objective FL methods address this through client re-weighting, gradient correction, or common-descent formulations, but often change the global...
-
47
Beyond Temporal Proximity: Explanatory World Models for Credit Assignment in Reinforcement Learning
Oriol Corcoll, Qiyao Wei, Ciarán Mark Gilligan-Lee
cs.LG · cs.AI · cs.IR
Generative Recommenders increasingly use Reinforcement Learning (RL) to optimize outcomes that may appear hundreds or thousands of interactions after a decision. Standard RL propagates reward according to temporal proximity rather than explicit relevance, attenuating the signal assigned to distant but important actions and creating a source of sample inefficiency. We introduce Explanatory World Models (EWM), an attribution framework that uses...
-
48
PPG-Bench: A Unified Benchmark for PPG Foundation Models
Arvasu Kulkarni, Mahir Jain, Aditya Ray Mishra, Sandeep Singh, Siddharth Panwar
cs.LG
Photoplethysmography (PPG) is increasingly recorded by wearable and clinical devices at enormous scale, and a growing number of PPG foundation models (FMs) have been developed to compute reusable representations for various downstream tasks. These models are difficult to compare: different evaluations use different datasets, task definitions, preprocessing, downstream heads, hyperparameter tuning, and baselines. We present PPG-Bench, a...
-
49
In-Context Learning for System Identification: Probabilistic Formulation, Long-Sequence Handling, and Benchmark Validation
Matteo Rufolo, Dario Piga, Filippo Pura, Marco Forgione
cs.LG · cs.AI · eess.SY
Traditional system identification typically involves two steps: initially, training data is processed to estimate a model of the system of interest; then, the model is used to predict outputs for new input queries. In contrast, recent in-context learning approaches learn a meta model that maps directly from training data and query inputs to outputs in a zero-shot fashion, bypassing explicit model estimation. This approach leverages a meta...
-
50
Accelerating Linear Bandits with Persistent Candidate Sets
Bingkui Tong, Daria Korovaitceva, Marina Sheshukova, Sergey Samsonov, Junpei Komiyama
cs.LG
Exhaustive optimistic scoring can make action selection costly in linear bandits with a large fixed set of $K$ actions, requiring $\mathcal{O}(K)$ score evaluations per round. We introduce Persistent Candidate LinUCB (\textsc{PC-LinUCB}), which maintains $k$ active actions and samples $q$ challengers uniformly from the remaining $K-k$ actions at each round. By scoring only these $k+q$ candidates, \textsc{PC-LinUCB} reduces the number of score...
-
51
Why Linear Models Can Still Work in a Complex and Changing World
Fangzhou Lin, Dengzhe Hou, Yinxing Li, Kazunori D Yamada
cs.LG
Linear models can predict well in complex domains even when underlying mechanisms are nonlinear. We argue that this can occur because the complexity of the world exceeds the complexity that can be learned reliably from finite, partially observed data. Flexible models can exploit curvature, thresholds, and interactions that fit one hospital, period, population, policy setting, or measurement process but fail elsewhere. When this instability...
-
52
PlacePulse: Learning Transferable Place Representations from Structural Geographic Data
Lucía García-Duarte, Miguel Álvarez, Diego Vicente, Javier Pérez Trufero, Javier de la Torre
cs.LG
Geospatial machine learning often depends on task-specific feature engineering over large collections of sociodemographic, business, and environmental variables. Geospatial foundation models offer reusable representations of place, but leading approaches typically encode dynamic signals, such as mobility and search activity, or visual signals from satellite imagery. Few models represent the static structural composition of places (who lives...
-
53
BLOKS: Scaling Kernel Solvers with Sketched Block Preconditioning
Giordano Vitale, Giacomo Meanti, Marco Rando, Blanca Ayuso de Dios, Lorenzo Rosasco
cs.LG · stat.ML
Kernel Ridge Regression (KRR) is a theoretically sound algorithm for nonparametric learning, which is also often used to investigate properties of deep learning. However, the exact KRR formulation incurs time and memory costs preventing applicability to large datasets, motivating the development of approximate solvers. These algorithms rely on (i) iterative methods, to avoid the computational burden of direct solutions; (ii) sketching or...
-
54
Opening the Black Box of Classifier-Free Guidance via Information Bottleneck
Jiayang Gao, Tianyi Zheng, Jiayang Zou, Fengxiang Yang, Luyao Fan, Lv Tang, Bo Li, Jia Wang
cs.LG
Classifier-free guidance (CFG) is a standard technique for improving conditional generation in diffusion models, yet its activation schedule is usually selected heuristically. In practice, most existing CFG-based methods rely on a pre-defined activation policy throughout inference, while recent dynamic variants often depend on hand-designed schedules without a principled theoretical explanation of when conditional information is most useful....
-
55
Hidden Data Challenges in Clinical Machine Learning
Carolin Cissée, Dennis Fast, Raneen Younis, Alexander Löser, Wolfgang Nejdl, Zahra Ahmadi
cs.LG
Widely reused preprocessing pipelines for electronic health records can encode assumptions about when information became available, what clinical episode a label represents, and what missingness means. These assumptions are rarely audited systematically and may propagate across subsequent studies when preprocessing pipelines are reused or adapted. Using MIMIC-IV as a case study, we identify and quantify violations of three complementary forms...
-
56
Understanding Mamba-3 Through Oscillatory Dynamics
Shutian Liang, Yue Song
cs.LG
Mamba-3 introduces input-dependent rotations into recurrent state updates, but their contribution beyond state retention remains unclear. We interpret these ro- tations as underdamped second-order dynamics, providing a framework for in- vestigating the role of oscillation in sequence modeling. Controlled comparisons show that underdamped dynamics outperform non-oscillatory damping regimes on state-tracking tasks, while language-model...
-
57
AgentTune: A Unified Training Lifecycle for Tool-Using LLM Agents
R E Zera Marveen Lyngkhoi, Abhivansh Gupta, Vidushee Vats, Ram Mohan Rao Kadiyala, Vinay Kumar Sankarapu, Pratinav Seth
cs.LG
Tool-using agents increasingly require a common substrate for building, training, evaluating, distilling, and maintaining agentic systems, yet existing tooling typically addresses these stages in isolation. We present AgentTune, a source-available Python library that provides a unified framework for developing and optimizing tool-using agents. At its core, AgentTune introduces a two-tier EventLog representation that preserves agent...
-
58
PDE-PFN: Prior-Data Fitted Neural PDE Solver
Jaehyeon Park, Mingu Kang, Dongseok Lee, Woojin Cho, Kookjin Lee, Anthony Gruber, Youngjoon Hong, Noseong Park
cs.LG
Motivated by the success of large language models (LLMs) with broad generalizability and robustness to noisy or unreliable pre-training data, we explore in-context prediction of PDE solution fields using approximate pre-training data. In addition, inspired by the posterior predictive mean-based inference mechanism of the in-context learning in prior-data fitted networks (PFNs), we propose PDE-PFN, a prior-data fitted neural solver that...
-
59
DuoRate: Two-Rate Per-Expert Precision Allocation for Mixture-of-Experts Serving
Mengqi Guo, Zhe Wei, Yuan Yuan, Jiunn Bin Lim, Boyi Pan, Michael Bi Mi
cs.LG
Serving a Mixture-of-Experts (MoE) model pays for expert precision twice: the resident footprint pays for every expert the router can reach, while each batch-one decode step pays only for the experts it selects. These are two different rate functionals over the same per-expert widths, yet published per-expert MoE allocators constrain only the first. We call the difference the store-read gap; in the released BitsMoE checkpoint, 2.54 bits per...
-
60
SBNO : Schrödinger Bridge Neural Operator for the Forward and Inverse Problems under Missing Information
Jaehyeon Park, Inhyeok Jeong, Noseong Park
cs.LG
Partial differential equation (PDE) surrogate modeling often relies on either dense observations or access to governing equations/physics constraints. In practice, observations can be sparse and partial, and the underlying physics may be partially specified or unavailable. We introduce SBNO, an equation-free paired-data diffusion bridge matching framework that learns conditional operators for both forward and inverse PDE problems under...
-
61
Sustainable Cloud-Fog-Edge Systems for Smart Buildings: Algorithmic Adaptation under Memory Constraints and Dynamic Orchestration Strategies
Mamadou Sow, Christophe Cerin
cs.LG · cs.DC · math.NA
The escalating environmental footprint of the Information and Communication Technol- ogy (ICT) sectorwith global Data Center electric- ity usage potentially reaching 3% by 2030 and the imperative for real-time responsiveness in smart buildings demand a foundational shift towards Edge Computing. This article proposes a holis- tic architectural and algorithmic approach for cre- ating autonomous, resilient, and privacy-preserving learning...
-
62
Structure-Aware Numerical Stabilization for Half-Precision Proximal Policy Optimization
Linxuan Wu, Liang Chen
cs.LG
Local online training on resource-constrained platforms requires low computational and training-state overhead. We study the numerical stability and computational efficiency of half-precision proximal policy optimization (PPO) on commodity GPUs. Directly converting PPO training to FP16 is unreliable because numerical errors feed back through policy updates and subsequent data collection, while PPO repeatedly compares a current policy with a...
-
63
MORAD: Reinforcement Learning with Multi-Objective Rewards for RNA Inverse Design
Jixin Tang, Ji Guo, Jun Zhao
cs.LG · q-bio.BM
We introduce MORAD, an online reinforcement learning framework for backbone-conditioned RNA diffusion models that trains a single shared policy across diverse target backbones. Structure-conditioned RNA inverse design seeks nucleotide sequences compatible with a target three-dimensional backbone, yet existing reinforcement learning approaches either optimize a separate policy for each target backbone or rely on offline preference...
-
64
DAG Structure Learning with Node-Varying Heavy-Tailed Disturbances
Robin Francis, Sundeep Prabhakar Chepuri
cs.LG
DAG structures can be learned from observations by incorporating acyclicity via smooth constraints, such as DAGMA's log-determinant function. Typically, observations and the underlying DAG are related through linear structural equation models, with Gaussianity assumption on the disturbances at each node. Extending DAG learning methods that account for scale and tail behavior of disturbances that vary across nodes remains a challenge. We...
-
65
Recovering Irreducible Three-Body Interactions with Lagrangian Neural Networks
Sena Kalabalik, Sukru Caglar, Abdullah Umut Hamzaogullari, Arkadas Ozakin
cs.LG · physics.atom-ph · physics.comp-ph
Lagrangian Neural Networks (LNNs) learn mechanical systems from trajectory data by parameterizing a scalar Lagrangian and deriving the dynamics through the Euler-Lagrange equations, biasing the model toward motion that respects the variational structure of mechanics. The interactions studied so far have been pairwise-additive, leaving open whether LNNs can recover an irreducible many-body interaction from motion alone. We address this using...
-
66
MHR-Router: Multimodal Heterogeneous Retrieval-Enhanced Adaptive Routing for VLM Inference
Wenjuan Zhu, Zheming Yang, Yunqing Hu, Qi Guo
cs.LG
Vision-language models (VLMs) exhibit complementary capabilities, yet selecting the most suitable expert for a given image-question query remains challenging due to heterogeneous multimodal inputs and varying expert strengths. Existing routing approaches rely on text-based cues or static model profiles, failing to capture fine-grained visual evidence and expert-specific failure patterns. To address this, we propose MHR-Router, a multimodal...
-
67
CopRoute: Dependence-Aware Conformal Routing for Language Models
Siva Rajesh Kasa, Santhosh Kumar Kasa, Karan Gupta
cs.LG
Set-based language model routing aims to increase the coverage of the event where at least one model answers a query correctly while controlling the number of inference calls made. Existing risk-controlled routing policies bound the all-miss risk (the probability that every model in the selected set fails) by calibrating a single conformal threshold and applying it to a nested family of model sets. Because each model is scored in isolation,...
-
68
Data-Adaptive Mahalanobis Metric Learning for Cross-Head Attention in Transformers
Zhenke Duan, Xin Li, Jiqun Pan, Dong Xiaofei, Hanwen Ning, Xinyuan Song
cs.LG
In this paper, we propose Data-Adaptive Mahalanobis Cross-Head Attention (DAMCHA), a simple and effective attention mechanism that generalizes Multi-Head Attention (MHA) through the lens of metric learning. Standard MHA implicitly learns a static, head-wise Euclidean similarity corresponding to the diagonal blocks of a unified cross-head metric matrix, discarding all off-diagonal interactions across heads. DAMCHA addresses this limitation via...
-
69
Neural Surrogate Modeling of Ultrasonic Wave Propagation in Heterogeneous Media
Burla Nur Korkmaz, Petro Moilanen, Ari Salmi, Harsha Vardhan Tetali, Arto Klami
cs.LG
Neural surrogates provide computationally efficient auto-regressive modeling of wave propagation, enabling e.g. fast inverse solutions. Despite recent progress, the current surrogates are not yet sufficient for the needs of ultrasonic sensing, characterized by high frequency and heterogeneous propagation media. We show how methods building on Fourier Neural Operators (FNO) fail to capture essential signal features, such as reflections,...
-
70
CAST: Credit Alignment for Cross-Tokenizer On-Policy Distillation
Jingru Yang, Jinhang Wu, Minjing Shi, Shengfeng He, Xuyan Li, Man Tan, Biye Li
cs.LG · cs.CL
Cross-tokenizer on-policy distillation (OPD) requires aligning teacher and student sequences before their scores can be compared. Decoded-text alignment provides such correspondence at the chunk level, but leaves a distinct problem unresolved: credit alignment. A chunk-level signal specifies how much a span should change, but not how that signal should be distributed across student tokens, while tokens without teacher correspondence remain...
-
71
Tabular Foundation Models Are Effectively Shallow
Irene Cannistraci, Julia E. Vogt
cs.LG · cs.AI
In-context-learning Tabular Foundation Models (TFMs) are uniquely suited to scenarios with limited resources or privacy constraints, as they solve new tasks without additional training. However, their architecture consists of multiple transformer blocks, resulting in high computational and memory footprints. Meanwhile, traditional alternatives like Gradient-Boosted Decision Trees (GBDTs) remain highly competitive and run efficiently on CPUs,...
-
72
State-OPD: State-Grounded On-Policy Distillation for Multi-Turn Agents
Zhentao Zhang, Yanzhao Zheng, Yanlin Liu, Yuanqiang Yu, Tianze Xu, Chao Ma, Jihuai Zhu, Lelun Wu, Baohua Dong,...
cs.LG
On-Policy Distillation (OPD) has emerged as a powerful paradigm for transferring reasoning capabilities from frontier models to smaller models, while extending its success to multi-turn agentic environments remains challenging. We study agentic OPD through interaction states, distinguishing the objective task and executed action-observation record from the internal semantic trajectory of student reasoning. Through this lens, we identify two...
-
73
Learning Schwarzschild Geodesic Motion with Lagrangian Neural Networks
Sukru Caglar, Sena Kalabalik, Abdullah Umut Hamzaogullari, Arkadas Ozakin
cs.LG · gr-qc
Lagrangian Neural Networks (LNNs) learn dynamics from trajectory data by parameterizing a scalar Lagrangian and deriving accelerations through the Euler--Lagrange equation, providing a variational inductive bias that improves physical consistency over purely data-driven models. LNNs have recently been extended to general relativistic settings, leaving Schwarzschild geodesics as an open challenge. Training on complex relativistic systems is...
-
74
ReSFlow: Rectifying Categorical Flows on Statistical Manifolds for One-Step Generation
Xiaohuan Jia, Renzhe Xu, Xiao Wang, Jiayun Wu, Shaohua Fan
cs.LG · cs.AI
Generative modeling for discrete data has been significantly advanced by learning continuous vector fields on the statistical manifolds to transport simple priors to target discrete distributions. However, because these continuous-time flows are not constrained to follow geodesics on the statistical manifold, thereby requiring multi-step numerical integration for simulation and leading to substantial computational costs during inference. In...
-
75
Tensor-based Fusion of Multi-modal Data for Predicting Cancer Risk Levels
Ali Anaissi, Yuanzhe Jia, Qishun Peng, Weidong Huang, Ali Braytee, Junaid Akram
cs.LG · q-bio.QM
Cancer diagnosis and treatment increasingly rely on multi-modal diagnostic technologies that generate high-dimensional datasets. Although these datasets are rich in pathological information, they pose considerable challenges in interpretation, dimensionality reduction, and clinical applicability. Conventional machine learning approaches often fail to extract meaningful cross-modal features, thereby limiting predictive accuracy in patient risk...
-
76
Monarch-MoE: Structured Experts for Multimodal Continual Instruction Tuning
Xi Zhang, Boyu Diao, Zesen Chen, Ruiqi Liu, Hangda Liu, Yongjun Xu
cs.LG
Learning new tasks in multimodal continual instruction tuning can cause catastrophic forgetting of previously acquired knowledge. Although multi-expert LoRA distributes learning across experts, updates to a dense low-rank expert can still affect many output features. We introduce Monarch-MoE, which replaces the dense factors in LoRA experts with block-diagonal factors connected by a fixed permutation. This structure provides blockwise...
-
77
How Much Computation Does Every Forecast Need? Structured Block Skipping in Two Time-Series Foundation Models
Max Bourgeat
cs.LG
Time-series foundation models use the same network depth for every input, although forecasting windows differ in structure and difficulty. We test whether whole Transformer blocks can be removed at inference time in two pretrained models: Chronos-Bolt and TimesFM 2.5. Our bypasses avoid the original block computation and preserve the dense model exactly when all blocks are enabled. Masks are selected only on calibration windows and evaluated...
-
78
Rethinking Relative Policy Differences Transfer of On-Policy Distillation
Rong Wu, Daocheng Fu, Yu Yang, Xuemeng Yang, Siqi Li, Shu Zou, Yao Zhu, Botian Shi, Bo Zhang, Cong Zhang, Pinlong...
cs.LG
On-policy distillation (OPD) trains a target policy on its own rollouts using token-level supervision from a stronger teacher, but typically relies on the teacher being substantially more capable than the target. Recent work relaxes this requirement by transferring the relative policy difference between an expert and a reference policy rather than the expert's absolute distribution. Yet it remains unclear which differences provide useful...
-
79
S$^3$R: Fast and Faithful RL Rollouts via Self-Sparse Speculative Decoding
Yuhang He, Hongyu Ge, Zihan Lin, Siyi Liu, Yongqi Zhang
cs.LG · cs.CL
Fast rollout generation for long-context reinforcement learning with verifiable rewards (RLVR) requires reducing KV-cache traffic without changing the response groups used for policy optimization. In our profiled RLVR workloads, attention accounts for 72--76\% of autoregressive decoding kernel time. Sparse attention reduces these reads, but sampling directly from a sparse policy changes the response groups used for learning. For...
-
80
Does Learning Recurrent Networks Matter in Partially Observable RL?
Zekang Wang, Zhe He, Gaojie Jin
cs.LG · cs.AI
Reinforcement learning (RL) under partial observability typically re- lies on recurrent networks to encode temporal information, with the recurrent module trained jointly with the policy or value function. This raises a natural question: is learning the recurrent network actually necessary? In this paper, we investigate whether a frozen recurrent network can provide sufficient memory for solving par- tially observable Markov decision...
-
81
WaSA: Wavelet Scattering Autoencoders for Unsupervised Offline Detection of Malicious Clients in Federated Learning
Alessandro Licciardi
cs.LG
Federated Learning (FL) enables collaborative model training without exposing raw data, yet remains vulnerable to malicious clients injecting poisoned datasets. Existing defenses are typically reactive (operating during training) or supervised (requiring prior knowledge of attack types) - assumptions that rarely hold in open-ended IoT deployments. We propose WaSA, a fully unsupervised, offline framework for malicious client detection. Before...
This edition is part of The Daily Abstract — cs.LG archive. Subscribe to receive these in your inbox each morning, automatically translated to Spanish, with reply-to-PDF: arxivdaily.ignorelist.com.
#D99C5E. Built and served on an always-free VM. The masthead is set 14% letterspaced because newspapers do that and it works.