cs.LG · 2026-11-03 · No. 162
Machine Learning, 2026-11-03.
76 new papers in cs.LG. Titles, authors,
abstracts. Links to arXiv. Want this in your inbox every morning? Subscribe →
01 — The papers
76 entries-
01
Pretraining Data Scale Reverses the Muon-AdamW Fine-Tuning Gap
Catherine Li, Jingchu Gai, Andrej Risteski, Jacob Mitchell Springer, Aditi Raghunathan
cs.LG
Muon has recently emerged as a promising optimizer for pretraining large language models, offering a speedup over AdamW in minimizing pretraining loss. However, it remains unclear whether Muon's advantage persists after fine-tuning. In this paper, we study the downstream behavior of Muon and AdamW and uncover a surprising trend: beyond a sufficiently large pretraining data budget, AdamW outperforms Muon after fine-tuning. Moreover, as model...
-
02
Not All Reasoning Generalizes: A Case Study on Learning Generalizable Reasoning Strategies in RLVR
Ziqi Pang, Yu-Xiong Wang
cs.LG
Reinforcement learning with verifiable rewards (RLVR) can enable generalizable improvement for large language model (LLM) reasoning. This paper studies how to incentivize such generalization via reasoning strategies. We study this question with a controllable equation-solving framework: finding multiple inputs x for equations f(x)=y, where f is a non-bijective string function. This setting makes reasoning strategies observable while...
-
03
FADE: Factored Ability--Difficulty Filtering for Online RLVR Scheduling
Tomas Duncun, Jonah Ellison, Mateo Ruiz, Hanae Fujita, Ingrid Solberg
cs.LG
Group-based reinforcement learning with verifiable rewards (RLVR) learns only from prompts whose sampled completions disagree: all-pass and all-fail groups produce zero relative advantage and burn generation budget without moving the policy. Adaptive schedulers try to steer rollouts toward informative prompts, yet they rely on per-prompt pass-rate trackers that entangle two quantities evolving at very different speeds---the policy's shifting...
-
04
Thought-deciphering via hierarchical EEG-to-text alignment
Xiaopeng Mao, Morten Mørup, Xin Xu, Sang Wan Lee, Sadasivan Puthusserypady
cs.LG · eess.SP
Communication is important for our well-being, but it can be impaired by disabilities, such as locked-in syndrome (LIS), cerebral palsy (CP), and dysarthria. Non-invasive brain-to-text brain-computer interfaces (BCIs) offer a promising solution by providing an alternative communication pathway for people with communication disabilities. However, most recent breakthroughs have relied on large, cumbersome, and costly neuroimaging modalities,...
-
05
Benchmark Prediction with Sparse and Unpaired Cached Responses
Hayden Helm, Avijit Ghosh, Ben Johnson, Carey Priebe
cs.LG · stat.ME · stat.ML
Evaluating a generative model on all queries from all available benchmark tasks is prohibitively expensive. As such, models are often evaluated on different subsets of tasks and/or different subsets of queries from a given task. The resulting collection of responses and the corresponding (model, task) score matrix are sparse and unpaired. In this paper, we introduce the Product Kernel Perspective Space (PKPS), a collection of low-dimensional...
-
06
Efficient Closure Learning in Dynamical Systems: A Comparison of Data-Assimilated and Embedded Approaches
Rileigh Bandy, Teresa Portone, Mohammad Khalil, Eric Crislip, Kyle Neal
cs.LG · math.DS · stat.ML
Increased computing power has transformed computational science and improved predictive capabilities for complex systems, but many phenomena still exceed current simulation capabilities or require rapid evaluations for fast design iterations. Closure models address this challenge by approximating unresolved effects in reduced or incomplete dynamical systems. Data-driven closures offer a flexible mechanism for improving model accuracy and...
-
07
Should We Combine Our Data? Algorithms and Limits for Privately Assessing Data Sharing
Mahdi Haghifam, Kumar Kshitij Patel, Avrim Blum, Giulia Fanti, Juba Ziani
cs.LG · cs.DS · cs.IT
In this paper, we study when two parties should pool their data to train a machine learning model. Two agents, Alice and Bob, each hold a sensitive labeled dataset and want to train a classifier from a given hypothesis class $\mathcal{H}$. Even when each party's dataset can be perfectly classified on its own, the two parties may follow conflicting labeling standards or have selection biases that affect their features and label sets, such that...
-
08
DynaSolver: A Neural Operator for Autoregressive 4D Flow Prediction with Dynamic Boundaries
Rui Qian, Shibo Dai, Mengjun Lin, Yuk-Kit Hou, Saining Zhang, Bohan Li, Junhao Chen, Hao Zhao, Chaojian Li
cs.LG
Neural operators provide efficient surrogates for computational fluid dynamics (CFD), but most are designed for static geometries and are not equipped to model unsteady flows induced by continuously moving boundaries. We study dynamic-boundary 4D CFD prediction for articulated moving geometries and introduce RoboCFD-4D, a temporally resolved robotic CFD benchmark spanning diverse morphologies and flow conditions. To address the computational...
-
09
Evaluating Uncertainty Calibration in Probabilistic Time Series Foundation Models
Thomas Decker, Volker Tresp
cs.LG · cs.AI
Time series foundation models (TSFMs) are increasingly used for probabilistic forecasting across diverse domains, yet the reliability of their uncertainty estimates remains poorly understood. In this work, we study uncertainty calibration in probabilistic TSFMs from a broader methodological and empirical perspective. We introduce a taxonomy of how TSFMs represent predictive uncertainty and develop a fine-grained evaluation framework for...
-
10
A Scaling Theory of Multi-Epoch Pretraining: Late-Time Overfitting and Ensembling
Zhen Yang, Yidi Miao, Blake Bordelon
cs.LG
In the near future language model pretraining will become bound by total data rather than total compute. Such a regime requires revisiting scaling laws and scaling law theory. Motivated by prior works that have found utility in ensembling, we study how the number of training epochs and ensemble size jointly shape generalization when training data are fixed. In language-model experiments on FineWeb, logit averaging reduces late-training...
-
11
Sonar-MoE: In-Backward Expert Updates for Memory-Efficient Mixture-of-Experts Training
Avni Mittal, Avinash Anand, Ashutosh Kumar, Dikshant Kukreja, Kritarth Prasad, Chaitanya Lakhchaura, Rishitej Reddy...
cs.LG · cs.AI · cs.CL · cs.PF
Most parameters of a Mixture-of-Experts (MoE) model belong to its experts, and standard training keeps every expert gradient in GPU memory until the optimizer step. This memory grows with the number and size of the experts, so it rises as MoE models scale. We introduce Sonar-MoE, which updates each expert tensor during the backward pass and frees its gradient right away. It brings optimizer-in-backward from dense models to batched MoE expert...
-
12
BISECT: Bilevel Spectral Decomposition for Low-Rank Continual Learning
Tom Saliencro, Wen-Hsuan Liu, Rohan Desai, Amara Okonkwo, Maya Lindqvist
cs.LG
Exemplar-free continual learning on pre-trained Vision Transformers increasingly relies on Low-Rank Adaptation (LoRA), yet prevailing methods govern the low-rank update space through strict subspace isolation. By projecting each incoming task into the approximate null space of historical token activations, isolation-based learners equate any cross-task feature overlap with catastrophic interference. We show that this orthogonality premise...
-
13
Variational Functional Neural Networks (vfNN)
Chinmay Katke, C. Nadir Kaplan
cs.LG · cond-mat.soft
We present a variational functional neural network (vfNN) algorithm that optimizes physically motivated continuum functionals in integral form (e.g., Lagrangian, Hamiltonian, or free energy) to find the states of a system. That way, vfNN bypasses solving the equations of motion (a.k.a. the Euler-Lagrange equations), which are in the form of differential equations. To this end, our approach is fundamentally different from conventional...
-
14
Library of Layers (LoL): Adaptively Ordering Transformer Layers at Runtime
Abdellah Rahmani, Sean McLeish, Ang Li, Sanae Lotfi, Pascal Frossard, Micah Goldblum
cs.LG · cs.AI · cs.CL
Computer programmers chain together the same reusable computational primitives, for example, conditional statements or loops, but in different orders to implement different algorithms. In contrast, existing language model architectures always execute their layers in the same order. We introduce Library of Layers (LoL), an architecture that dynamically orders its layers during inference to solve the problem at hand. LoL has a single...
-
15
Ask, Don't Equip: Question-Driven Coding Agents for Deep Kernel Optimization
Gabriele Oliaro, Jaeseong Lee, Zhaoyuan Su, Sreeram Vennam, Zhihao Jia, Samyam Rajbhandari
cs.LG
LLM agents for GPU kernel optimization sit at two ends of a structure-freedom tradeoff. Scripted agents fix what the model does at each step, via a learned policy, an optimization menu, or fixed roles, and are sample-efficient where the recipe applies and stall where it does not. Open-ended agents let a self-directed model mutate a population of kernels and pay for the freedom in trials: thousands of evaluations, repeated edits, and wrong...
-
16
RASP: Read-All, Sparse-Plasticity Continual Learning in a Stationary-Size LoRA
Wen-Hsuan Liu, Amara Okonkwo, Tom Saliencro, Rohan Desai, Maya Lindqvist
cs.LG
Parameter-efficient continual learning on pre-trained Vision Transformers largely relies on a costly premise: preventing catastrophic forgetting requires either spawning a dedicated LoRA branch per session---causing adapter parameters to scale linearly with task horizon---or applying sparse Mixture-of-Experts gating symmetrically to both forward and backward passes, which sacrifices representational rank at test time. We argue that reading...
-
17
ST-GFM: Spectral-Temporal Graph Forecasting with Measured Perturbation Bounds and Masked Spectral Reconstruction
Aashish Bohra, Vivek Vijay
cs.LG · eess.SP · stat.ML
Graph forecasters for sensor networks are trained in one regime and deployed in others, where sensor coupling and signal frequency content both change. We split this spectral shift into an operator channel and a frequency channel, each with a dimensionless measure. ST-GFM (Spectral-Temporal Graph Forecasting Model) filters every input step with Chebyshev and bidirectional diffusion filters on one learned, non-symmetric operator and alternates...
-
18
A Hybrid Visitor Prediction Framework using Interpretable Temporal features, Integrating IQR and Anomaly Detection in Lakes in Nebraska
Israt Zarin, Christopher Chizinski
cs.LG
For effective resource management, infrastructure planning, and sustainable tourism, it is really important to predict accurate visitors. This paper estimates the daily visitation of Nebraska lakes using spatial, basic temporal, and interpretable temporal features. Moreover, we evaluate the base models with multiple metrics. Using the best three models, our method will ensemble hybrid models and choose the best hybrid technique. Further, we...
-
19
Tilted Convexity and an Exact Offline-Online Separation
Mohammad Pedramfar, Rajiv Khanna, Christopher John Quinn, Vaneet Aggarwal
cs.LG · cs.CC · math.OC · stat.ML
Convexity has three equivalent descriptions: support by affine functions, monotonicity of the subdifferential, and increasing secant slopes. We weaken all three by one distortion with parameter $γ\in(0,1]$, which multiplies positive changes by $γ$ and divides negative ones by $γ$. The resulting classes are the convex functions at $γ=1$ and, for $γ<1$, are distinct and nested, with the parameter dropping by at most a square along the chain....
-
20
Unrolled gradients in disguise: bridging interpolation-based and Jacobian regularization for stable neural dynamics
Maya Janvier, Etienne Meunier
cs.LG · math.NA
Hybrid models and neural differential equations (NDEs) are becoming increasingly important for modelling physical systems; however, they often encounter stability and accuracy issues during long-term integration. Training on unrolled trajectories is known to limit these divergences but quickly becomes too computationally expensive due to the need to compute gradients over an iterative process, making this method impractical for large-scale...
-
21
Online Class Posteriors for Interpretable and Controllable Discrete Diffusion
Felix Koulischer, Gianni Van De Velde, François Remy, Chris Develder, Thomas Demeester
cs.LG
Discrete diffusion models remain far less understood than their continuous counterparts, despite recent breakthroughs across modalities, most notably in text. Leveraging tools from continuous diffusion, we formulate a theoretical framework to track the class-conditional posterior in an online manner by combining conditional and unconditional model predictions. In masked language modeling, this posterior offers highly interpretable dynamics:...
-
22
Improving Hamiltonian Learning through Local Energy Decomposition
Yaojun Li, Yulong Yang, Christine Allen-Blanchette
cs.LG
Hamiltonian neural networks provide a principled framework for learning conservative dynamics. A scalar Hamiltonian is parameterized by a neural network and the vector field is derived using Hamilton's equations. However, preserving Hamiltonian structure does not guarantee that the energy function itself is represented accurately. When modeled by a conventional multilayer perceptron, the Hamiltonian inherits the spectral bias of...
-
23
Bridging Neural Tangent Kernel Analysis and Reinforcement Learning for Deep Function Approximation
William St-Arnaud, Margarida Carvalho, Golnoosh Farnadi
cs.LG
Despite their widespread success across machine learning, the learning dynamics and generalization properties of neural networks remain poorly understood. The neural tangent kernel (NTK) provides an analytical characterization of the learning dynamics of infinitely wide neural networks and serves as a useful proxy for finite-width networks in supervised learning. In this work, we extend the NTK framework to reinforcement learning (RL) by...
-
24
A Living Leaderboard for Graph Machine Learning
Martin Ritzert
cs.LG
Graph machine learning benchmarking results are scattered across papers that use quite different sets of datasets and baseline methods, making it difficult to tell whether a reported gain reflects architectural progress or merely an outdated comparison or weakly tuned baseline. We introduce a living, graph-learning-specific leaderboard that organizes reported results by dataset, putting new results into context. As of 15 October 2026 our...
-
25
Temporal Island Sparse Autoencoders for Interpreting Clinical Time-Series Models
Muhan Yeo, Bong Gyun Kang, HyunGi Kim, Sungroh Yoon
cs.LG
Deep learning models for electronic health records achieve strong predictive performance but provide little insight into what clinical concepts they represent internally. Post-hoc attribution methods produce per-cell importance scores that lack temporal coherence, cross-variable structure, and reusability across patients. We introduce the Temporal Island Sparse Autoencoder (TI-SAE), which learns a backbone-specific, patient-shared clinical...
-
26
Finite-Memory Control of POMDPs: Fundamental Limits and Efficient Design
Emrecan Kutay, Atilla Eryilmaz, Ness Shroff
cs.LG · cs.IT
Optimal control of partially observable Markov decision processes (POMDPs), in general, requires a complete history. The belief state, a distribution over hidden states, summarizes this history and provides the classical sufficient statistic to control POMDPs. However, the continuous nature of the belief space generally prevents a finite-memory controller from maintaining it exactly. This raises the question we address in this work: what must...
-
27
Low-Rank Clebsch--Gordan Coupling for Equivariant Spherical Forecasting
Yaojun Li, Zhixuan Zhao, Dongzhe Zheng, Christine Allen-Blanchette
cs.LG
Spherical-harmonic neural operators provide an efficient representation for global forecasting, but their discretized pointwise nonlinearities can break rotational equivariance through aliasing. Clebsch-Gordan (CG) tensor products provide exactly equivariant interactions between spherical-harmonic representations, but their computational cost grows rapidly with spectral bandwidth and has limited their use at forecasting scale. We introduce...
-
28
A Geometry-Informed Neural Operator surrogate of LISFLOOD-FP for rapid pluvial-fluvial flood forecasting: the case of Forlì, Italy
Samuele Maria Garofalo, Fabio Limardo, Simonetta Bodojra, Mohammad Amin Nabian, Mehdi Taghizadeh
cs.LG · physics.geo-ph
Physically based two-dimensional hydraulic models are too slow for real-time inundation forecasting and large ensembles. We present a time-dependent geometry-informed neural operator (GINO) surrogate of LISFLOOD-FP for the municipality of Forlì (Italy), severely affected by the May 2023 Emilia-Romagna floods. The surrogate is designed for an operational setting in which only river stage gauges and rain gauges are available. A reproducible...
-
29
RangeFinder: Predicting Ligand Property Ranges from Protein Pockets
Alan Nadelsticher Ruvalcaba, Chris Alvin, Supratik Mukhopadhyay, Adam Bess
cs.LG · q-bio.BM
Identifying ligands for a protein target requires searching a vast chemical space for a comparatively small set of compatible molecules. Existing machine learning approaches primarily address this problem by generating candidate ligands or scoring individual protein-ligand pairs, but provide limited information about the broader chemical space compatible with a binding pocket. We introduce RangeFinder, an SE(3)-equivariant transformer...
-
30
Characterizing When Language Models Take Harmful Actions to Protect Their Assigned Goals
Brandon Jaipersaud, Elliot Creager
cs.LG · cs.AI · cs.CL
Recent language model (LM) safety failures have largely been studied as disparate phenomena. In agentic misalignment, a model blackmails an executive to avoid being replaced. In alignment faking, it complies with harmful requests during training to keep its values from being modified. We argue that these behaviours share a common decision structure: a model assigned a goal must either take a harmful action to preserve it or refuse and lose...
-
31
Multidimensional Suicide Risk Assessment with Machine Learning using Electronic Health Records
Nathan Scollar, Kemal Davaslioglu, Nathan Conger, Sastry Kompella, Yalin E. Sagduyu, Nathaniel D. Bastian
cs.LG
Suicide prevention remains a critical public health challenge, and electronic health records (EHRs) provide an opportunity to characterize suicide-related risk indicators using documented clinical, behavioral, medical, and social-contextual information. This paper presents a multidimensional machine learning framework to model and estimate suicide-related outcomes using MIMIC-IV EHR data. We construct clinically motivated diagnosis categories...
-
32
Certifying Graph Neural Networks Against Label and Structure Poisoning
Lukas Gosch, Xichuan Chen, Yan Scholten, Stephan Günnemann
cs.LG
Robust machine learning for graph-structured data has made significant progress against test-time attacks, yet certified robustness to poisoning - where adversaries manipulate the training data - remains largely underexplored. For image data, state-of-the-art poisoning certificates rely on partitioning-and-aggregation schemes. However, we show that these methods fail when applied in the graph domain due to the inherent label and structure...
-
33
NormMoX: A Normative Mixture of Experts for Explainable Autism Classification from Structural and Functional MRI
Muhammad Hasan Masrur, Fani Deligianni
cs.LG
While structural and functional magnetic resonance imaging (sMRI, fMRI) offer profound potential for researching autism spectrum disorder (ASD) and understanding individual neurodivergence, current classifiers lack the explanatory power needed for scientific study, clinical use and personal-level insights. Most models learn from imaging features that also vary with age, sex, scanner site and head motion. They fuse the two modalities without a...
-
34
FLAMIA: FLAsh MIcroscaling Attention for Fast and Causally Safe Training on Blackwell
Fedor Velikonivtsev, Mikhail Koltakov, Denis Kuznedelev
cs.LG
Attention remains a substantial computational cost in Transformer training, particularly at long context lengths. NVIDIA Blackwell GPUs provide high-throughput low-precision matrix multiplication, but exploiting it in attention requires addressing quantization overhead, causal information leakage through shared scales, and numerical inconsistencies between the forward and backward passes. We present FLAMIA, a family of attention kernels for...
-
35
LLMs Depend on Specialist Tools for Out-of-Distribution Generalization in MS/MS Structural Elucidation
Gabriel Asher, Cailum M. K. Stienstra, Tomo Oga, Devesh Shah Antonio H. de O. Fonseca, Niall O'Connor, Michael Widrich
cs.LG · cs.AI · cs.CL · stat.ML
A frontier language model was recently reported to solve 1D NMR structural elucidation without tools. We ask whether this capability transfers to tandem mass spectrometry (MS/MS) in the de novo setting, where the model must propose a structure directly from the spectrum without a candidate list or reference library. To separate reasoning from recall, we build a 100-analyte benchmark from MassSpecGym, split by each analyte's textual footprint...
-
36
Calibrated Multiaccuracy: Deterministic and Randomized Sample Complexity
Jiuyao Lu, Georgy Noarov, Aaron Roth
cs.LG · cs.DS · math.ST
Calibrated multiaccuracy asks that a predictor be globally calibrated and multiaccurate with respect to a bounded test family $G$; that is, its residuals should be uncorrelated with every test $g(x)\in G$. This weakening of multicalibration suffices for many applications. Its sample complexity was not known to improve on that of full multicalibration, which is $\widetildeΘ\left(\frac{\log|G|}{\varepsilon^2} + \frac{1}{\varepsilon^{3}}\right)$...
-
37
An Axiomatic Framework for Reinforcement Learning in Open Multi-Agent Systems
Jianhong Wang, Yang Li, Samuel Kaski, Jonathan Lawry
cs.LG · cs.AI · cs.MA
Open multi-agent systems, where only a subset of agents are controllable, are central to real-world domains such as smart grids, yet lack a principled foundation for reinforcement learning algorithm design. We study this setting through n-agent ad hoc teamwork (NAHT), a recent framework for learning in open multi-agent environments. We propose an axiomatic framework that derives value function learning from the Shapley axioms (Efficiency,...
-
38
Inferring Transferable Rewards via Active Inverse Reinforcement Learning
Victor Villin, Till Freihaut, Andreas Schlaginhaufen, Christos Dimitrakakis, Maryam Kamgarpour, Giorgia Ramponi
cs.LG · cs.HC
Inverse Reinforcement Learning (IRL) aims to recover expert rewards that transfer across environments with differing dynamics. Prior work has shown that such transfer is possible when gathering expert demonstrations across sufficiently different dynamics. In practice, however, expert demonstrations are costly, making it critical to select the right dynamics to reduce the quantity of expert data required for learning transferable rewards. We...
-
39
ENGINE: Endogenous Variational MultiScale Optimization for Zeroth-Order LLM Fine-Tuning
Zhuoli Ouyang, Changxi Chi, Siyuan Li, Tailin Wu
cs.LG
Zeroth-order (ZO) optimization enables memory-efficient large language model (LLM) fine-tuning, and recent methods incorporate curvature-aware strategies. However, in the high-dimensional parameter space of LLMs, these methods inherently suffer from exploding gradient variance and prohibitive costs. Conversely, confining updates to a low-dimensional subspace reduces variance but discards critical high-dimensional information, degrading...
-
40
Strictly proper scoring rules for node selection tasks on graphs
Lorenz Hilfiker, Nathan Cincera Brack, Martin Sterchi
cs.LG · cs.SI · stat.ML
In node selection tasks, the goal is to identify a single unknown target node in a graph. We consider probabilistic supervised node selection, where the estimator outputs a probability distribution over the nodes of the graph and the ground-truth node is available for evaluation. Classical evaluation metrics for such tasks are either statistically ill-conceived or blind to the graph structure. To fill this gap, we introduce two strictly...
-
41
A Human-Verified AI Workflow for Annotating Environmental Sound Data
Michael Barasa Murunga, Allan Vikiru
cs.LG · cs.SD
Large-scale annotation of environmental sound recordings presents significant challenges under time and resource constraints, particularly when sound categories are imbalanced and only part of a dataset has been labeled. This paper describes the development and evaluation of a human-AI workflow used during annotation of the Kenya Forest Sounds Dataset (KFSD), using a project-stage working collection of 7,531 audio recordings across 21...
-
42
DeepWaveRL: Self-Supervised Full Waveform Inversion via Reinforcement Learning
Peng Jin, Yinan Feng, Yinpeng Chen, Youzuo Lin, Sharon Huang
cs.LG · physics.geo-ph
Full Waveform Inversion (FWI) is a fundamental inverse problem in geophysics to estimate subsurface properties, such as velocity, from seismic measurements. Recently, self-supervised learning methods remove the need for ground-truth velocity maps during training by leveraging a forward operator to simulate seismic data from predictions and backpropagate the gradients of reconstruction loss through it. In this work, we consider a more...
-
43
The Cost of Mismatch: Noise Amplification in Zeroth-Order Reinforcement Learning
Lianmin Chen, Junbin Qiu, Chenxing Wei, Yao Shu, Kun He
cs.LG
Zeroth-order (ZO) optimization has emerged as a promising approach for memory-efficient large language model (LLM) post-training, yet its reliable application to reinforcement learning with verifiable rewards (RLVR) remains largely unexplored. Motivated by the distinct data regimes of on-policy RL and fixed-data optimization, we develop a generic variance-theoretic framework to investigate what limits the reliability of ZO gradient...
-
44
Rethinking Fairness Metrics for Multi-Class and Multi-Group Assessment: Not Everything is Binary
Matteo Grazioso, Fabio Naccarella, Chiara Gallese, Marco S. Nobile
cs.LG · cs.AI
Artificial Intelligence, and Machine Learning in general, is becoming pervasive across all domains and disciplines. By means of this data-driven approach, algorithms can learns complex task just by learning from samples. However, if the data-set is biased, they might lead to the development of unfair computer systems, that is, characterised by a prejudice towards specific individuals or groups, which might get an unmotivated output from the...
-
45
A Common Language for Dynamic Graphs: Interaction Tokenization Enables Foundation Models
Fuyuan Ma, Yili Wang, Xin He, Xin Zhang, Ying Wang, Xin Wang
cs.LG · cs.SI
Dynamic graphs capture evolving interactions in Web systems, including online communication, user engagement, and recommendation. Graph foundation models aim to reuse predictive capabilities across networks, but extending this capability to dynamic graphs faces two challenges. First, node identities are graph-specific, and similar interaction patterns can appear at different temporal and activity scales, making histories difficult to...
-
46
Forecasting How RLHF Responds to Reward-Model Edits Before Training
Zixiao Wang, Duzhen Zhang, Xingbo Du, Dongxia Wu, Le Song
cs.LG
Responses with headers, lists and greater length receive high scores from reward models (RMs), and RLHF policies learn to produce these features even when true quality does not improve. Practitioners therefore edit their RMs, but they learn whether an edit works only after an RL run. We show that the response of a policy to an RM edit can be forecast before training, from samples of the current policy and the scores of the old and new RM. If...
-
47
What Structurally Breaks an Agent? Auditing Difficulty in Stateful Tool-Use Workflows
Shereen Anand, Ju Wang, Yali Du
cs.LG
An aggregate task score does not distinguish failures of scheduling, state retention, and candidate selection. We develop a small executable instrument that varies dependency depth, candidate count, active state width, and access to current-state snapshots while holding the number of required operations fixed. Two dependency-graph families show that increasing depth at fixed work changes both scheduling freedom and the number of reachable...
-
48
CERBO: Certified Recourse at a Requested Rate for Gradient-Boosted Trees
Farhan Shahriyar Hossain, Taufikur Rahman Fuad, Md. Zarif Bin Hasnat, Md Abrar Jahin, Md Rizwan Parvez
cs.LG · cs.AI · cs.CY
Recourse-aware learning trains models so that people who receive an adverse decision can reverse it with an affordable change. For gradient-boosted decision trees (GBDTs), the existing method, RABIT, exposes a single recourse weight whose effect on the recourse rate is abrupt and data-dependent, and its training bound cannot certify recourse for any rejected individual. We present Certified Recourse Boosting (CERBO), which takes a requested...
-
49
On the Convergence of Vanilla Adam under Heavy-Tailed Noise and Index Sampling
Kanata Oowada, Hideaki Iiduka
cs.LG
Understanding how stochastic gradient noise affects the convergence of Adam is an important problem in stochastic optimization, particularly when the noise exhibits heavy-tailed behavior. In this paper, we investigate the convergence of vanilla Adam in both the expected risk and the empirical risk minimization, focusing on the effects of heavy-tailed noise and its relationship to index sampling. For the expected risk minimization, we...
-
50
CISA: Retrofitting Dense Attention Models with Chunk-Indexed Sparse Attention
Mohammed Dabbah, Izhak Golan, Oren Tropp, Daniel Afrimi, Vladimir Anisimov, Tomer Asida, Nave Assaf, Ran El-Yaniv,...
cs.LG
Long-context LLM inference is costly because attention repeatedly reads a growing key-value (KV) history. We introduce Chunk-Indexed Sparse Attention (CISA), which retrofits pretrained dense attention models without gradient updates. CISA selects fixed-size KV chunks using compact summaries of existing keys, avoiding token-level indexing and a separate full-length index cache. It uses one indexing query per KV group and shares selected chunks...
-
51
ViAdminShift: A Vietnamese Benchmark for Event-Level Knowledge Updates Built from the 2025 Administrative Reform
Phan Binh Nguyen Lam, Nguyen Thai Anh
cs.LG · cs.CL
On July 1, 2025, Vietnam reduced its 63 provinces to 34, abolished its 696 districts, and replaced its 10,035 commune-level units (wards) with 3,321. ViAdminShift, a Vietnamese language resource about this event, consists of 7,995 benchmark items, 2,281 retrieval passages from pre- and post-reform Wikipedia revisions and dated templates, and old-to-new tables that a released script rebuilds from the official conversion table (not...
-
52
LaBraM at Scale: Scaling Data and Models for EEG Pretraining
Jun-Yu Pan, Ruicheng Yin, Bao-Liang Lu, Dongsheng Li, Wei-Long Zheng, Yansen Wang
cs.LG · cs.AI
LaBraM, an early EEG foundation model, demonstrated that large-scale EEG pretraining can yield strong representations for diverse downstream tasks. Building on this foundation, we expand pretraining data and model capacity while retaining its core encoder and objective. We train ten encoders with 5.8 million to 1.97 billion parameters on three EEG collections spanning 25,067 to 79,910 hours, adapting signal preparation and training to...
-
53
Dissecting Geometric Constraints in Grokking: Interaction-Dependent Acceleration in Modular Arithmetic
Zachary A. Hampton, Wasiq Khan, Bilal Khan
cs.LG · cs.CC
Spherical transformer configurations can accelerate grokking through simultaneous modifications of several model properties. The present study investigated the contributions of these modifications in the published Sphere+Fourier configuration using 960 controlled modular-arithmetic runs. A complete factorial analysis varied residual projection, readout weight normalization, logit scale, and weight decay. Across four matched settings with...
-
54
PaddleMaterials: An End-to-End Deep Learning Toolkit for Materials Science
Xuwei Liu, Zhimin Zhang, Xiaoguang Hu, Zhongwei Liu, Yuanhui Sun, Jun Huo, Hua Wu, Haifeng Wang
cs.LG · cond-mat.mtrl-sci · cs.CE
Machine learning supports materials design, atomistic simulation and experimental characterization, yet models and workflows remain fragmented. We present PaddleMaterials, an open-source framework integrating heterogeneous materials-learning tasks while preserving task-specific representations and scientific semantics. It unifies data processing, training, evaluation, pretrained-model management and workflows across six task classes: property...
-
55
From PID to Reinforcement Learning in Building HVAC Control: A Control-Architecture Review of Tuning Strategies, Formulations, and Deployment Trade-offs
Erfan Haghighat Damavandi, Davide Papurello, Mahdi Alibeigi, Armin Keshavarz
cs.LG
This paper reviews control architectures for building heating, ventilation and air-conditioning (HVAC) systems, addressing the tension among energy demand, occupant comfort and equipment operability, and situating proportional-integral-derivative (PID) regulation within a layered control stack that also includes rule-based and model-based supervisory control and data-driven reinforcement learning (RL). It examines the thermostat-PID layer...
-
56
FlowBatt: Flow Matching for Probabilistic Battery Degradation Prediction
Hamidreza Eivazi, Eslam Sharaawy, Stefan Wittek, André Hebenbrock, Raphael Ginster, Steffen Blömeke
cs.LG · physics.chem-ph
Battery degradation remains a central challenge in the development and broad application of sustainable energy technologies. Accurate degradation prediction is challenging, as battery aging emerges from complex and heterogeneous interactions among cycling behavior, operating conditions, and cell chemistry. Existing machine-learning approaches typically focus on deterministic end-of-life predictions or modeling degradation curves within fixed...
-
57
Controlling Sharpening in LLM Post-Training
Simon Matrenok, Nicolas Menet, Nicolas Flammarion, Andreas Krause
cs.LG
LLM post-training via reinforcement learning has shown consistent improvements in accuracy at the cost of reducing behavioral coverage - a phenomenon known as sharpening that can stall progress on difficult tasks. Repeated sampling objectives such as $\mathrm{pass@k}$ and $\mathrm{max@k}$ are known to mitigate sharpening, but the mechanism behind it remains poorly understood. Moreover, existing approaches for $\mathrm{@k}$ optimization use...
-
58
Diagonal Equivalence in State Space Models: Characterization, Retrieval Rank, and Selective Extensions
Dennis Rochau
cs.LG · math.OC · stat.ML
We classify real and complex state-space models (SSMs) with linear transitions, distinguishing fixed-basis diagonal equivalence from input-output equivalence and tracking coefficient-conditioning windows under coordinate changes. We characterize diagonal equivalence through commuting transition families whose members are diagonalizable over the chosen field. With time-shared token-local matrix maps, we identify this class's state-mediated...
-
59
Optimal Subgroup Discovery at Every Support Threshold
Lincen Yang, Qi Huang, Niki van Stein, Thomas Bäck, Matthijs van Leeuwen, Zhong Li
cs.LG
Subgroup discovery aims to identify regions of the feature space where a target variable deviates from its marginal distribution. A fundamental challenge in this problem is the trade-off between a subgroup's deviation and its support: small subgroups can be highly deviating but statistically meaningless, while large ones cannot deviate much by construction. Existing methods tackle this challenge by collapsing the two into a single scalar...
-
60
TSB-SEG: A Systematic Time-Series Segmentation Benchmark
Félix Chavelli, Arik Ermshaus, Fan Yang, Patrick Schäfer, John Paparrizos, Paul Boniol
cs.LG
Time-series segmentation, studied as either Change Point Detection (CPD) or State Detection (SD), underpins a broad range of monitoring and diagnostic applications. However, the field suffers from a persistent divide between the statistical community (focusing on CPD) and the machine learning and data mining communities (focusing on SD). Consequently, the literature remains fragmented: evaluations typically cover narrow methodological...
-
61
When Does Design Thinking Work? A Convergence Theorem for the Iterate-Test Core
Paweł Kuraś, Adrian Michalski, Alicja Gerka, Patryk Organiściak
cs.LG
Design thinking is widely taught as an iterative loop of ideation, prototyping, and testing, yet conditions under which such a loop must improve innovation outcomes remain unspecified. Taking problem framing as given, we formalize the iterate-test core as search over a finite prototype space whose utility is learned through noisy evaluations. We prove that the prototype a team believes to be best converges almost surely to a local optimum...
-
62
Hybrid Federated Learning for Multimodal Fusion under Missing Modalities and Site Heterogeneity
Xiaofei Wu, Rongmei Liang, Jian Qing Shi
cs.LG · stat.AP
Multimodal fusion is studied under hybrid federation, where data are partitioned across samples and feature blocks. The proposed two-dimensional federated smooth multiple-kernel large-margin learning method (2D-FedSMKL) integrates availability-corrected modality scores, group-kernel regularization, and smooth worst-site risk aggregation. For fixed maps and propensities, the objective is convex and smooth. Curvature bounds and exact...
-
63
Improved regret bounds in tabular RL under local differential privacy
Hugo Richard
cs.LG
We study regret minimization for time-homogeneous episodic tabular reinforcement learning with $S$ states and $A$ actions under non-interactive $ε$-local differential privacy (LDP). In each episode $k \in [K]$, the decision-maker sends a policy to a user, the user executes the policy for $H$ steps, and sends an $ε$-locally differentially private message back to the decision-maker. When $H=1$, the problem reduces to an LDP contextual bandit...
-
64
The Tower of Babel in artificial intelligence: who should speak, and when it pays to listen
Mohammad Salahshour
cs.LG · cs.MA
Two machines can each recognize an object well and still fail together. We study classifiers that see different parts of one object and send short messages to a receiver that identifies its class. Coordination asks how to design messages together; cooperation asks whether the team outperforms one sender given the whole budget. On three digit datasets, messages designed together beat messages designed apart by 18.07 to 22.68 percentage points...
-
65
Comparative study of global pooling for pre-departure ship fuel consumption prediction: Advantages over per-ship and adaptation-based approaches
Heungjun Im, Bongmin Cha, Agung Nugraha, Donghyun Kim, Seongphil Kang, Minwook Choi, Jeongmin Park, Jihwan Lee
cs.LG · cs.AI
Most fuel oil consumption (FOC) models for voyage planning are per-ship models and thus generalize poorly beyond each vessel's records. Transfer and federated learning have been reported to perform better, but under different protocols that often use inputs unavailable before departure or random data splits. It therefore remains unclear which approach performs best. In this study, we compare 11 per-ship, adaptation-based, and global methods...
-
66
Utility-Aware Generative Recommendation: Utility-Enriched Semantic Codes and Utility-Guided Decoding
Yang Hu, Duo Wang, Ning Li, Kang Li, Junda She, Chengcheng Guo, Kuo Cai, Bin Liu, Youhua Liu, Xiaoshuang Chen
cs.LG
Generative recommendation has been widely adopted in large-scale industrial recommendation systems, where multimodal item embeddings are quantized into offline semantic identifiers (SIDs) and items are retrieved by autoregressively generating their SIDs. However, conventional SID construction often underrepresents item utility reflected in post-exposure behaviors such as shares, long views, and video completion, making semantically similar...
-
67
SHq: Learnable Grids for LLM Weight Quantization
Michael Helcig, Earl Killian, Dan Alistarh
cs.LG
Block-scaled lookup-table (LUT) formats, which represent weights as indices into a small value set (the grid), have become standard for low-bit LLM quantization, ranging from NF4 and BOF4 to the 3-bit LUT instruction in NVIDIA Rubin. While these formats leave the grid definition flexible, existing implementations rely on fixed grids derived either from numerical data types (INT, FP) or assumed weight distributions (NF, BOF). We demonstrate...
-
68
Exact Fusion of Softmax Attention Heads Can Require Exponentially Many Atoms
Hiroyasu Usami, Keisuke Hara, Ayato Tsuboi, Naohiko Matsuda
cs.LG · cs.NE · stat.ML
Fusing independently normalized attention heads can increase the number of explicit key-value entries, or atoms, needed to preserve their output. We prove a sharp separation for $W_d(q)=\sum_{j=1}^dσ(q_j)$: its $d$ two-atom heads use $2d$ head-atom terms, whereas its minimum single-head representation has exactly $2^d$ atoms, even with arbitrary fixed replacement keys, scalar values, and positive masses. The same lower bound holds for every...
-
69
ILR-SMO: Iterative Latent Refinement for Robust Spatial Multi-Omics Integration
Anqi Yu, Xudong Xu, Jianzhi Lu, Yuqi Sun, Bingguo Chen, Chenxi Ma, Weimin Tan, Bo Yan
cs.LG
Spatial multi-omics technologies jointly profile diverse molecular modalities with spatial context, providing a comprehensive view of cellular heterogeneity and tissue organization. To integrate spatial multi-omics data and identify spatial domains, a wide range of unsupervised methods has been proposed. However, recent approaches rely on single-step fusion, directly aggregating heterogeneous modalities into a shared representation, making...
-
70
Unimodal Augmentation-based SSL Objectives Induce Cross-Modal Alignment
Zohar Mashiah, Nir Ben-Ari, Ronen Talmon, Uri Shaham
cs.LG · stat.ML
Recent works have observed implicit cross-modal alignment between independently trained unimodal models. However, the mechanisms underlying the emergence of this phenomenon remain unresolved. In this paper, we demonstrate that unimodal augmentation-based self-supervised objectives implicitly induce cross-modal alignment. We show that low unimodal losses force high cross-modal alignment, whereas a high unimodal loss prevents such alignment. We...
-
71
Gibbs Variational Last Layers for Long-Horizon Dynamics Learning
Paul Brunzema, Henrik Hose, Sebastian Trimpe
cs.LG · cs.RO
Accurate long-horizon probabilistic predictions of dynamical systems are a prerequisite for model-based decision-making. Variational Bayesian last layers (VBLLs) are an attractive model class for this setting, offering tractable uncertainty at near-deterministic cost, yet they are trained exclusively with single-step likelihood objectives that may be insufficient for reliable multi-step rollouts. We revisit VBLLs through the lens of Gibbs...
-
72
A Universal Policy Wrapper With Guarantees
Georgiy Malaniya, Anton Bolychev, Pavel Osinenko
cs.LG · eess.SY · math.DS
Many control tasks require driving the system into a prescribed goal set and subsequently keeping it there. The considered setting assumes that a working controller is available that provably reaches and maintains the goal, albeit with suboptimal performance. This controller is referred to as the fallback policy. Conversely, although state-of-the-art reinforcement-learning (RL) algorithms can discover policies with impressive empirical...
-
73
Extending Aircraft Coverage of the Poll Schumann Model Using Public Data
Gabriel Jarry, Ramon Dalmau, Philippe Very
cs.LG
The Poll Schumann model requires airframe and engine parameters, but published tables cover only part of the fleet. Applications must obtain missing parameters, substitute another aircraft type or exclude unrepresented types. We reconstruct parameters using geometry from certification documents and manufacturer manuals, together with ICAO engine records. Published equations supply the physical parameters, regression estimates the propulsion...
-
74
Identifiable Signed Causal Learning via Latent Stochastic Differential Equations for Inferring Brain Dynamic Effective Connectivity
Yiding Wang, Chen Qiao
cs.LG
Understanding how brain regions interact to support cognitive tasks requires characterizing directed causal influences among them, known as effective connectivity (EC). As inter-regional coupling changes over time, a dynamic EC (dEC) estimate is required to track this temporal variation. Existing dEC learning methods typically lack a causal rationality guarantee or an identifiability analysis, and output unsigned connectivity that cannot...
-
75
TensorLOB: Structure-Preserving Tensor Learning for Limit Order Book Forecasting
Yuxuan Gu, Danilo Mandic
cs.LG · q-fin.ST
The limit order book (LOB) records outstanding buy and sell orders across multiple price levels. Its state evolves over time across market side, price level, and order attribute, naturally forming a multiway structure. However, existing LOB models typically flatten these dimensions into a matrix model, thus obscuring the existing structural relationships. Starting from first principles and the market-microstructure intuition of the LOB, we...
-
76
Kolmogorov-Arnold Networks vs. Multi-Layer Perceptrons: A Rigorous Statistical Benchmark on Clinical Tabular Data
Rafał Wodziński, Paweł Szczerbiak
cs.LG · cs.AI
Kolmogorov-Arnold Networks (KANs) have recently been proposed as a promising alternative to Multi-Layer Perceptrons (MLPs). However, empirical evaluations on structured medical datasets remain scarce, and commonly reported metrics like AUROC can yield overly optimistic performance estimates on imbalanced clinical cohorts. In this study, we present a standardized benchmark comparing eleven architectures - nine KAN variants (FastKAN, WavKAN,...
This edition is part of The Daily Abstract — cs.LG archive. Subscribe to receive these in your inbox each morning, automatically translated to Spanish, with reply-to-PDF: arxivdaily.ignorelist.com.
#D99C5E. Built and served on an always-free VM. The masthead is set 14% letterspaced because newspapers do that and it works.