cs.LG · 2026-08-06 · No. 76
Machine Learning, 2026-08-06.
67 new papers in cs.LG. Titles, authors,
abstracts. Links to arXiv. Want this in your inbox every morning? Subscribe →
01 — The papers
67 entries-
01
The Loss Does Not See the Basis, but Adam Does
Devender Singh
cs.LG
Gradient descent on a factored model $W = UV^\top$ is implicitly biased toward low-rank solutions, while Adam, starting from the same small initialization, is not. We trace the difference to the gauge symmetry of the loss, its invariance under $(U, V) \mapsto (UQ, VQ)$. Gradient flow's low-rank mechanism is available to an optimizer only if that optimizer is gauge-equivariant, a condition necessary for the transfer but not sufficient for...
-
02
SSTQ:Privacy-Preserving Vector Quantization via Subsampled Stochastic TurboQuant
Adel Javanmard, David P. Woodruff, Vahab Mirrokni
cs.LG · cs.AI · stat.ML
Achieving local differential privacy in distributed optimization while maintaining low communication cost remains challenging. Existing vector quantization methods, such as vqSGD, use high-dimensional geometric constructions but incur unfavorable dimension-dependent variance. In this work, we propose Subsampled Stochastic TurboQuant (SSTQ), a framework that combines overcomplete equal-norm tight frames, coordinate subsampling, and...
-
03
DASyR-LLM: Domain-Aware Symbolic Regression with LLMs for Kinetic Model Discovery
Roberto Aliaga Medina, Paulina Quintanilla, Antonio del Rio Chanona
cs.LG · cs.CE · cs.SC
Kinetic model discovery is a central challenge in chemical engineering, as accurate rate expressions are essential for understanding and controlling chemical and biological processes. Symbolic regression (SR) has emerged as a powerful data-driven approach for identifying interpretable kinetic models, but usually operates without domain knowledge, often exploring physicochemically implausible models. Large language models (LLMs) offer a...
-
04
Reward Structure Shapes the Interaction Between Episodic Exploration and Neural Memory in Reinforcement Learning
Jai Malegaonkar, Rohan Patil, Henrik I. Christensen
cs.LG
In partially observable reinforcement learning, agents face a dual bottleneck: they must explore to encounter rewarding states and retain that experience in memory to optimize their policies. Exploration bonuses and memory architectures are traditionally evaluated in isolation, leaving their interaction unmeasured, and standard notions of sparse reward conflate temporal signal density with what the reward actually supervises. We present a...
-
05
BnBERT-iPET: Sparse Few-Shot Language Modeling for Bengali via Lottery Ticket Pruning
Sajib Hossain, Md Kamrus Samad, Anan Ghosh, Labib Imam Chowdhury, Nabeel Mohammed
cs.LG
Deep neural networks have shown impressive success in NLP tasks owing to their complex structure and huge number of edges. Achieving state-of-the-art performance in natural language processing with a large pre-trained model such as BERT is expensive and time-consuming, carries a large carbon footprint, and is difficult to realize on machines with minimal computational capability. This creates a barrier to training complex models for...
-
06
Multimodal Spatiotemporal Atmospheric Data Assimilation with Latent Flow-matching
Dibyajyoti Chakraborty, Romit Maulik
cs.LG · math-ph · physics.ao-ph · physics.flu-dyn
Data assimilation (DA) uses Bayesian inference to update the state of a numerical forecast model with observed data. In this study, we propose a fundamentally different, unified approach to atmospheric data assimilation. We use latent video flow-matching to sample temporally consistent trajectories from a prior trained using ERA5 reanalysis (69 variables over an 8-day window). We also use posterior sampling to assimilate real observation...
-
07
MALT: Lightweight Curvature-Aware Muon via Diagonal Preconditioning
Tongle Wu, Huanyu Dong, Ying Sun, Ziye Ma
cs.LG
Muon has recently emerged as a promising alternative to AdamW for language model pretraining by orthogonalizing momentum matrices using Newton-Schulz iterations. Although Muon mitigates gradient anisotropy, it does not explicitly account for the curvature geometry of the loss landscape and may therefore remain sensitive to curvature anisotropy. We bridge this gap by proposing MALT (Muon Augmented by Lightweight Two-sided Preconditioning),...
-
08
Capability-Gated Planning: Cost-to-Goal Discovery and the Limits of Myopic Experiment Selection
Ahmed Hassoon, Mark Dredze
cs.LG · cs.AI
Systems that automate scientific discovery must repeatedly decide which experiment to run, which hypothesis to test, which tool to build, and when to stop. Many systems make these decisions by maximizing a myopic score such as expected information gain per unit cost or a learned plausibility score. We identify a structural limitation of this approach. Some actions are constructive: they acquire an epistemic capability (an instrument, assay,...
-
09
Learning When to Stop: Prefix-Optimal Dynamic Diffusion Policies for Continuous Control
Rohit Kumar Salla, Manoj Saravanan, Simon Stepputtis
cs.LG · cs.RO
Diffusion policies are a powerful policy class for continuous control, but their iterative denoising process creates a substantial computational bottleneck. Reducing this cost requires adapting the number of denoising steps to the difficulty of each action while preserving task performance. We introduce Prefix-Optimal Generative Policies (POGP), a framework that learns a prefix value function at every intermediate denoising step through a...
-
10
Optimizing What Policies Learn From: Recoverability-aware Rollout Intervention Learning
Zheyuan Zhang, Manqing Mao, Hong Wang, Zhuoer Wang, Samson Koelle, Jie Yuan, Yanjun Lin, James Feng, Nikki Lijing...
cs.LG · cs.CL
Critic-free group-based reinforcement learning has become a scalable approach for post-training large language models. However, most existing methods allocate the same number of rollouts to every task and trajectory state, even though some rollouts provide much more useful learning signals than others. Recent work has started to treat rollout generation as an adaptive decision, but two important limitations remain. First, intervention...
-
11
MultiPathFormer: Towards a Foundation Model for Multipath Wireless Propagation
Blessed Guda, Kayley Sze, Carlee Joe-Wong
cs.LG · cs.AI · eess.SP
Recent advances in machine learning have enabled training of wireless foundation models, which aim to support tasks such as channel estimation, beam prediction, and localization based on wireless signals. Existing wireless foundation models typically pretrain on channel tensors using masked reconstruction over subcarriers, antennas, or time but ignore the physical characteristics of wireless propagation. In this work, we propose to instead...
-
12
Canonical Joint Energy-Based Model on CIFAR-10: failure modes and practical indistinguishability of Predictor-Corrector and SGLD samplers
Dmytro Knopov
cs.LG · stat.ML
Joint Energy-Based Models (JEM) unify classification and generation within a single network and support out-of-distribution (OOD) detection. Canonical JEM training relies on stochastic gradient Langevin dynamics (SGLD); a theoretically motivated alternative, the Predictor-Corrector (PC) sampler, has not previously undergone a systematic replication test on the canonical model. We reproduce canonical JEM on WideResNet-28-10 without...
-
13
Stochastic Emulation using Generalized Stratified Sampling for Performance-Based Risk Optimization of Structures
Isabela D. Rodrigues, Seymour M. J. Spence, Henrique M. Kroetz, André T. Beck
cs.LG · stat.CO · stat.ML
Metamodels are instrumental in reducing the computational burden associated with nested reliability analyses and optimization loops in Performance-Based Risk Optimization (PBRO) of structures under stochastic loads. In this context, stochastic emulators are particularly useful because they approximate response distributions while accounting for the intrinsic stochasticity of the simulator. Among these methods, Stochastic Polynomial Chaos...
-
14
EvolveNet: Collaborative Harness Evolution for Agent Self-Improvement
Jun Nie, Yonggang Zhang, Qianshu Cai, Yiu-ming Cheung, Xinmei Tian, Bo Han
cs.LG
The capabilities of an LLM agent depend not only on its model but on the harness: the executable program that constructs context, invokes tools, verifies results, and recovers from failure. Recent work shows that evolving the harness yields persistent improvements without updating model weights. Existing approaches, however, assume that all execution experience can be routed to a single optimizer, which evolves one harness along a sequential...
-
15
SpecRoll: Fast-Slow Verifier-Feedback Adaptation for Speculative Reinforcement Learning Rollouts
Nhat Minh Pham, Duy Tung Doan, Thi Duyen Ngo, Vinh Van Nguyen, Khac-Hoai Nam Bui
cs.LG · cs.CL
Reinforcement learning (RL) post-training improves the reasoning capabilities of large language models, but autoregressive rollout generation remains a major efficiency bottleneck. Speculative decoding can accelerate generation, yet applying it during RL is difficult because the target policy continually evolves: static proposers become stale, while frequent drafter updates add substantial overhead. We introduce SpecRoll, a speculative...
-
16
CheMLFlow: An Open-Source Platform for Cheminformatics and Materials Informatics Applications
Brendan Smith, Susana Lopez-Moreno, Eric Dolores-Cuenca, Sangil Kim, Jose L. Mendoza-Cortes, Nijamudheen Abdulrahiman
cs.LG · cond-mat.mtrl-sci · cond-mat.other · cs.AI · physics.chem-ph
CheMLFlow is an open-source platform for building and executing end-to-end, high-throughput, and agentic workflows for scientific and technological applications. CheMLFlow targets a common bottleneck in scientific machine learning development, where researchers often need to assemble data acquisition, curation, representation, model training, validation, screening, interpretation, and reporting into a reproducible pipeline, even when their...
-
17
SVI-DAG: A Structured Variational Inference Approach to Bayesian Causal Discovery
Shrenik Zinage
cs.LG · cs.AI
Bayesian causal discovery seeks to determine the posterior distribution of causal theories, which are interpreted as directed acyclic graphs (DAGs) that explain the observed data. The resulting posterior allows systematic reasoning regarding epistemic uncertainty within these theories. Nonetheless, finding such graphs is difficult due to identifiability problems and limited observational data. Furthermore, precisely approximating posterior...
-
18
Optimal Training-Time Scaling in Gradual Adaptation
Zonghuan Xu, Krishna Harish
cs.LG
In gradual adaptation, how should the training time on each task change as the number of intermediate tasks increases? We study this question for overparameterized linear regression tasks that change smoothly and share a zero-loss solution. With $N$ tasks and training time $s_N$ on each, the final learning progress converges to a continuum curve when $Ns_N\toτ$. The limiting progress is $Θ(τ)$ for small $τ$ and $Θ(τ^{-1})$ for large $τ$, so...
-
19
Consistency-Driven Co-Evolution for Self-Supervised Cross-Representation Learning
Xuehang Guo, Pengyuan Li, Tom Hope, Tirthankar Ghosal, Manling Li, Qingyun Wang
cs.LG · cs.AI · cs.CL
As chart images, tabular data, and visualization code play increasingly important roles across diverse domains, cross-representation understanding across these modalities poses fundamental challenges for AI systems: the relationships across representations are inherently \textit{one-to-many}, supervision is ambiguous and costly, and model optimization lacks a principled signal that is both direction-adaptive and representation-generalizable...
-
20
Variational Bounds for Perceptron Learning from Structured Data
Francesco Camilli, Pierluigi Contucci, Federica Gerace, Emanuele Mingione
cs.LG · cond-mat.dis-nn · math-ph · stat.ML
We introduce a variational approach to a finite-temperature continuous-spin perceptron trained on a Gaussian mixture. The model allows for a broad class of concave utilities and log-concave separable prior measures on the spins. By combining the interpolation method with log-concavity and concentration estimates, we derive lower and upper minimax variational bounds for the limiting quenched pressure. Remarkably, the two bounds differ only in...
-
21
Training Crossroads for Recurrent Vision Transformers: Recurrence, Neural ODEs, and Deep Supervision
Grzegorz Gruszczynski, Pawel Olszowiec, Michal Byra, Grzegorz Stefanski, Alberto Presta
cs.LG · cs.CV
Vision Transformers (ViTs) achieve strong image-recognition performance, but their parameter count grows linearly with depth when each block is independently parameterized. Single-block recurrent ViTs (bViT) remove this growth by repeatedly applying one shared block. Rather than proposing a new architecture, we fix a bViT and provide a controlled empirical characterization of three training and inference regimes under a common CIFAR-100...
-
22
Robust Control under Stationary Ambiguity
Konrad J. Mueller, Amira Akkari, Ben Wood, Lukas Gonon
cs.LG · q-fin.CP
Control policies optimized in simulation can perform poorly in the real system when the parameters $x$ of the simulator are estimated from limited data but the resulting parameter uncertainty is not represented inside the simulation. A common way to incorporate such ambiguity is to simulate each trajectory of the system under a randomly drawn value for $x$. Since the policy cannot observe the drawn value, it must initially choose controls...
-
23
MGSB: Manifold Gated Signature Branch Pressure-Domain Baseline Architecture for Two-Phase Pipeline Flows Under Distributional Shift
Issah Suleiman, Sormeh Serpoosh, Nadine Elkholy, Hicham Ferroudji, Mohammad Azizur Rahman, Matthew Hamilton
cs.LG
Leak detection models for multiphase pipelines often degrade when deployed under flow regimes that differ from training. Existing evaluations typically assess performance under in-distribution operating conditions, masking failures caused by regime transitions such as bubble-to-slug flow. We propose the Manifold Gated Signature Bias (MGSB), a regime-aware architecture combining regime-conditioned feature fusion, a TT-RoughPath encoder, and...
-
24
Above-ground Biomass Estimation with Geospatial Foundation Models
Ghjulia Sialellia, Linus Scheibenreif, Jan Dirk Wegner, Konrad Schindler
cs.LG
Accurate estimation of Above-Ground Biomass (AGB) from satellite imagery is essential for the large-scale monitoring of carbon stocks, yet it remains a challenging regression task at global scale. Geospatial Foundation Models (GFMs) have recently emerged as a promising machine learning paradigm to derive general-purpose representations from Earth observation data, but their utility for quantitative regression tasks like biomass estimation...
-
25
Agentic Reinforcement Learning with Observation-Calibrated Self-Distillation
Yi Yang, Cong Qin, Xiaodan Liu, Chishui Chen, Qing Dong, Yan Zhang, Cao Liu, Zhao Yang, Lu Pan, Jiaye Lin, Yi Feng
cs.LG · cs.AI · cs.CL
Large language model agents are commonly trained through reinforcement learning with sparse trajectory-level rewards, which offer limited guidance on how strongly individual tokens should be updated. On-Policy Self-Distillation (OPSD) addresses this by re-scoring generated tokens under a privileged replay view to obtain dense, token-level supervision. However, we identify a confounding issue: the resulting support may reflect both the...
-
26
Continual-Learning Physics-Informed Neural Networks for Parameterized Partial Differential Equations
Xujia Chen, Xinyue Hu, Letian Chen, Yi Liu, Wenhui Fan
cs.LG
Physics-informed neural networks (PINNs) incorporate governing equations into neural-network training and can approximate PDE solutions without requiring large observational datasets. Parameterized PINNs (ParamPINNs) further take physical parameters as inputs, allowing a single model to represent a family of PDE solutions over a parameter domain. Existing ParamPINNs, however, still face inefficient training, uneven accuracy across parameters,...
-
27
IMFACT: Counterfactual Explanations for Time Series via Intrinsic Mode Function Substitution
Udo Schlegel, Julian Rakuschek, Thomas Seidl, Andreas Holzinger, Tobias Schreck, Javier Del Ser
cs.LG · cs.AI
Oscillatory signals, such as vibration, carry class-discriminative information in specific frequency bands; perturbing them in raw feature space for counterfactual analysis easily destroys their temporal structure and produces physically implausible results. In this work, we introduce IMFACT (IMF-based counterfACTuals), a model-agnostic framework for generating plausible counterfactual explanations for time series classifiers that operates in...
-
28
Attention, Anomalies! Handling Attention Layers in Unsupervised Federated Outlier Detection
Mihailo Ilić, Miloš Savić, Vladimir Kurbalija, Mirjana Ivanović, Giancarlo Fortino, Dušan Jakovetić
cs.LG
Attention layers are the backbone of today's most powerful and impactful models. Models with multi-million and billion parameters rely on contextual knowledge provided by attention layers. However, their use goes well beyond just being the core component of large language models. One particularly interesting application is in Memory Augmented Autoencoders (MemAE), specifically for unsupervised representation learning in outlier detection...
-
29
A 6G Integrated Sensing and Communication Framework for Railway Intrusion Detection and Collision Prediction
Ajeet Kumar Yadav, Sankaran Balasubramaniam, Aritra Chatterjee, Vinod Aduru, Yogesh Simmhan, Pandarasamy Arjunan
cs.LG · cs.AI · eess.SP
Integrated Sensing and Communication (ISAC) combines sensing and communication to efficiently utilize wireless resources and is emerging as a key paradigm for next-generation wireless networks. By leveraging the wide bandwidth, high frequencies, and massive antenna arrays of 5G-Advanced and 6G systems, ISAC enables physical-layer sensing using Channel State Information (CSI). The 3rd Generation Partnership Project (3GPP) Release 19 identifies...
-
30
Benchmarking Deep Learning Models for Dense Event Classification of Offshore Wind Infrastructure in Sentinel-1 Time Series
Thorsten Hoeser, Felix Bachofer, Claudia Kuenzer
cs.LG
Monitoring of offshore wind energy infrastructure life cycles, especially during the deployment phase, is an important contribution for stakeholders to make informed decisions in a phase of increasing deployment activities. ESA's Sentinel-1 Synthetic Aperture Radar (SAR) mission produces large data archives that enable the global monitoring of offshore wind infrastructure. Turning these high-volume archives into information requires...
-
31
Design Choices That Matter: A Functional ANOVA Analysis for Remote Sensing Multi-Label Classification
Maryam Gholami Shiri, Eva Tuba, Sašo Džeroski, Tome Eftimov, Ana Nikolikj
cs.LG · cs.AI · cs.CV
Benchmarking deep learning (DL) models for multi-label classification (MLC) of remote sensing images (RSI) typically yields rankings that do not generalize beyond the evaluated datasets. In this work, we move beyond rankings by employing functional analysis of variance (fANOVA) to systematically quantify the contributions of individual design choices and their interactions to performance variability. We conduct two empirical analyses covering...
-
32
Personalized Federated Sparse Adaptation of Time-Series Foundation Models
Priyanka Nihalchandani, Naman Srivastava, Varun Ojha, Pandarasamy Arjunan
cs.LG · cs.AI · stat.ML
Federated adaptation of time-series foundation models (TSFMs) is attractive for building energy forecasting because meter data are private, distributed, and highly non-IID. However, a single parameter-sharing strategy is unlikely to serve all pretrained TSFMs or building clients: fully shared adapters can suppress building-specific temporal behavior, while fully local adaptation discards cross-building transfer. We propose a personalized...
-
33
The Sample Complexity of Distributionally Robust PAC Learning under Cressie--Read Divergences
Elad Aigner-Horev, Daniel Rosenberg, Roi Weiss
cs.LG
We study distributionally robust PAC learning for the $0$--$1$-loss, where adversarial perturbations of the data distribution are constrained by a Cressie--Read divergence of order $k>1$ and radius $ρ\geq 0$. For hypothesis classes with VC dimension $d$, we establish realizable and agnostic sample-complexity bounds tight up to constant and logarithmic factors, respectively; ordinary empirical risk minimization attains both rates up to...
-
34
Diverse and Plausible Algorithmic Recourse via Tractable Recourse Distributions
Anagha Sabu, Hrithik Suresh, Narayanan C. Krishnan
cs.LG
Algorithmic recourse seeks to help individuals reverse unfavorable automated decisions by recommending actionable changes that achieve a desired outcome. As an individual usually has several distinct routes to a favorable decision, and different people can act on different ones, a recourse system should offer multiple realistic alternatives rather than one. Existing approaches formulate recourse as an optimization problem that constructs one...
-
35
Differentiating Through Dual Prices: End-to-End Policy Learning Under Capacity Constraints
Mohammadsaeed Haghi, Mahdi Salmani, Nima Kelidari
cs.LG
Many social services assign scarce resources, such as housing assistance or hospital interventions, to people who arrive one at a time: each arrival must receive a decision immediately, and the long-run usage of every resource must stay within its capacity. We study how to learn such an assignment policy from logged observational data. The standard pipeline is decision-blind: fit one outcome model per arm by regression, price each capacitated...
-
36
Active Learning Guided Design Space Refinement for Scalable Multi-Objective Bayesian Optimization in Materials Discovery
Alexandros Ntagiantas, Panagiotis Tsilimidos, George Giannakopoulos, Christoforos Rekatsinas, Panagiotis Krokidas
cs.LG · cond-mat.mtrl-sci
Advanced materials discovery increasingly relies on machine learning and Bayesian optimization to explore large discrete design spaces under limited evaluation budgets. However, conventional Bayesian optimization (BO) can become inefficient as candidate spaces grow, often evaluating low-value regions before reaching informative areas. We propose an active-learning (AL)-guided adaptive search-space refinement framework combined with...
-
37
An entropic explanation of insistence on sameness in autism
Przemysław Śliwiński
cs.LG · q-bio.NC · stat.ML
An information theory-based framework is proposed in attempt to explain insistence on sameness in autism as an instance of a general behavior pattern in which an individual tries to reduce surprise and uncertainty. It offers a new definition of autism as an impairment in which cognitive functions are restricted to discrimination, memorization and prediction of tangible properties of the environment. An analogy between insistence on sameness...
-
38
Why Ranking Anomaly Detection Algorithms Isn't as Reliable as You May Think
Simon Klüttermann, Jérôme Rutinowski, Frederik Polachowski, Alice Kirchheim
cs.LG
Anomaly detection is a safety-critical machine learning problem with applications ranging from fraud detection to network intrusion prevention and industrial monitoring. Despite the large number of proposed anomaly detection algorithms, many novel methods claim state-of-the-art performance. However, many authors do so under benchmark settings that are not aligned with one another. This lack of comparability raises concerns regarding the...
-
39
Rethinking Reservoir Pruning: A Dynamical Perspective for Echo State Networks
Sudip Laudari, Puspa Raj Adhikari
cs.LG · cs.AI · math.DS
Echo State Networks (ESNs) offer an efficient framework for temporal prediction, but their randomly initialized reservoirs are often over-parameterized and dynamically redundant. Existing pruning methods largely rely on static connectivity or activation statistics, which may overlook neurons that shape input-driven state transitions. We propose Dynamical Mode Pruning (DMP), a reservoir pruning method that ranks neurons by their contribution...
-
40
A Model Merging Approach for Continual MLLM Unlearning
Yuhang Wang, Linlin Zhang, Haoxuan Ji, Xianmin Ye, Zhenxing Niu, Haichang Gao
cs.LG · cs.AI
Multimodal large language model (MLLM) unlearning methods have been proposed to remove private, sensitive, or proprietary information from well-trained models. However, most existing MLLM unlearning methods are designed for one-shot requests and fail to adequately address continual scenarios, as repeatedly applying one-shot operations leads to cumulative utility degradation, unlearning rebound, and retention drift. We introduce Merging for...
-
41
Learning Compression Rules for Network Traffic
Quentin Lampin, Éloi Sainte-Beuve, Louis-Adrien Dufrène, Guillaume Larue, Massih-Reza Amini
cs.LG · cs.NI
We study the problem of learning compact rule-based compressors for structured network traffic. Each packet is a record of header fields that are highly redundant within a flow, and a compressor is a small set of rules matching such records and replacing predictable fields with short codes. We cast rule learning as a two-stage problem: (i) an unsupervised structure-discovery stage that recursively partitions training packets using a...
-
42
Discretization and Statistical Consistency of Functional Flow Matching
Lennon J. Shikhman
cs.LG · math.PR · stat.ML
Functional flow matching is posed on distributions of functions but implemented from finitely many coefficients or point values. Under scattered or adaptive refinement, the resulting conditioning sigma-algebras need not be nested, so martingale convergence does not justify the sensor limit. We prove strong $L^2$ convergence of finite conditional velocity targets for every strongly consistent sequence of finite-rank reconstructions, with...
-
43
Local Violation Certification for Linear Predict-Then-Optimize Pipelines
Ş. İlker Birbil, Wenhao Chi
cs.LG · math.OC
Data-driven decision pipelines combining predictive machine learning models with downstream optimization software are increasingly used to make high-stakes operational decisions. Certifying the safety, fairness, and reliability of these decisions is essential, yet traditional scenario generation methods rely on repeated random testing, which becomes computationally prohibitive when failure events are rare and offers little insight into why...
-
44
Beyond Linear Dynamics: Neural Bilinear Dynamical Models for Time Series Forecasting
Mengzhou Gao, Huangqian Yu, Pengfei Jiao
cs.LG · cs.AI
Time series in real-world applications are often generated by nonlinear dynamical systems, making accurate forecasting challenging. Existing approaches that explicitly model system dynamics typically rely on linear assumptions or Koopman-based linearizations, which may inadequately capture complex nonlinear behaviors and lead to error accumulation in long-horizon prediction. To address this limitation, we propose the Neural Bilinear Dynamical...
-
45
Tropical Algebraic Geometry for Neuronal Representations: An Arakelov-Green Measure Based Descriptor for Graph Learning
Yuyang Zhang, Weihan Xu, Xuehai Zhou, Shucheng Cao, Qihuang Zhang
cs.LG · cs.AI · cs.CG
The quantitative analysis of 3D neuronal morphologies requires capturing both graph topology and spatial geometry. Current message-passing Graph Neural Networks (GNNs) are bounded by the 1-Weisfeiler-Lehman (1-WL) test, limiting their ability to capture cycles induced by spatial proximities. To address this, we propose a training-free geometric prior based on tropical algebraic geometry. We apply the recently established tropical Abel-Jacobi...
-
46
Robustness Emerges Early in Training Dynamics, but Is Not Preserved
Jiangang Yang, Wenhui Shi, Lu Hu, Jing Xing, Jian Liu
cs.LG · cs.CV
Robustness to natural corruptions remains a fundamental challenge for deep neural networks. In this paper, we identify a robustness fading phenomenon where shallow layers spontaneously develop robust representations and flat loss landscapes in early training, yet these properties are not preserved during standard convergence. To address this, we propose a framework that performs strategic interventions on training dynamics to stabilize the...
-
47
Generative Optimization for Incentivized Advertising with Global Level Constraints
Gege Chen, Ning Luo, Hao Jiang, Da Li, Wenzheng Shu, Teng Sha, Yanxiang Zeng, Wenxin Tai, Fan Zhou, Xialong Liu
cs.LG · cs.AI
Incentivized advertising allocates monetary or virtual rewards to drive user engagement, where a key challenge is optimizing continuous incentive magnitudes under strict global constraints. This problem is complicated by high-frequency interactions, delayed feedback, and non-Markovian user dynamics such as fatigue, which limit the effectiveness of existing uplift modeling and constrained reinforcement learning approaches. To address these...
-
48
SPOT: Sparse Probing and Outcome Calibration for On-Policy Distillation
Zikun Qu, Min Zhang, Mingze Kong, Zhiwei Shang, Yikun Ban, Shuang Qiu, Zhongxiang Dai
cs.LG · cs.AI
On-policy distillation (OPD) provides dense teacher supervision on student-generated trajectories, but standard reverse-KL training can assign insufficient probability to other plausible continuations. Teacher entropy alone does not reveal whether uncertainty is concentrated among a few plausible next tokens or dispersed over a long probability tail, nor whether the student already represents those candidates well. Moreover, local teacher...
-
49
Not Every Divergence Should Be Suppressed: Counterfactual Recoverability in On-Policy Distillation
De Jiang, Zhengyang Zhang, Kehong Yuan, Shaohua Ma
cs.LG · cs.AI
On-policy distillation (OPD) supervises student-visited trajectories, yet divergence-based rules cannot determine whether an erroneous prefix remains correctable. We formulate this decision as counterfactual recoverability and replay each error state through budget-matched teacher-continuation and rollback branches. Based on their relative success, states are categorized as recoverable, irreversible-but-avoidable, or ambiguous, and these...
-
50
MESH: Memory-Efficient Sinkhorn Optimization for Mixture-of-Experts Training
Masato Fujitake
cs.LG · cs.AI · cs.CL
Memory-efficient matrix optimizers such as Sinkhorn gradient descent remove most AdamW optimizer state for dense Transformer matrices, but direct application to Mixture-of-Experts (MoE) training is unreliable. We study this failure in a controlled 110M-parameter nanowhale DeepSeek-style MoE pretraining setting. A SAGE/Sinkhorn hybrid reduces optimizer state from 0.883GB to 0.331GB but degrades evaluation loss to 3.8265, far above the AdamW...
-
51
Training-Free Hashing-Based Attention via Binary Principal Components
Daohai Yu, Zhanpeng Zeng, Keyu Chen, Wenhao Li, Zhifeng Shen, Luxi Lin, Ruizhi Qiao, Xing Sun, Rongrong Ji
cs.LG · cs.AI · cs.CL
Long-context large language models (LLMs) are increasingly deployed in real-world applications, yet self-attention remains a major efficiency bottleneck -- especially during decoding -- due to the necessity of repeatedly processing ever-growing key-value (KV) caches. Existing sparse attention reduce computation by attending to fewer KV pairs, but often suffer from substantial accuracy degradation, require additional training, or rely on...
-
52
Elbow-Based MoE Routing: A Training-Free Inference Time Plugin for Expert Selection
Robin Pan, Raymond Liu, Daniel Fang, Adelina Andrei, Rosa Wu
cs.LG
Mixture-of-Experts (MoE) models enable model scaling while maintaining low inference-time compute by activating only a subset of experts per token. However, conventional routing relies on a fixed top-k selection, forcing the model to spend the same compute regardless of how many experts are relevant. We introduce elbow-based routing, a training-free inference-time modification that dynamically adjusts the number of experts on a per-token...
-
53
When Proxy Prediction Becomes Equation Reconstruction: Diagnostics and Residual Learning for Factor-Derived Proxy Supervision
Chayan Lahiri, Ahmed Shafee, Cody Fehringer
cs.LG
Scientific machine learning often relies on proxy targets computed from known domain factors when direct observations are limited. When those same factors are used as model inputs, however, high predictive accuracy may reflect reconstruction of the proxy-generating equation rather than robustness to degraded factor information. We study this problem in RUSLE-derived soil-loss proxy prediction under controlled degradation of the...
-
54
NeuroPB: Scaling Neural Decoding with Pretrained Behavioral Representations
Luyao Jin, Yonghao Song, Huan Zhao, Vincent C. K. Cheung, Wei-Hsin Liao
cs.LG
Decoding continuous motor trajectories from neural activity is essential for developing practical brain-computer interfaces (BCIs). However, current neural decoders are constrained by the limited scale and heterogeneity of neural recordings. In contrast, behavioral data can be collected more readily and at substantially larger scale from humans, animals, simulations, and robotic systems. Here, we introduce NeuroPB, a framework that scales...
-
55
Non-asymptotic implicit bias of logistic regression at early-stage gradient descent dynamics
Han Bao
cs.LG · stat.ML
Gradient descent has been of particular interest in modern machine learning beyond sole focus on optimization. Implicit bias emerging from optimization, though not being encoded by the learning objective, often prevents from overfitting to spurious patterns. A typical instance is the max-margin implicit bias of a linear classifier, widely established for exponentially tailed loss functions. Even after having a given dataset separated, the...
-
56
NodeJEPA: Structure-Conditioned Latent Prediction for Node-Level Graph Self-Supervised Learning
Tinghe Zhang, Jian Xu, Jiaheng Chen, Jiaxing Li, Yucheng Xiao, Qiang Wang
cs.LG · cs.AI
Self-supervised learning on graphs is largely shaped by contrastive methods that depend on carefully designed augmentations, and by generative methods that reconstruct node attributes in the input space. Both paradigms can entangle representations with low-level input statistics rather than with relational structure. Joint-embedding predictive architectures (JEPA) instead learn by predicting latent targets rather than reconstructing inputs....
-
57
Towards Trustworthy Hypergraph Neural Networks under Label Noise
Mengyao Zhou, Zhiheng Zhou, Xiao Han, Guiying Yan
cs.LG · cs.AI
Hypergraph neural networks (HGNNs) have demonstrated remarkable capabilities in processing complex higher-order relationships. However, their performance is highly dependent on labeled data, making them vulnerable to label noise. Despite advances in learning with label noise (LLN) and graph learning with label noise (GLN), noisy-label learning on hypergraphs remains underexplored. In this paper, we present a systematic study of hypergraph...
-
58
EvtGraph: Event-Adaptive Compression for Sparse Temporal Graph Learning in Multimodal Time Series
Ziqian Wang, Tingxiong Xiao, Yuxiao Cheng, Jinli Suo
cs.LG
Multimodal temporal data are inherently irregular and uneven in information density, yet most models rely on uniform discretization, leading to inefficient representations. We propose \textbf{EvtGraph}, a unified framework that aligns computation with temporal salience under explicit budget constraints. EvtGraph reparameterizes sequences into event-level tokens via event-adaptive compression (EAMC), selects a compact subset with a node budget...
-
59
Manipulation-Proof Oblivious Audits against Deceptive Model Providers
Augustin Godinot, Sofiane Azogagh, Julien Ferry, Sébastien Gambs
cs.LG · cs.CR · cs.CY
Audits have emerged as a critical instrument for algorithmic governance, providing a mechanism for external scrutiny and governance of machine learning models. However, ensuring the integrity of such assessments remains a challenging issue. For instance in regulatory contexts, audits are typically declared or easily detected, thus enabling model providers to manipulate the process, whether intentionally or inadvertently. This vulnerability is...
-
60
Looking in the Mirror: Introspecting Side-Effect Misalignments Induced by Fine-Tuning
Kotaro Yoshida, Laura Gomezjurado Gonzalez, Yukinori Yamamoto, Yuji Naraki, Ryotaro Shimizu, Wenya Wang
cs.LG
Fine-tuning enables a source model to acquire desired capabilities and behaviors in a target domain while retaining much of its general-purpose competence. However, this adaptation process can also degrade alignment properties that were present in the source model. Recent work has shown that large language models can be trained using LoRA-based modules known as introspection adapters (IAs) to describe behavioral changes induced by...
-
61
ATLAS: Adaptive Topological Learning with Abstract Successors for Continual Learning
R. Blake Lawlor, Daniel S. Brown
cs.LG · cs.AI
Contemporary model-free reinforcement learning algorithms can achieve very high performance, but have low sample efficiency and are not robust to changes in the environment. Model-based algorithms have much higher sample efficiency, but still fail when the environment shifts. This paper introduces Adaptive Topological Learning with Abstract Successors (ATLAS) to combat these challenges. ATLAS uses a Grow When Required network with Successor...
-
62
Cost-Aware Multi-Objective Bandits: Theory and Application to Budgeted LLM Configuration Evaluation
Bo Xue, Zhi Hong, Jiayi Li, Yuanyu Wan, Ji Cheng, Shuang Qiu
cs.LG
Large language model (LLM) configuration evaluation is challenging due to limited evaluation budgets, varying costs, and multiple competing objectives. In this paper, we formulate LLM configuration evaluation as a cost-aware multi-objective bandit problem, where each configuration evaluation incurs a configuration-dependent cost and yields a noisy vector-valued outcome. Under this framework, we study two fundamental problems: online...
-
63
Real-time probabilistic tsunami forecasting via generative AI
Yusuke Oishi, Takashi Furumura, Fumihiko Imamura
cs.LG
Explicit onshore tsunami inundation forecasting can improve public risk awareness, but deterministically predicted inundation boundaries under highly uncertain conditions, such as near-field tsunamis generated by megathrust earthquakes, may falsely imply safety outside the boundaries. Consequently, current warnings primarily target coastal tsunami height, not onshore inundation. Although machine learning enables instant inundation...
-
64
Efficient Online Lexicographic Generalized Low-Rank Matrix Bandits
Bo Xue, Ji Cheng, Haodong Jing, Hongzong Li, Shuang Qiu
cs.LG · cs.AI
This paper studies generalized low-rank matrix bandits with multiple prioritized objectives. At each round, the learner selects a matrix-valued arm and observes a vector-valued reward, whose components correspond to multiple objectives with different priority levels. Each objective is governed by an objective-specific generalized low-rank matrix model, and the learner evaluates arms according to a lexicographic preference order, prioritizing...
-
65
ArborEnum: Decision Tree Rashomon Sets over Continuous Features
Zakk Heile, Hayden McTavish, Margo Seltzer, Cynthia Rudin
cs.LG · stat.ML
The Rashomon effect describes the phenomenon that many models can achieve nearly equivalent performance on the same learning task, with significant ramifications for robustness, feature importance, and customizability. These use cases motivate the computation of Rashomon sets: the set of all models whose regularized loss is near-optimal. Decision trees are one of the few model classes for which Rashomon sets can be fully enumerated, but this...
-
66
Adaptive Finite-Budget Training for CVaR Risk-Aware Q-Learning
Yifan Wu, Junjie Lei, Wenjie Huang
cs.LG · q-fin.RM
Risk-aware Q-learning (RaQL) provides a model-free, two-timescale estimator for dynamic risk objectives, but its finite-budget behavior remains fragile: fixed inner-loop hyperparameters can produce unstable value estimates, persistent Bellman residuals, and inefficient sample reuse. This paper proposes an adaptive training controller for Conditional Value-at-Risk (CVaR) RaQL and evaluates it on a daily Bitcoin trading task. The controller...
-
67
Sample Complexity of Multicalibration for Multilevel Properties
Jiuyao Lu, Krishnakumar Balasubramanian, Aleksandr Podkopaev, Shiva Prasad Kasiviswanathan
cs.LG · math.ST · stat.ML
Calibration requires a predictor to be unbiased after conditioning on its own predictions. Multicalibration asks for this guarantee simultaneously across a collection of groups. Many prediction tasks ask for several related features of the same conditional outcome distribution: variance is defined relative to the mean, skewness relative to both mean and variance, and conditional value at risk relative to a quantile. We study multicalibration...
This edition is part of The Daily Abstract — cs.LG archive. Subscribe to receive these in your inbox each morning, automatically translated to Spanish, with reply-to-PDF: arxivdaily.ignorelist.com.
#D99C5E. Built and served on an always-free VM. The masthead is set 14% letterspaced because newspapers do that and it works.