cs.LG · 2026-11-04 · No. 163

Machine Learning, 2026-11-04.

75 new papers in cs.LG. Titles, authors, abstracts. Links to arXiv. Want this in your inbox every morning? Subscribe →

01 — The papers

75 entries
  1. 01

    Aligning Language Models with General Preferences: An Empirical Study of Direct Nash Optimization

    Austin Tsai, Saurav Nanda, Ganapathy Parthasarathy, Bhuvnesh Kumar, Parivesh Choudhary

    cs.LG · cs.AI · cs.CL

    Direct Nash Optimization (DNO) aligns a language model to a general preference function by approximating the Nash equilibrium of a two-player preference game. Its practical variant, DNO-Prct, iterates on-policy sampling, LLM-based rating of the samples together with a teacher's gold response, large-margin preference-pair selection and a DPO-style contrastive update. We re-implement DNO-Prct independently and study it on two open-weight...

    arxiv.org/abs/2611.05373 · PDF

  2. 02

    Agnostic Learning by Imitating

    Sílvia Casacuberta, Parikshit Gopalan, Varun Kanade, Omer Reingold, Konstantinos Stavropoulos, Pranay Tankala

    cs.LG · cs.CC · cs.DS

    Given a class of machine learning models and sample access to a data distribution with binary labels, which computational task is harder: training a discriminative model that distinguishes positive and negative labels as well as the best member of the class, or training a generative model whose outputs are indistinguishable from real data by all members of the class? We prove that these two fundamental tasks in computational learning theory...

    arxiv.org/abs/2611.05346 · PDF

  3. 03

    CATF: A Manager-Worker Framework for Context-Aware Time-Series Forecasting

    Yueqiao Chen, Dongyu Liu

    cs.LG

    We study context-aware time-series forecasting, in which a single target variable is predicted from its own history and a set of contextual variables, such as weather and calendar information. As the context becomes more diverse, a single model struggles to capture all of its regimes, while ensemble learning and Mixture-of-Experts approaches rely on manual data partitioning or soft routing, which increases the inference cost and hinders...

    arxiv.org/abs/2611.05325 · PDF

  4. 04

    $Φ$asmaNet: A Staged Spectral Token-Refinement Architecture for Fast Gravitational-Wave Signal-to-Noise Ratio Prediction

    Ioannis Tsioulis, Anastasios Tefas, Nikolaos Stergioulas

    cs.LG

    Quantifying selection effects in gravitational-wave population studies requires millions of costly signal-to-noise ratio (SNR) calculations. We present $Φ$asmaNet, a compact neural architecture for structured tabular regression, with staged spectral refinement as an explicit architectural prior. Learned input tokens are refined through ordered attention-based $Φ$-Stages, each augmented by a $Φ$-Refractor subspace-mixing module, then revisited...

    arxiv.org/abs/2611.05309 · PDF

  5. 05

    LoopVQ: Sub 1-bit KV Cache Quantization for Looped Transformers

    Minsoo Cheong, Sungjoo Yoo

    cs.LG · cs.AI · cs.CL

    Looped transformers are a parameter-efficient alternative to standard transformers for language modeling. By reapplying the same layers across multiple loop steps, recent looped language models match the performance of non-looped models several times their size. While this architecture effectively reduces parameter memory, each loop step still stores its own keys and values, so the key-value (KV) cache grows linearly with the number of loop...

    arxiv.org/abs/2611.05304 · PDF

  6. 06

    Universal Hardness From Masking Channels

    Parikshit Gopalan, Manoj Prabhakaran, Pranay Tankala

    cs.LG · cs.CC

    We prove a broad learning-theoretic generalization of the celebrated hardcore lemma concerning the average-case complexity of computing a Boolean function (Impagliazzo, 1995). Our starting point is the observation that the classic notion of a hardcore measure, which reweights a function's input distribution, is equivalent to an input-dependent masking channel that injects randomness into the function's output distribution until it appears...

    arxiv.org/abs/2611.05296 · PDF

  7. 07

    De Rham cohomological architectures for PINNs and PDEs with topological holes

    Andrew Gracyk

    cs.LG · math.NA

    We attempt to alleviate failure modes of classical physics-informed neural networks (PINNs) in scenarios of topological pathologies corresponding to punctures, holes, poles, and voids by introducing de Rham cohomological augmentations, or Hodge-de Rham PINNs. Vanilla PINNs often encounter pitfalls in hole-based settings due to discontinuous interfaces, nonzero closed path integration, path dependence, correspondence to exact forms via the...

    arxiv.org/abs/2611.05276 · PDF

  8. 08

    iGC: Antibody Optimization via Reinforcement Learning over a Learned Somatic Hypermutation Prior

    Thao Nguyen, Robert G. Alberstein, Ewa Nowara, Homa MohammadiPeyhani, Holly R. Steach, Sarah A. Robinson, Yuval...

    cs.LG · q-bio.BM

    Antibody affinity maturation is a natural optimization process in which B cells acquire mutations through somatic hypermutation (SHM), undergo selection for improved antigen binding, and iteratively evolve toward higher affinity. Computationally modeling this process requires capturing both biologically plausible mutations and antigen-specific selection. However, existing mutation models do not directly optimize binding, while affinity...

    arxiv.org/abs/2611.05270 · PDF

  9. 09

    Stabilizing On-Policy Targeted Guidance in Agentic Reinforcement Learning

    Kevin Frans, Alexander M Rush, Sergey Levine, Pieter Abbeel, Ashvin Nair

    cs.LG

    Naive on-policy RL is often bottlenecked by the capabilities of the base model --- insufficient coverage can result in an inability to learn from certain rewards. At the same time, rewards in modern post-training systems often come from rich textual sources, e.g. judge models with a rubric of criteria. Can we utilize such structures to provide a rich directional signal that enables improving on otherwise difficult to optimize rewards? In this...

    arxiv.org/abs/2611.05266 · PDF

  10. 10

    NovoCRF: Globally Normalized De Novo Peptide Sequencing on a Prefix-Mass Lattice

    Rui Qiao, Lei Xin, Baozhen Shan

    cs.LG

    De novo peptide sequencing has moved from dynamic programming over hand-designed or generative scores to large transformers that learn the task end to end. We present NovoCRF, a conditional random field (CRF) whose states are prefix masses, so that its paths are exactly the peptides within +/-0.06 Da of the precursor mass. A small network scores each edge from physics-derived ion-match features and is trained with its scores globally...

    arxiv.org/abs/2611.05260 · PDF

  11. 11

    Going Deeper With Memories

    Afshin Khadangi

    cs.LG · cs.AI

    Byte level language models avoid tokenizer dependence and provide universal coverage, but processing every byte at full resolution is expensive. We introduce DeepMem, a causal multiscale architecture that separates semantic compression from exact byte preservation. A learned invertible subband transform constructs a hierarchy of shorter sequences, while a lossless carrier and explicit detail sidebands preserve byte identity. Fast weight...

    arxiv.org/abs/2611.05255 · PDF

  12. 12

    Logarithmic Regret for Alternating Mirror Descent

    Tianlong Nan, Christian Kroer

    cs.LG · cs.GT · math.OC · stat.ML

    Alternating online mirror descent (AltOMD) is a simple learning rule for zero-sum games, which improves upon simultaneous OMD numerically and theoretically. Previous work obtained $O(T^{1/3})$ and $O(T^{1/5})$ regret bounds for AltOMD, by leveraging modified-energy-function-based analyses. However, settling whether AltOMD achieves near-constant regret has remained an open question. In this paper we resolve the question of whether AltOMD...

    arxiv.org/abs/2611.05243 · PDF

  13. 13

    Shrinkage prior for implicit generative models

    Shreya Sinha-Roy, Christian P. Robert, Ritabrata Dutta

    cs.LG · stat.CO

    Implicit generative models based on deep neural networks have revolutionized many scientific fields, from weather research to language generation. They successfully model complex random objects arising in real-world problems. In this work, we propose a sample-efficient training mechanism for these models. Our approach leverages regularized inference with shrinkage priors and generalized Bayes. In the large $p$ (parameter dimension) and small...

    arxiv.org/abs/2611.05241 · PDF

  14. 14

    Post-Inference Model Chaining: A Problem Statement and a Model-based Bootstrap for Training End-to-End Digital Twins

    Lukas Madenach, Timo Bailer, Tobias Eilert

    cs.LG · stat.ME

    End-to-end digital twins can be constructed by fitting separate stage models and connecting them after inference. We investigate the Frequentist prediction properties of a commonly used Monte Carlo simulation scheme that forwards complete predictive draws through such a chain when measured stage outputs contain independent normal errors, but the latent stage means are deterministic. Analytical two-stage examples show persistent asymptotic...

    arxiv.org/abs/2611.05225 · PDF

  15. 15

    Fairness Metrics based on the AUC in Clinical Predictive AI

    João Matos, Ben Van Calster, Richard D. Riley, Paula Dhiman, Gary S. Collins

    cs.LG · cs.AI

    Fairness evaluation of clinical prediction models for binary outcomes often involves fairness metrics that compare the area under the receiver operating characteristic curve (AUC, or c-statistic) across subgroups. However, the AUC has been shown to be a non-collapsible metric, meaning that its subgroup specific and overall values are not directly comparable as often implicitly assumed. This raises important questions about how existing...

    arxiv.org/abs/2611.05222 · PDF

  16. 16

    Fusing Earth observation embeddings and spaceborne terrain with meteorological forcing for flood peak depth prediction from high-water marks

    Wen Zhou, Shaowen Wang, Upmanu Lall, Venkatesh Merwade, Dingqi Ye

    cs.LG

    Flood peak depth on normally dry land governs damage and risk to life, but is observed only sparsely, by post-event high-water mark (HWM) surveys, and depth derived from satellite flood extent is produced after the event and inherits the vertical error of the digital elevation model (DEM) it is computed from. Deep-learning flood models have in turn been trained on hydrodynamic simulations rather than on observed depths. We ask whether flood...

    arxiv.org/abs/2611.05220 · PDF

  17. 17

    Privacy for Free in Efficient Agnostic Halfspace Learning

    Giorgio Racca, Amartya Sanyal, Vikrant Singhal

    cs.LG · stat.ML

    We provide the first computationally efficient, differentially private algorithms for agnostically learning halfspaces. In particular, we study three different settings of agnostic learning: improper learning under Gaussian and log-concave marginals; proper learning under Gaussian marginals; and semi-agnostic learning under Gaussian marginals. In all three aforementioned settings, the added cost in the sample complexity due to privacy matches...

    arxiv.org/abs/2611.05208 · PDF

  18. 18

    Multi-Objective Generative Optimization via Expected Hypervolume Fine-Tuning

    Juan Guevara, Riccardo De Santi, Andreas Krause, Raul Astudillo

    cs.LG

    Scientific discovery requires plausible designs spanning useful trade-offs among competing properties: a molecule with strong binding affinity, for example, may be difficult to synthesize. Pretrained generative models, e.g., flow and diffusion generators, provide a foundation for proposing such designs, but steering them toward high-quality, complementary trade-offs remains challenging. Inference-time search incurs high per-sample costs,...

    arxiv.org/abs/2611.05207 · PDF

  19. 19

    Empirical Bayes Flow Matching for Continuous Cryo-EM Heterogeneity

    Daniel Aibinder, Azmi Haider, Dan Rosenbaum

    cs.LG · q-bio.BM

    We address the problem of learning generative priors over high-dimensional latent variables from indirect, noisy observations, with a focus on continuous heterogeneity in cryo-electron microscopy (cryo-EM). In this setting, 3D atomic structures $x \in \mathbb{R}^{3N_a}$ are never directly observed and must be inferred from 2D projection images $y$ generated by a complex, non-linear forward operator. We propose Empirical Bayes Flow Matching...

    arxiv.org/abs/2611.05202 · PDF

  20. 20

    Ensembles of Random Partition-Based Models under Covariate Shift

    Mary Versa Clemens-Sewall, Eliza O'Reilly

    cs.LG · math.ST

    Distribution shifts, including covariate shifts, are ubiquitous in real-world applications of machine learning. Empirical evidence suggests that, across a variety of models with and without downstream adaptation, ensembles tend to be more robust to covariate shift than single models. We extend the theory of covariate shift severity from single models to ensembles, providing theoretical justification for the empirical trends. We introduce new...

    arxiv.org/abs/2611.05195 · PDF

  21. 21

    Stepsize modulation for cross entropy from local quadratic optimization

    Bruno Bauwens, Viacheslav Krivodub, Bingyu Zhang

    cs.LG

    Two methods are presented that perturb a given learning rate schedule in stochastic gradient descent with cross entropy loss. Both methods use an extra forward pass to estimate various derivatives of the minibatch loss. The first method selects a value $η$ by linearising the softmax values and minimizing the squared error $\tfrac 1 2 ||y - q - ηq'||_F^2$, where $q$ is the matrix representing the softmax values of the minibatch and $y$ is the...

    arxiv.org/abs/2611.05191 · PDF

  22. 22

    Provably Learning Mixtures of Linear Experts

    Adrian Candocia, Aamod Varma, Mark Davenport

    cs.LG

    Mixture-of-Experts (MoE) models are widely deployed in modern machine learning. However, few convergence guarantees exist for gradient-based training of MoEs: the objective is nonconvex, and even simple mixtures trap first-order methods in spurious local optima. We analyze a canonical gated mixture featuring linear experts and a gate induced by Gaussian clusters in the input space, and give, to our knowledge, the first global convergence...

    arxiv.org/abs/2611.05184 · PDF

  23. 23

    Basin2Vec: Learning Basin-Centric Embeddings for Scalable Hydrological Modeling

    Muhammad Talha, Xiaofeng Liu, Yuan Yang, Pang-Ning Tan, A. Pouyan Nejadhashemi

    cs.LG · cs.AI

    Reliable streamflow prediction across time and in ungauged basins requires representations that capture hydrologically relevant basin characteristics. Existing methods rely on either fixed handcrafted attributes or general-purpose Earth observation embeddings not tailored to basin structure. We introduce \textbf{Basin2Vec}, a self-supervised framework that learns compact basin representations from hydroclimatic and physiographic data without...

    arxiv.org/abs/2611.05164 · PDF

  24. 24

    Macrocanonical Generator Networks: Data-Efficient Neural Surrogates for Amortized Physics Simulation

    Niall Jeffrey, Benjamin D. Wandelt

    cs.LG · astro-ph.CO · stat.ML

    Many scientific simulations are computationally expensive, limiting their use in simulation-based inference, uncertainty quantification, and decision making. We introduce macrocanonical generator networks, a framework for learning fast, data-efficient neural surrogates for amortized simulation of stationary physical processes by matching multiscale statistics of a target process. Inspired by microcanonical maximum-entropy synthesis, which...

    arxiv.org/abs/2611.05157 · PDF

  25. 25

    Beyond Foundation Models: PFN Priors Improve Tabular Hyperparameter Optimization

    Artem Chistyakov, Renat Sergazinov, Artem Babenko

    cs.LG

    Synthetic priors have so far served one purpose: pretraining tabular foundation models. We ask whether the same priors can improve conventional tabular models, which are trained from scratch on each dataset. We propose PriorTune, a method that meta-tunes hyperparameter configurations on synthetic tasks alone. Offline, for a given model, we fit a pool of configurations on datasets sampled from a published PFN prior and greedily select the...

    arxiv.org/abs/2611.05141 · PDF

  26. 26

    PAC Learning with Multiple Correct Answers

    Steve Hanneke, Farnam Mansouri, Alireza F. Pour, Amirreza Shaeiri, Kun Wang

    cs.LG

    We study PAC learning when each example comes with a set of at most $\ell$ correct labels and a prediction is correct if it belongs to this set. Training may reveal the full set, only whether a proposed label is correct (bandit feedback), or one correct label chosen by a demonstrator (emonstrations). Our main finding is that demonstrations can be much harder than full sets, and even than bandit feedback, both in which classes can be learned...

    arxiv.org/abs/2611.05133 · PDF

  27. 27

    Do Ray-Traced Cities Make a Better Channel Prior for an Unseen Site? A Pre-Registered Study in Simulation, and a Test on Measured Channels that Reverses It

    Krish Patel

    cs.LG · cs.IT · eess.SP

    Channel estimators at a newly deployed base station usually start from 3GPP stochastic channel models, which describe no particular street. We ask whether a channel prior pooled from many ray-traced cities serves an unseen site better than the 3GPP CDL models, and whether learned estimators make the difference. With Sionna RT on 14 city scenes, LMMSE estimation from covariance priors, leave-one-city-out evaluation, and pass/fail bars written...

    arxiv.org/abs/2611.05128 · PDF

  28. 28

    Towards an Optimal Control Foundation of Early Exits - Turnpikes and ResNets

    Jens Püttschneider, Simon Heilig, Asja Fischer, Timm Faulwasser

    cs.LG · eess.SY

    The feed-forward propagation through neural networks is crucial for training and inference. To trade off efficiency and accuracy, it has been proposed to consider intermediate outputs of hidden states as data-dependent regularization during training and, ultimately, as early-exits during inference. In this work, we analyze the joint optimization of loss functions computed for the output layer and intermediate layers using optimal control...

    arxiv.org/abs/2611.05120 · PDF

  29. 29

    Beyond Multi-Index Models: Learning Intersections of Degree-2 PTFs

    Ilias Diakonikolas, Daniel M. Kane, Mingchen Ma, Lisheng Ren, Trung Tran

    cs.LG · cs.DS · math.ST · stat.ML

    We give an algorithm for PAC learning intersections of $k$ degree-2 polynomial threshold functions over $R^d$ under a Gaussian distribution. Our algorithm achieves classification error $ε$ using $d^{O(k)}2^{O_k(\log^3(1/ε))}$ time and samples, improving over the $d^{O(k^2/ε^2)}$ running time of the polynomial regression approach. To our knowledge, our result is the first to obtain a dimension exponent independent of the accuracy parameter,...

    arxiv.org/abs/2611.05114 · PDF

  30. 30

    Molecular Prior Networks

    Pawel Gomoluch, Anthony Bradley

    cs.LG · cs.AI · cs.CE

    Predicting affinity between small molecules and their protein targets remains one of the central problems in drug discovery. Due to the substantial cost of wet-lab experiments, machine learning models of drug-target affinity are usually trained on relatively small datasets, ranging from tens to hundreds of molecules. Given the inherent difficulty of generalization between different targets and chemical series, existing datasets of...

    arxiv.org/abs/2611.05103 · PDF

  31. 31

    Dual-VaDE: Efficient and Robust Prototype Learning via Variational Dual-Domain Mixture Priors

    Wugeng Zheng, Song Wang, Jian Wang

    cs.LG

    In this paper, we propose Dual-VaDE, an unsupervised generative framework for learning interpretable anatomical prototypes from unlabeled medical images. In contrast to previous variational methods that rely on a single spatial latent code, which often entangles global geometry with local textures and fails to capture their cross-domain dependencies, Dual-VaDE introduces a dual-domain Gaussian mixture prior that explicitly couples spatial...

    arxiv.org/abs/2611.05099 · PDF

  32. 32

    Effects of Parton Pair Correlations on Direct-Photon Yields and Elliptic Flow in Heavy-Ion Collisions

    Xingjian Lu, Shuzhe Shi

    cs.LG

    Direct photons escape heavy-ion collisions with little rescattering, carrying information from the full emission history. Yet describing the measured direct-photon yield and the sizable elliptic flow $v_2$ together remains a long-standing challenge. Here we explore how correlations between emitting partons may affect both observables. The local $2\to2$ photon rate depends on the joint distribution of incoming partons, so a connected...

    arxiv.org/abs/2611.05042 · PDF

  33. 33

    Positional Encoding is All You Need for Scalable Equivariance Constraint Relaxation

    Hagay Michaeli, Haggai Maron, Daniel Soudry

    cs.LG

    Equivariant neural networks incorporate known symmetries of the data, such as permutations, rotations, or translations, directly into their architecture, often improving sample efficiency and generalization. However, strict equivariance can make optimization difficult, for example, when the data only approximately satisfy the assumed symmetry, or when the equivariance constraints induce a challenging loss landscape. Recent studies have shown...

    arxiv.org/abs/2611.05041 · PDF

  34. 34

    Federated Transfer Learning for Medical Image Classification under Mixed Skew

    Alexandre G. Leclercq, Sébastien Bougleux, Aurélien Corroyer-Dulmont, Romain Hérault

    cs.LG · cs.CV

    Deep learning in medical imaging is constrained by data scarcity and by privacy regulations that make centralizing patient data across institutions impractical. Federated Learning (FL) addresses these constraints by keeping data local, but its distributed nature exposes training to strong statistical heterogeneity. Multicentric medical data is particularly affected, exhibiting a mixed skew that combines feature, label, concept, and quantity...

    arxiv.org/abs/2611.05035 · PDF

  35. 35

    DRAP: Dynamics-Risk Adaptive Plasticity for Continual Learning in Physical AI

    Oswaldo Ludwig

    cs.LG · cs.RO

    Continual learning in physical AI differs from conventional continual learning because small per-step policy errors can accumulate over a trajectory. Each deviation may move the agent outside the training distribution, increase subsequent prediction errors, and ultimately cause closed-loop task failure. This work introduces Dynamics-Risk Adaptive Plasticity (DRAP), a replay-free method that estimates the historical importance of adaptable...

    arxiv.org/abs/2611.05029 · PDF

  36. 36

    Finite Spectral Depth Controls Gaussian Entrance Rank and Avoidance of the First Gradient-Flow Saddle

    Yanlin Long

    cs.LG · math.OC

    Matrix factorization provides a simple model for studying the training dynamics of neural networks. Recent studies of gradient flow under small initialization have analyzed how trajectories escape from a neighborhood of the origin and approach their first nonzero stationary point. In this paper, we study continuous-time dynamics that applies a finite number of Newton-Schulz iterations to the normalized matrix gradient (Muon). We show that the...

    arxiv.org/abs/2611.05022 · PDF

  37. 37

    Rethinking GA2Ms: performance and out-of-distribution generalisation

    George Dimitriadis, Damian Machlanski, Paul Clarke, Spyridon Samothrakis

    cs.LG · cs.AI

    Generalised Additive Models (GAMs) are a cornerstone of interpretable machine learning, offering transparent predictions through structured additive decompositions. Despite advances from classical GAMs to modern neural and spline-based variants, two notable gaps remain. First, comprehensive benchmarking of spline-based GAMs on large-scale tabular learning tasks, the dominant modality in applied settings, is lacking. Second, their robustness...

    arxiv.org/abs/2611.05012 · PDF

  38. 38

    Unifying and Beyond the Theory of Pervasive Capability Degradation in LLM Post-training SFT from Feature Interference

    Hang Chen, Jiaying Zhu, Wenya Wang

    cs.LG

    Pervasive Capability Degradation (PCD) refers to the phenomenon in which supervised fine-tuning (SFT), while adapting a large language model (LLM) to a target task, often erodes the general capabilities acquired during pre-training. Prevailing explanations attribute PCD to macroscopic distribution shift and therefore lack microscopic manipulability, leaving existing mitigation methods without a complete theoretical foundation. In this paper,...

    arxiv.org/abs/2611.05003 · PDF

  39. 39

    Auditing Multi-Proxy Explanations of Web User Preferences: Identification Boundaries and Abstention

    Yu Zhang, Zhihan Wang, Yuhang Lu, Shuai Li

    cs.LG · cs.IR

    Web systems infer user preferences from behavior, ratings, text, and LLM-generated profiles, yet explaining a proxy may reveal its measurement process rather than the preference of interest. We develop an audit framework distinguishing point identification, partial identification, and abstention relative to an explicit reference distribution. Under feature-dependent, arbitrarily signed proxy bias, a strict majority of uncontaminated proxy...

    arxiv.org/abs/2611.04997 · PDF

  40. 40

    Exact Selective Inference for Spectral Clustering via Piecewise-Constant Graph Perturbation

    Naotoshi Nakamura

    cs.LG · stat.ML

    Cluster labels are often used twice: first to discover groups, and then to test whether the discovered groups differ. This double use of the data makes ordinary two-sample $p$-values anti-conservative. Selective inference fixes this by conditioning on the clustering event, but for spectral clustering the event appears difficult because the algorithm combines an affinity map, an eigendecomposition, and a rounding step. We show that, for...

    arxiv.org/abs/2611.04995 · PDF

  41. 41

    ZEMA: confidence-calibrated signed descent with only one state

    Ali Dadsetan, Domenic Rosati, Frank Rudzicz

    cs.LG

    Recent work characterizes the Adam optimizer as sign descent modulated by gradient uncertainty: the update is the sign of the batch gradient, scaled by how strongly the batch supports that sign. We apply this view directly and introduce ZEMA (Z-score Evidence Momentum Adaptation), which measures that support with a per-coordinate $z$-score computed from the per-example gradients in the current batch, converts it into a sign probability, and...

    arxiv.org/abs/2611.04987 · PDF

  42. 42

    Deep Neural Operator Learning for stochastic heat equations and applications

    Hengrong Du, Qi Feng, Weinan Wang

    cs.LG · math.AP · math.PR

    In this paper, we establish a universal approximation theorem for solution operators of semilinear stochastic heat equations in spatial dimension \(d\ge2\), extending deep neural operator approximation results to SPDEs with distribution-valued Gaussian forcing. A smooth Fourier cutoff reduces the forcing to finitely many measurements, while Besov and Schauder estimates control the resulting solution error. The finite-dimensional solution map...

    arxiv.org/abs/2611.04955 · PDF

  43. 43

    Modulating Merging Strengths via Joint Loss Estimation for LoRA-based Continual Learning

    Kun Gu, De Cheng, Zhipeng Xu, Lingfeng He, Di Xu, Nannan Wang

    cs.LG

    Continual learning (CL) aims to adapt models to new tasks while preserving previously learned knowledge. Recently, Low-Rank Adaptation (LoRA), a representative Parameter-Efficient Fine-Tuning (PEFT) method, has gained increasing attention in CL due to its efficiency and scalability. Several LoRA-based merging methods maintain a single inference model by scaling the newly learned LoRA update with one coefficient and merging it into the...

    arxiv.org/abs/2611.04931 · PDF

  44. 44

    Cool Graphs: Active Property Search Towards Quantum Nano-Refrigerators

    Sofia Blyufer, Amir Kleiner, Uri Peskin

    cs.LG

    Nanodevices that absorb energy from their thermal environment can act as quantum refrigerators. Finding suitable configurations, however, is a needle-in-a-haystack problem: the many-body state space scales exponentially with the size of the device, where each candidate configuration requires an expensive quantum-chemical computation. We introduce a search framework that couples a Graph Variational Autoencoder (GVAE) with an uncertainty-driven...

    arxiv.org/abs/2611.04907 · PDF

  45. 45

    Attending to Memory Writes Reduces Seed Variability in Recurrent Length Generalization

    Sungwoo Goo, Hwi-yeol Yun, Sangkeun Jung

    cs.LG

    Recurrent memory requires learning useful updates, but attention added to assist learning can also compromise length generalization. We study bounded delta attention: causal attention over uncorrected update proposals whose bounded correction enters the recurrent update. Its implementation in TraceRelay combines additive phase recurrence with attention computed before a parallel scan. On prefix-supervised modulo-5, trained at lengths 1--40,...

    arxiv.org/abs/2611.04902 · PDF

  46. 46

    Fault-tolerant shallow recurrent decoders for sparse sensing with missing sensor measurements

    Matteo Romilio Pasqual, Matteo Tomasetto, J. Nathan Kutz, Andrea Manzoni

    cs.LG · cs.AI

    Real-world sensing systems are inherently imperfect: sensors may fail, intermittently disconnect, operate with asynchronous sampling rates, and be subject to noise, leading to measurement streams that are irregular, incomplete, and heterogeneous. Nevertheless, sensing strategies for real-time state reconstruction typically assume synchronous and continuously active sensors, a constraint that substantially limits their applicability in...

    arxiv.org/abs/2611.04893 · PDF

  47. 47

    DLER: Dual-Logit Experience Replay for Online Continual Learning

    Huiwei Lin, Shanshan Feng, Kong Aik Lee

    cs.LG

    Online continual learning aims to continuously train neural networks on a non-stationary data stream in a single pass. Rehearsal-based methods alleviate catastrophic forgetting by replaying stored samples, but the imbalance between current and previous data often leads to predictions biased toward current classes. Many approaches have been proposed to overcome the forgetting problem by correcting the bias; however, they still need to be...

    arxiv.org/abs/2611.04885 · PDF

  48. 48

    Found by Search, Broken by a Shortcut: Transfer and Tampering of Searched Intrinsic Rewards

    Chase J. Metoyer

    cs.LG · cs.AI

    Can a search over innate drives find intrinsic rewards that help a reinforcement-learning agent learn, transfer to a task it never saw, and resist an exploit that was absent during the search? Each candidate reward combines four generic events (a new view, a pickup, a drop and a toggle), each with a weight, a relief and a refill rate. We ran three pre-registered stages; the search protocol was locked in a public, timestamped commit before any...

    arxiv.org/abs/2611.04838 · PDF

  49. 49

    Anomaly Detection on Resource-Constrained Devices with Reservoir Computing

    Marco Angioli, Mathias Persson, Hiroto Tamura, Hadi Banaee, Amy Loutfi, Gouhei Tanaka, Alberto Giaretta, Denis Kleyko

    cs.LG

    Anomaly detection at the edge requires the deployment of models that achieve competitive performance, while minimizing computational cost and memory footprint. In this article, we investigate the suitability of the Mahalanobis Distance of Reservoir States (MDRS), an unsupervised time-series anomaly detection method, for resource-constrained devices and propose two optimizations to reduce its computational requirements. Specifically, we...

    arxiv.org/abs/2611.04835 · PDF

  50. 50

    Sleep-Disordered Breathing Detection with Quantum Recurrent Neural Networks

    Alejandro Mayorga-Redondo, José Daniel Viqueira, Diego Alvarez-Estevez, Eduardo Mosqueira-Rey, Andrés Gómez

    cs.LG · quant-ph

    This work investigates the application of Quantum Recurrent Neural Networks to the binary classification of sleep-disordered breathing events in time-series data, aiming to distinguish between apneic and normal breathing episodes. A variational classical--quantum architecture is proposed and evaluated against classical Recurrent Neural Network models. Experiments were conducted on multiple real-world datasets with varying levels of complexity...

    arxiv.org/abs/2611.04818 · PDF

  51. 51

    On the Complexity of Finite-Sum Nonconvex-Strongly-Concave Minimax Optimization with Exact Proximal Oracles

    Yuxing Peng, Zhiqing Tang, Weijia Jia

    cs.LG

    Finite-sum nonconvex-strongly-concave minimax problems arise in adversarial training and distributionally robust learning. Their known lower bounds cover only linear-span algorithms, zero-respecting algorithms, or a restricted proximal protocol. They leave open the power of exact component proximal steps, whose responses are global saddle points of regularized component problems and can expose directions that gradients do not reveal. We show...

    arxiv.org/abs/2611.04806 · PDF

  52. 52

    Training Specialist LLMs on Your Phone: Forward-Only Optimization via Block-Spectral Coordinates

    Liyan Tan, Yequan Zhao, Ziyue Liu, Yupeng Su, Ruijie Zhang, Zheng Zhang

    cs.LG · eess.SY

    Large language models (LLMs) on personal devices keep user data local, but limited memory restricts their general capability. Since most user-specific tasks do not require the breadth of frontier models, a practical solution is to adapt a shared on-device model into lightweight specialists. However, backpropagation is prohibitive in memory, and users rarely provide labeled data. We present MobileSpec, a system that trains specialists with...

    arxiv.org/abs/2611.04797 · PDF

  53. 53

    Density-Aware Reward Scaling: A Reinforcement Learning Framework for Reward-Proportional Sampling

    Ria Shekhawat, Abhijeet Sinha, Yingtao Zhu, Dianbo Liu

    cs.LG · cs.AI

    Reinforcement learning objectives that maximize expected return can concentrate probability mass on a small subset of high-reward outcomes, limiting diversity when multiple distinct solutions are desirable. We introduce Density-Aware Reward Scaling (DARS), a simple reward-processing mechanism that promotes reward-proportional sampling in continuous outcome spaces while retaining standard policy-gradient training. DARS estimates the local...

    arxiv.org/abs/2611.04792 · PDF

  54. 54

    Cooperative Multi-Agent Reinforcement Learning with Submodular Trajectory Rewards

    Yasaman Zolfimoselo, Nikolas Geroliminis, Maryam Kamgarpour

    cs.LG · cs.MA

    We study cooperative multi-agent reinforcement learning with decoupled dynamics and a monotone submodular reward function over joint trajectories. Due to the non-additive reward, an optimal policy may depend on the history of all agents. This dependence makes exact optimization exponential in the horizon and the number of agents. To overcome this challenge while retaining theoretical guarantees, we introduce pre-window curvature $c_m$, which...

    arxiv.org/abs/2611.04774 · PDF

  55. 55

    Koopman Generative Operators for Efficient Probabilistic Time-Series Forecasting

    Raz Marshanski, Liran Nochumsohn, Mayank Jauhari, Boris N. Oreshkin, Omri Azencot

    cs.LG

    Probabilistic time-series forecasting requires models that simultaneously capture structured temporal dynamics, expressive uncertainty, and efficient inference, yet existing approaches fall short of this goal: latent dynamical models impose structure but rely on restrictive generation mechanisms, while modern generative methods such as diffusion and flow matching achieve flexibility at the cost of iterative and computationally expensive...

    arxiv.org/abs/2611.04760 · PDF

  56. 56

    Non-Interfering Asymmetric AdamW Updates for Constrained Exploration of Equifinality in Hybrid Models

    Ekaterina Gikalo, Zavud Baghirov, Christian Reimers, Markus Reichstein, Martin Jung, Marco Körner

    cs.LG

    Hybrid models let neural networks supply the unresolved parameters of a process-based simulator, and are fitted to observations that constrain aggregate fluxes and storages but not the internal partitioning that produces them. Such a fit is equifinal: configurations indistinguishable in the observed aggregates can disagree substantially about the unobserved internals, and therefore about the mechanism they attribute to the same data. We study...

    arxiv.org/abs/2611.04756 · PDF

  57. 57

    What Does It Mean to Remove an Edge? Intervention-Aware Fidelity for Graph Neural Network Explanations

    Rocco Mattia Di Mauro, Marco Grassia, Giuseppe Mangioni

    cs.LG

    Graph neural network explanations are often evaluated via deletion fidelity, i.e., by deleting their top-ranked edges and measuring the resulting change in model output. Yet, in a message-passing GNN, "deleting an edge" does not specify a unique intervention. Structural deletion removes the edge from the graph and recomputes quantities such as degree normalization and attention, whereas message suppression leaves this structural context...

    arxiv.org/abs/2611.04751 · PDF

  58. 58

    TAPES: Task-Aligned Joint Predictive Pretraining for Transaction Event Sequences

    Ruobing Zuo, Weiyu Han, Huilin He, Haoyu Jiang, Ze Zhao, Jie Sheng, Xiao Tan, Shuai Chen, Tianyi Zhang

    cs.LG · cs.AI

    Transaction event sequences support diverse prediction tasks in online commerce and digital financial services. The same transaction records can yield prediction targets with different semantics, including values, changes, and relations to past events. These targets impose different constraints on history representations, but their predictability alone does not establish the downstream utility of a target set. Moreover, target-wise prediction...

    arxiv.org/abs/2611.04747 · PDF

  59. 59

    Surrogate Models for Stochastic Al-Cu Phase-Field Solidification

    Simon Savukoski, Tatu Pinomaa

    cs.LG · cond-mat.mtrl-sci

    Phase-field models resolve microstructural evolution during rapid alloy solidification, but their computational cost limits long trajectories, stochastic ensembles, and studies across process conditions. We present a variance-preserving denoising diffusion bridge model (VP-DDBM) for state-conditioned prediction of two-dimensional Al-Cu solidification. Given the current phase, concentration, and prescribed temperature fields, the model...

    arxiv.org/abs/2611.04695 · PDF

  60. 60

    Uncertainty-Aware Prototype Learning for Open-Set Test-Time Adaptation

    Jinwu Hu, Langyu Bian, Jian Sun, Dongjin Yang, Chao Shuai, Yufeng Wang, Kai Zhou, Bin Xiao, Mingkui Tan, Hongmin Cai, Wei Luo

    cs.LG · cs.AI

    Open-set test-time adaptation (TTA) aims to adapt source-trained models to unlabeled test streams where covariate-shifted in-distribution (csID) samples and covariate-shifted out-of-distribution (csOOD) samples may coexist. Existing TTA methods are typically designed under a closed-set assumption and therefore lack explicit mechanisms for reliably distinguishing csID from csOOD samples. Consequently, csOOD samples may be mistakenly included...

    arxiv.org/abs/2611.04689 · PDF

  61. 61

    Sheaf-MPNN: A Library for Sheaf Neural Networks as Message Passing

    Alessio Borgi, Mario Severino, Luke Braithwaite, Steve Azzolin, Rémi Bourgerie, Olga Zaghen, Emanuele Mule, Riccardo...

    cs.LG

    Standard Graph Neural Networks (GNNs) implicitly assume that neighboring nodes share the same feature space, which contributes to oversmoothing and poor performance on heterophilic graphs. Sheaf Neural Networks (SNNs) remove this assumption by giving each node and edge its own vector space and learning restriction maps that transform features as they move between neighbors. These maps define a sheaf Laplacian that drives diffusion over the...

    arxiv.org/abs/2611.04670 · PDF

  62. 62

    FedCSGA: Client Selection Based on Genetic Algorithm for Federated Learning

    Ghani Haider, Majid Kundroo, Leo Zhang, Jinchul Choi, Taehong Kim

    cs.LG · cs.DC

    Federated Learning (FL) enables collaborative training of machine learning models across distributed clients while preserving data privacy. In practical deployments, FL relies on selecting a subset of clients in each communication round to reduce communication cost, mitigate stragglers, and improve training efficiency under heterogeneous data and system conditions. In heterogeneous edge computing environments, both statistical heterogeneity...

    arxiv.org/abs/2611.04667 · PDF

  63. 63

    Feature-Based and Deep Learning Methods for Astronomical Light Curve Classification

    V. Baymakanov, Kh. Khamitov, A. Raimova, A. Isaeva, R. Taran, M. Zedan, M. Kornilov, M. Demianenko, K. Malanchev, M. Hushchyn

    cs.LG · astro-ph.IM

    We compare feature-based machine learning and end-to-end deep learning for variable star classification using the Zwicky Transient Facility (ZTF) StarEmbed dataset, which contains about 40,000 objects across seven imbalanced classes. Classical models trained on physically motivated features consistently outperform deep networks trained directly on raw light curves. We find that limited labeled data contributes to the performance gap, while...

    arxiv.org/abs/2611.04664 · PDF

  64. 64

    Best-of-Depth: Learning Exploratory Dynamics in Looped Transformers

    Shota Takashiro, Takeshi Kojima, Masanori Koyama, Kohei Hayashi, Yusuke Iwasawa

    cs.LG · cs.AI

    Looped Transformers extend inference-time computation through repeated latent-state updates, but final-depth training does not directly encourage useful intermediate alternatives. We introduce Best-of-Depth (BoD), which trains a recurrent trajectory as a candidate set by minimizing prediction loss across depths. Different valid targets can be matched at different depths without architectural changes or depth-specific labels. An exact-match...

    arxiv.org/abs/2611.04663 · PDF

  65. 65

    Leakage-free evaluation of residual graph convolutional networks for network intrusion detection

    Zahid Iqbal

    cs.LG

    Graph neural networks are increasingly proposed for network intrusion detection, yet many reported results come from protocols in which test flows shape the graph used for training, duplicated flows sit on both sides of the split, or a single split is reported. This study measures what a residual graph convolutional network gains over simpler models once these sources of optimism are removed. CICIDS2017 and NF-UNSW-NB15-v2 were cleaned with...

    arxiv.org/abs/2611.04660 · PDF

  66. 66

    Exploiting Biases in Attention for Transformers

    Charles Herr, Gabriele Spadaro, Enzo Tartaglione

    cs.LG

    Attention is known to be sparse in vision transformers. In particular, we analyze the unequal relevance of attention-matrix columns to the final computation and show that this phenomenon becomes increasingly pronounced with network depth. This can be interpreted as saying that tokens have an absolute importance for the attention process, independently of queries. Inspired by this, we propose a reformulation of the attention formula that makes...

    arxiv.org/abs/2611.04657 · PDF

  67. 67

    Does LinUCB Need Valid Confidence Sets?

    Ahmed Ben Yahmed, Marc Abeille, David Janz, Clément Calauzènes

    cs.LG · cs.AI

    We show that replacing LinUCB's usual confidence width $β$ by any fixed $α>0$ preserves sublinear regret when the action set is $M$-smooth and $m$-strongly convex. Our high-probability regret bound for this generalisation, $α$-LinUCB, establishes that valid confidence sets are not necessary for the success of LinUCB. We identify the mechanism behind this result: action-set curvature drives $α$-LinUCB to explore every direction, even when...

    arxiv.org/abs/2611.04641 · PDF

  68. 68

    Self-Privileged Fine-Tuning

    Yangrong Hu, Xueyu Zhou, Yue Wu, Jian Huang

    cs.LG · cs.CL

    Supervised fine-tuning (SFT) is a key component of language model post-training, enabling effective task adaptation and serving as an important foundation for subsequent reinforcement learning. Recent methods seek to improve SFT by introducing token-level weights into the training objective, where these weights are typically derived from a model's output distribution at each token position. To better understand how such weighting schemes...

    arxiv.org/abs/2611.04632 · PDF

  69. 69

    Policy Network Diffusion

    Jiate Liu, Deheng Ye

    cs.LG

    Diffusion models have shown remarkable progress in AI-generated content, including images and videos. In this paper, we explore whether diffusion models can generate the parameters of policy networks in reinforcement learning (RL), and whether the generated policies exhibit generalization capabilities. To achieve this, we propose Diffusion-based Conditional Policy Network Parameter Generation (DPPG), which consists of a Transformer-based...

    arxiv.org/abs/2611.04614 · PDF

  70. 70

    CAPSkip: Cancellation-Aware Prefix Skipping for Selective Backpropagation

    Heejin Choi

    cs.LG

    Per-step selective back-propagation forwards every example of a mini-batch but back-propagates only some of them. Most rules rank examples by loss or gradient magnitude, but such scalar scores ignore cancellation between per-example gradients: the gradient removed by a set can be much smaller than its individual magnitudes suggest. We introduce CAPSkip (Cancellation-Aware Prefix Skipping). Among the examples the model already classifies...

    arxiv.org/abs/2611.04611 · PDF

  71. 71

    SCANO: Restoring Boundary-Condition Completeness in Sampled Meshes with a Sliced-Cross-Attention Neural Operator for 3D Thermal Fields

    Tianyu Li, Zhiwei Cao, Qingang Zhang, Ruihang Wang, Binyang Song, Yonggang Wen

    cs.LG · physics.comp-ph

    Training neural operators on million-cell meshes can exceed GPU memory limits. Models are therefore trained on subsamples and deployed on whole meshes or selected regions. However, subsampling can distort boundary information, degrading accuracy. We ask when boundary conditioning under sampling stays consistent with whole-mesh training, and how to aggregate and inject boundary data. Representing the solution as an integral of boundary data...

    arxiv.org/abs/2611.04603 · PDF

  72. 72

    Ground-Wing: A Parametric 3D Wing-in-Ground-Effect Dataset for Benchmarking Neural Surrogate Models

    Andrea Cenzato, Lukas Karch, Andre Luckow, Yoran Pichon, Francesco Pasqua, Abouzar Ghasemi, Rishikesh Ranade, Ira...

    cs.LG

    This paper presents a CFD dataset of a highly parametrized 3D wing operating in ground effect, designed for benchmarking neural surrogate models for aerodynamic prediction, using a wing shape and flow conditions typically encountered in motorsport applications. The dataset comprises 770 RANS simulations of 194 unique wing configurations, sampled across 27 airfoil profile variants and 4 distances from the ground, spanning variations in wing...

    arxiv.org/abs/2611.04571 · PDF

  73. 73

    Curriculum unlearning in humans and neural networks

    Jirko Rubruck, Alice Zhang, Andrew Saxe, Christopher Summerfield

    cs.LG · q-bio.NC

    During learning, humans often benefit from curricula that teach simple rules before exceptions. However, we lack a computational explanation for this phenomenon. Here, using a canonical behavioural task and modelling with neural networks, we develop and test a theory for why simple-to-complex curricula succeed. We show that during learning, both humans and neural networks show simplicity biases, a tendency to learn simple rules that are...

    arxiv.org/abs/2611.04565 · PDF

  74. 74

    CANDO: Conservative Action-Noise Directional Optimization for Residual Reinforcement Learning

    Zeyu Gao, Chengkai Hou, Chaoyue Li, Zhuangzhe Wu, Jiaming Liu, Yao Mu, Peng Jia, Li Zhang, Shanghang Zhang

    cs.LG · cs.AI

    Pretrained generative policies provide strong behavior priors for robot control, but adapting them efficiently with online reinforcement learning remains difficult. A central challenge is how to improve a frozen generative policy without destroying the structure learned from offline demonstrations. Noise-space residual methods address this by modifying the policy's input noise, but existing noise-space methods typically regulate adaptation...

    arxiv.org/abs/2611.04543 · PDF

  75. 75

    The Price of List Replicability: Tight Bounds for Bias Estimation

    Michael Chen, Atieh Bastan, A. Pavan, N. V. Vinodchandran

    cs.LG

    We study the sample complexity of list replicable algorithms for estimating the biases of $d$ coins. An algorithm is $(k,\varepsilon,δ)$-list replicable if for every bias vector $\mathbf{p}\in[0,1]^d$, there is a set of outputs $L_{\mathbf{p}}\subseteq \mathbb{R}^d$ of at most $k$ elements such that every $\mathbf{y}\in L_{\mathbf{p}}$ satisfies $\lVert\mathbf{y}-\mathbf{p}\rVert_\infty\leq\varepsilon$, and the algorithm output is in...

    arxiv.org/abs/2611.04532 · PDF

This edition is part of The Daily Abstract — cs.LG archive. Subscribe to receive these in your inbox each morning, automatically translated to Spanish, with reply-to-PDF: arxivdaily.ignorelist.com.

Colophon Set in Georgia, with system sans for interface chrome and a monospaced stack for code and paper identifiers. Sole accent: amber #D99C5E. Built and served on an always-free VM. The masthead is set 14% letterspaced because newspapers do that and it works.