cs.LG · 2026-10-14 · No. 143
Machine Learning, 2026-10-14.
53 new papers in cs.LG. Titles, authors,
abstracts. Links to arXiv. Want this in your inbox every morning? Subscribe →
01 — The papers
53 entries-
01
Exact Topological Compliance: Generating Persistence-Equivalent Graphs
Mattie Ji, Indradyumna Roy, Vikas Garg
cs.LG · math.AT
Persistent homology summarizes the topological evolution of a graph as persistence diagrams (PDs). Generating graphs whose topology matches a target descriptor is increasingly important, yet existing topology-aware generative models rely on soft regularization, producing graphs whose PDs may only be approximately similar. We initiate a principled investigation of the generation problem: produce graphs that realize a target PD exactly. We...
-
02
Two Clocks: Why Larger Rollout Batches Learn Slower but Fail on Schedule in Reinforcement Learning with Verifiable Rewards
Yuante Li, Junwei Quan, Kai Qiu, Kate Zhang, Yicheng Tao
cs.LG · cs.AI
Reinforcement learning with verifiable rewards (RLVR) relies on costly model-generated responses, motivating larger rollout batches and square-root learning-rate scaling. Reported to approximately preserve progress per sample with one optimizer step per rollout, this rule is less understood in practical multi-step training. We develop a two-clock framework that separates the samples needed to make progress from those consumed before collapse....
-
03
Fast Rates for Offline Q-Learning with KL Regularization
Yize Hao, Guangxin Jiang, Shuang Qiu
cs.LG · cs.AI
This paper investigates offline Q-learning for finite-horizon Markov decision processes with Kullback-Leibler (KL) regularization. Recent studies have shown that KL regularization can yield fast rates in offline decision-making problems. However, it remains unclear whether Q-learning, which adopts incremental stochastic approximation updates, can achieve such fast rates in the offline setting. We first propose KL-regularized Q-Learning (KLQ),...
-
04
Scaling Online Optimal Transport Minibatches for Flow Matching
Peter Chen, Sergey Levine
cs.LG · cs.CV
A central challenge in scaling flow models is maintaining stability in training and integration. In flow matching, this stability depends critically on how source samples are paired with data. Independent random pairing introduces substantial stochasticity, motivating minibatch optimal transport (OT) to construct online source-target couplings that reduce this randomness (Tong et al., 2024). However, as applications extend to larger and...
-
05
Statistical Vignettes on the Grokking Phenomenon
Baye Malick Gning, Elvis Dohmatob
cs.LG
In this work, we study grokking in a simple, analytically tractable setting whose rich phenomenology allows us to disentangle the effects of initialization, signal strength, weight decay, sample size, and input dimension. These factors can produce superficially similar learning curves through distinct mechanisms, complicating their interpretation. Our analysis distinguishes early fitting through nuisance features, core feature learning, and...
-
06
Depth Memory
David González-Martínez, Philipp Nazari, T. Konstantin Rusch, Shiwei Liu
cs.LG
Residual connections propagate information through a network by additively accumulating layer outputs. Recently proposed alternatives, such as Manifold-Constrained Hyper-Connections (mHC) and Attention Residuals (AttnRes), instead use more expressive depth aggregation mechanisms that improve language modeling performance at scale. In this work, we consider the mixing of representations over depth as a problem of storing and retrieving...
-
07
SkillScaffold: Scaffolding Agentic Reinforcement Learning with Policy-Adaptive Skill Credit Assignment and Selective Injection
Wenlin Zhong, Hao Wang, Kun Yuan, Fan Yang, Jinluan Yang, Dian Jin, Xu Tan, Yiquan Wu, Kun Kuang
cs.LG · cs.CL
Agentic reinforcement learning (RL) enables large language models (LLMs) to learn from multi-turn interactions with verifiable outcomes. Textual skills can improve exploration by distilling prior interactions into reusable guidance. However, existing skill-based RL approaches often either assume skill injection is beneficial or rely on indirect feedback or rollout-intensive evaluation to assess its utility. Our pilot studies show that...
-
08
A Rate Transfer Method: Convergence Rates in Stochastic Gradient Descent via Stochastic Calculus
Neil Walton
cs.LG · cs.AI · math.OC · math.PR · stat.ML
We develop a rate-transfer method that converts finite-time Lyapunov estimates for an associated stochastic differential equation (SDE) into finite-time convergence bounds for stochastic gradient algorithms. Under generic moment and smoothness conditions, the convergence analysis of the stochastic gradient method reduces to a calculus argument verifying a Lyapunov generator inequality for the SDE. The method recovers established rates for...
-
09
Enabling Token-wise Importance Sampling in Off-Policy RLVR with Proximal KL Regularization
Hao Bai, Jiarui Yao, Tong Zhang
cs.LG
Off-policy data in reinforcement learning with verifiable rewards (RLVR) can arise from asynchronous generation, rollout-training mismatch, and response reuse. Token-wise importance weighting avoids multiplying likelihood ratios across a response but generally biases policy gradients. We prove that, under coverage and local expressiveness conditions, full future KL credit makes the exact proximal optimizer the update's unique stationary...
-
10
SP$^2$ec: Adaptive Self-Speculative Decoding for Vision-Language Models
Yuqi Huang, Xingyao Li, Yunlong Hou, Fengzhuo Zhang, Jiachun Pan, Vincent Y. F. Tan
cs.LG
Speculative Decoding (SD) has demonstrated its efficacy in accelerating large language model inference, but for Vision-Language Models (VLMs), the diverse input modalities make it difficult for a small trained drafter to achieve consistently high throughput. Self-speculative decoding reuses parts of the target model as a drafter which retains the multi-modal capability of the VLMs, yet its efficiency depends on choosing a suitable draft...
-
11
Learning When to Transfer: Reinforcement Learning for Carbon-Footprint Modeling in New Manufacturing Domains
Muhammad Umar Farooq, Daniel R. Cooper
cs.LG
Manufacturers often need to characterize a new material, process, or product before enough operating data are available to train reliable predictive models for this new, target domain. Existing manufacturing operations contain potentially useful knowledge to accelerate this characterization, but indiscriminate reuse of data from existing source domains can cause negative transfer in learning of the new, target domain when the manufacturing...
-
12
On the Monitorability of Natural Language Autoencoders: Identification vs Discrimination
Kevin Yandoka Denamganaï
cs.LG · cs.CL
Natural Language Autoencoders (NLAs) are proposed as oversight tools for language models: an Activation Verbaliser (AV) writes an English explanation of an activation vector, and an Activation Reconstructor (AR) maps the explanation back to a vector. Their value for oversight depends on a human monitor being able to tell a true account of an activation from a false one, given the Av's explanation. We formalise NLA training and NLA oversight...
-
13
Learning Source Structure: Bayesian Prediction with Transformers
Meir Feder, Ruediger Urbanke
cs.LG
A transformer trained on sequences whose source parameters are drawn afresh each time can learn only what the sequences share: the source family and its prior. The Bayesian predictor that knows both, which we call informed Bayes, is the unique minimizer of the training loss and is computable from the family alone; we use it as a quantitative, falsifiable reference. Where earlier work compared transformers with Bayes on sources whose contexts...
-
14
A Refresh Interval Does Not Fully Specify a Refresh Schedule: Learning-Rate-Dependent Placement in Kronecker-Preconditioned Language Model Training
Xinyu Wang, Zhiwei Zhai, Yu Gu, Zijun Yu, Sicheng Lyu, Jingrui Tian, Lei Ding, Ziyang Song, Xiao-Wen Chang
cs.LG
Kronecker-preconditioned optimizers recompute a cached curvature eigendecomposition only periodically, and implementations and adaptive methods control this through the refresh frequency. We show that frequency leaves part of a schedule free: at a fixed number of refreshes, when they occur changes held-out loss at the same compute. Training GPT-style models with the Mousse optimizer across model sizes, we move the middle refreshes of a...
-
15
ProAS: A Progress-Aware Advantage Shaping Framework for Multi-Turn Agentic Reinforcement Learning
Bowen Ding, Mingda Wan, Yuhan Chen, Jiyao Yuan, Qi Zhu, Futing Wang, Heyuan Deng, Fei Mi, Lifeng Shang, Tao Lin
cs.LG · cs.AI
Reinforcement learning (RL) has emerged as the primary paradigm for optimizing Large Language Model (LLM) agents, yet efficient critic-free algorithms (e.g., GRPO) suffer from a fundamental uniform credit assignment dilemma that broadcasts trajectory-level advantages equally to all actions. Our analysis reveals that this uniformity inherently reinforces structurally invalid actions in preferred trajectories and ignores the downstream...
-
16
Model-Based Online Decision Making via Generative Trajectory Planning
Haldun Balim, Yilun Du, Na Li
cs.LG · cs.AI · cs.RO · eess.SY
Online decision-making faces a tension between efficient inference and long-horizon reasoning. Value-based methods rely on short-horizon bootstrapping that limits the propagation of long-term information, while trajectory optimization methods plan explicitly but must solve a high-dimensional optimization problem at every decision step. We propose Generative Trajectory Planning (GTP), a model-based framework that addresses this tension by...
-
17
BridgeTwist: Twisting Schrödinger Bridges for Training-Free Conditional Sampling
Mengyu Li, Qianqian Qu, Jun S. Liu
cs.LG · stat.ML
Generative models are often required to serve many downstream conditional tasks. In the training-free setting, a pretrained unconditional model is reused as a prior and combined with a task-specific likelihood to sample from the corresponding posterior. While such training-free samplers have been developed for diffusion and flow models, their counterpart for Schrödinger bridge (SB) models remains unexplored, despite the growing appeal of SBs...
-
18
Bayesian Test-Time Inference of Task-Aligned Similarity from Weak Interactive Feedback
Weida Liang, Kenji Kawaguchi
cs.LG · cs.CL
Language model systems repeatedly make test-time decisions by comparing similarity: they retrieve exemplars, rank candidates, and revise responses. Yet similarity is not unique: lexical, semantic, structural, and stylistic resemblance can point to different candidates, and the task-aligned notion can change across instances. We introduce BASIS (Bayesian Axis Selection via Interactive Signals), a training-free Bayesian method that uses weak...
-
19
Kwai Summary Attention V2 Technical Report
OneRec Team, Chenglong Chu, Guohong Mu, Guowang Zhang, Hao Peng, Hui Wang, Jian Liang, Jiangxia Cao, Jiao Ou, Jiaxin...
cs.LG · cs.AI · cs.IR
Summary-token architectures such as Kwai Summary Attention (KSA) compress distant context into learnable Summary token per chunk, and hybrid variants interleave these layers with full attention to retain exact access to the original text. In such hybrid KSA-Full architecture, the interleaving inserted summary tokens are served as a streaming flow role across different layers. Although effective, however, it also introduce some bottlenecks:...
-
20
Does Pretraining Teach Chemical Language Models Chemistry?
Leif Sieben, Yoel Zimmermann
cs.LG · cs.CL · physics.chem-ph
Chemical language models (CLMs) are pretrained on large corpora of molecules with the aim of learning representations that transfer to diverse downstream tasks. Here, we systematically study the effects of pretraining using a series of such CLMs across 64 molecular property prediction datasets. We find that the downstream gains of pretraining in CLMs do not scale and performance saturates below gradient-boosting baselines. Across the regimes...
-
21
Tiny Recursive Message Passing
Tommaso Salvatori, Matteo Forasassi, Alexis Toumi, Giovanni de Felice
cs.LG · cs.AI
Message passing neural networks have shown remarkable performance in algorithmic and constraint reasoning, due to their ability to encode the structure of the task directly in the architecture. Recent work has shown that better results can be achieved using recursive transformers trained with deep supervision and truncated recursions, which make extremely deep loops stable to train, and equipped with test-time procedures such as discrete...
-
22
MultiAlloc: Budget-Aware Precision Allocation for KV Cache Quantization
Yangchengyu Zhou, Zhiping Wu, Runchang Jiang, Dongdong Ren, Youfang Han, Wei Zhao, Dongqi Cai, Wenbin Li
cs.LG
The Key-Value (KV) cache constitutes a dominant source of memory consumption during large language model (LLM) inference, particularly in long-context and complex reasoning scenarios. Existing KV cache quantization methods have explored different compression granularities, including layer-wise precision adjustment and token-level information preservation. In practical deployment, however, the key challenge is not merely compression, but how...
-
23
Geometric Calibration of Supervised Fine-Tuning
Jianzhu Bao, Jia-Chen Gu, Qingyuan Liu, Haozhen Zhang, Kai-Wei Chang, Nanyun Peng, Wenya Wang
cs.LG · cs.AI
Recent studies investigate the weight geometry through its spectral structure, which characterizes how the weights transform different input directions. Within this view, supervised fine-tuning (SFT) is shown to induce dense updates that perturb the spectral geometry of pre-SFT weights along their principal directions, potentially disrupting the model's prior knowledge. These observations motivate using the geometry of pre-SFT weights as a...
-
24
Deterministic Prediction as Global Velocity: Flow Matching for Probabilistic Time-Series Forecast
Jiajing Hu, Xu Han, Li-Ping Liu
cs.LG
Accurate uncertainty quantification is critical for many time series forecasting applications, and recent studies have increasingly adopted generative models to capture predictive uncertainty. Existing approaches often combine deterministic forecasting with generative modeling to improve predictive performance. However, their joint training is typically unstable because the deterministic model is embedded within the generative process,...
-
25
Does self-distillation improve reasoning?
Jason Ken Adhinarta, Raina Wu, Ashutosh Sharma, Avinash Balakrishnan, Rameswar Panda, Polina Golland, Yoon Kim,...
cs.LG · cs.AI · cs.CL
Self-distillation is an emerging approach that trains a model to imitate its own predictions when conditioned on privileged information, such as correct answers or environmental feedback. Recent methods have reported better sample efficiency and generalization than reinforcement learning (RL), but it remains unclear whether these gains extend beyond in-distribution evaluations and whether they stem from privileged distillation. In this paper,...
-
26
Fairness under Prediction Error in Predict-then-Optimize Pipelines
Hilda Adwubi Osei
cs.LG · cs.CY
Fairness constraints in predict-then-optimize pipelines are necessarily enforced on predicted outcomes $\hat{y}$, since true outcomes $y$ are unobserved at decision time. We study whether a demographic parity constraint enforced this way continues to hold against $y$, and how the resulting violation scales with prediction error. Using a budget-constrained allocation problem solved as a linear program, we show empirically that the true-outcome...
-
27
When More Objectives Make Identification Easier: The Role of Preferences and Feedback
Bo Xue, Ji Cheng, Yuanyu Wan, Shuang Qiu
cs.LG · cs.AI
Best-arm identification (BAI) aims to identify the optimal arm using as few samples as possible. While most existing work focuses on a single objective, many real-world applications involve multiple criteria, which motivates the study of BAI in multi-objective bandits. Intuitively, multi-objective identification seems harder, since the learner must identify optimal arms in a higher-dimensional reward space; however, the additional objectives...
-
28
Last-Iterate Convergence of Clipped SGD with Linear Decay
Wei Jiang, Haomin Bai, Zechao Li
cs.LG · math.OC
In this paper, we study the last-iterate convergence of clipped stochastic gradient descent under heavy-tailed noise. We assume that the stochastic gradients are unbiased and that the noise has a bounded central $p$-th moment, where $1<p\le2$. For smooth convex objectives, a linearly decreasing learning rate gives a high-probability error bound of order $O(LR^2\log(1/δ)/T+σR(\log(1/δ)/T)^{(p-1)/p})$. We further establish last-iterate...
-
29
Unbalanced W-Flow: Unbalanced Wasserstein Drifting under Finite-Batch Marginal Fluctuations
Jongwook Jeon, Mingue Park, Jiwon Choi, Jaihoon Kim, Yunhong Min, Minhyuk Sung
cs.LG · cs.CV
Teacher-free one-step generative models such as W-Flow learn directly from distributional objectives, with the Sinkhorn divergence providing the strongest empirical performance among the energies studied in W-Flow. However, Sinkhorn divergence is based on balanced optimal transport, whose exact marginal constraints can induce undesirable transport under finite minibatch fluctuations. We introduce UW-Flow, which replaces the balanced Sinkhorn...
-
30
Euclidean Distance-Based Flow Matching
Paul-Henrik Heilmann, Peter Ochs
cs.LG
Flow Matching has recently become popular as a technique for generative modeling with applications in many domains. In the standard Flow Matching formulation, we interpret the vector field as the solution of a least-squares regression problem, which is known to be sensitive to contamination. Therefore we leverage the viewpoint of robust statistics and replace the squared Euclidean distance with the Euclidean distance as a regression...
-
31
Two For The Price Of One: Curvature-Aware Pruning Preserves Influence Functions Too
Mehdi Makni, Natalia Ponomareva, Rahul Mazumder
cs.LG · stat.ML
Post-training pruning enables efficient deployment, but conventional methods do not explicitly preserve internal-gradient alignment between sparse and dense models. We study novel connections between downstream performance of compressed models and the ability to retain gradient information via influence-function preservation. Central to this connection is a curvature-aware objective that accounts for both layer activations and backpropagated...
-
32
Approximating Optimal Policies in Heterogeneous Asynchronous Multi-player Bandits
Sacha Arroues-Paykin, Etienne Boursier, Vianney Perchet
cs.LG
Motivated by applications to cognitive radio networks, multi-player multi-armed bandits (MMAB) have attracted considerable attention in recent years. The asynchronous variant, in which players are active only at a fraction of timesteps, better reflects real-world decentralized networks, where devices are not permanently active. Despite its practical relevance, this setting remains relatively understudied. Existing work on asynchronous MMAB...
-
33
Recalibrating Many Questions with Few Labels: Minimax Rates and Identification Limits
Xiaoyu Li, Chiwun Yang
cs.LG · stat.ML
Many related prediction problems, which we call questions, come with few labels each. What information can be shared across them, and what is actually necessary for good decisions? We answer both for recalibration, motivated by decision models such as Jev (TypeSafe AI, 2026) that return a probability for any yes/no question a developer writes. A predictor reports a probability $P$ with output law $μ$ on $m$ questions, each with $n$ labels and...
-
34
SoK: Rethinking Membership Inference in Tabular Machine Learning
Cristina Pêra, Tânia Carvalho, Maxime Cordy, Luís Antunes
cs.LG
Membership Inference Attacks (MIAs) are currently the dominant approach to evaluating privacy in machine learning applications. Despite their significance, several concerns remain unexplored, particularly with regard to tabular data. Existing evaluations focus predominantly on high-dimensional datasets, leaving the privacy behavior of small-to-medium datasets unexplored. In this paper, we present a new taxonomy for MIAs and conduct a...
-
35
ProTok: Unified Protein Tokenization via Modular Multi-Codebook Quantization
Romain Hardy, Xiaoman Zhang, Pranav Rajpurkar
cs.LG
Protein sequence, structure, and functional annotations each capture complementary aspects of protein biology, yet existing tokenizers encode each modality independently, limiting the expressiveness of discrete protein representations. We present ProTok, a unified protein tokenizer that maps all three modalities into a modular discrete vocabulary via product quantization with per-modality codebook groups and modality dropout. A single ProTok...
-
36
Graph Metanetworks for Transformer Based Language Models
Yonathan Wolanowsky, Guy Bar-Shalom, Fabrizio Frasca, Haggai Maron
cs.LG
Weight-space learning treats the weights of neural networks as data, processed by other models called metanetworks. A key challenge is that different weights can realize the same function, related by a group of symmetries the metanetwork should respect. While symmetry-aware metanetworks for simple architectures such as MLPs are well developed, a comprehensive treatment of transformers, and in particular transformer-based language models, is...
-
37
Readout Scaling Shapes Sparse Feature Recovery
Brain Xiao, Sarah Wilson
cs.LG · cs.AI · cs.CL
Readout scaling changes what a sparse autoencoder (SAE) learns, but the benefit of removing it depends on the data. We use readout-canonicalized decision-space (RC-DS) coordinates to remove singular-value gains while preserving retained decision information. At a fixed readout condition number, changing the generating distribution reverses which training coordinates recover known factors best. In our controlled coordinate-matching study,...
-
38
Teacher-Guided Tool Learning for Small Language Models via Bidirectional KL Constraints
Haoxuan Gao, Qin-Wen Luo, Xiang Chen, Yu Tian, Zequn Sun, Xiaofeng Cao, Jiangchao Yao, Sheng-Jun Huang
cs.LG · cs.CL
External tools can extend the capabilities of small language models (SLMs), enabling their deployment in resource constrained environments. However, their limited capabilities make it difficult for SLMs to discover effective tool use behaviors through reinforcement learning alone, while direct teacher supervision suffers from teacher student behavioral distribution mismatch and may propagate erroneous preferences. To address these challenges,...
-
39
Learning Perturbation-Specific Representations through Conditional Energy Prior for Cellular Response Prediction
Yuchen Yan, Yatao Bian
cs.LG · cs.AI
AI-based cellular perturbation prediction is essential to accelerate drug discovery and therapeutic identification. However, perturbation-specific signals are generally obscured by cellular complexity, making it difficult for existing methods to correctly capture perturbation effects. To address this problem, we propose PerturbEnergy, which introduces a conditional energy-based prior over perturbation representations. It assigns lower energy...
-
40
Not All Drift Is a Mechanism: Tracing Prompt Adaptation in LLMs
Mikhail Aleksandrov, Nail Bashirov, Maksim Ivanov, Andrew Tsukanov, Timofey Karyagin, Alexander Miasnikov, Gleb...
cs.LG · cs.AI · cs.CL
Prompt-based adaptation offers several ways to specialize a frozen language model, ranging from natural-language prompt optimization to learned continuous prompts. These methods are usually compared through downstream performance, although similar outputs need not arise from similar internal computations. We study how discrete prompt optimization, prompt tuning, and prefix tuning alter the processing of dense and mixture-of-experts language...
-
41
Multi-Stream Residual Computation in Diffusion Language Models
Ekaterina Alimaskina, Suren Kazarian, Gleb Molodtsov, Aleksandr Beznosikov
cs.LG · cs.CL
Diffusion language models reconstruct masked tokens in parallel, offering an alternative to autoregressive generation. Yet their Transformer backbones typically carry visible evidence and evolving masked predictions through a single residual stream. We equip masked diffusion models with multiple residual streams that persist through depth and adaptive routing between them, then investigate how they support these complementary roles. On the...
-
42
PRISM: Calibrating Black-Box Judges with Counterfactual Probes
Kakei Yamamoto, So Kuroki
cs.LG · cs.AI · cs.CL
Large language models are increasingly used as judges to reduce reliance on costly human evaluation, yet their scores can systematically disagree with human judgments. We ask whether responses to an explicitly imposed evaluative bias can help correct a judge's implicit scoring biases. We introduce PRISM (Probe-based Residual Inference of Scoring Misalignment), which holds the evaluated content fixed and measures score changes when attributes...
-
43
Beyond Task Success: A Theory of Representation Reuse
Zibo Meng, Andrew C. M. Austin, Alin Achim
cs.LG
A learned representation is useful beyond its source task only if later tasks can make use of what the encoder has already learned. Source-task performance, however, gives no guarantee that the representation itself will remain useful once the original head is removed. We study representation reuse with respect to a specified family of future tasks. For such a family, we define the task-relevant state as the coarsest deterministic state...
-
44
Over-parameterization Is Neither Robust nor Fragile in Matrix Sensing: A Geometric Theory of Solution Robustness
Runze Shi, Ziye Ma
cs.LG
Over-parameterization is ubiquitous in modern machine learning, yet its effect on robustness remains poorly understood. We study this question in matrix sensing, a canonical non-convex problem that can also be viewed, under an appropriate parameterization, as training a two-layer quadratic neural network. We develop a rank-aware robustness theory for the Burer-Monteiro factorization when the search rank k may exceed the target rank r_star,...
-
45
Bayesian Optimization of CVaR with Unknown Hyperparameters
Valentin Margraf, Jonas Hanselle, Viktor Bengs, Serafima Lebedeva, Eyke Hüllermeier, Marcel Wever
cs.LG
We study Bayesian optimization of the Conditional Value-at-Risk (CVaR) of an expensive black-box function that depends on a decision and an environmental variable. Existing no-regret algorithms assume that the correct Gaussian process (GP) kernel and its hyperparameters are known. In practice, however, they must be estimated from limited data, often yielding a misspecified surrogate. This is especially damaging for CVaR because...
-
46
Which Optimizer Leaks Most? Optimizer Implicit Bias as a Training-Data Privacy Controller
Sagi Habani, Yehonathan Refael, Ofir Lindenbaum
cs.LG · cs.AI
Releasing a neural network checkpoint can leak information about its training data, and existing reconstruction results rely on gradient-descent training. We ask whether the choice of optimizer affects the ability to recover the training data. Extending work connecting Karush--Kuhn--Tucker conditions to training-data reconstruction, we show that the implicit biases of popular optimizers (SGD, Adam, Muon) leave very different fingerprints of...
-
47
Reliable Risk Estimation with Kernel Mean Matching under Dependent Sampling
Alexey Zaytsev, Diana Koldasbayeva, Egor Serov
cs.LG
Kernel Mean Matching (KMM) estimates risk under covariate shift by reweighting labelled source samples so that their kernel mean embedding matches that of unlabeled target samples. Existing finite-sample guarantees are usually stated for independent samples, although many applications involve temporal, spatial, or clustered dependence. We give a finite-sample KMM analysis under dependent sampling by replacing nominal sample sizes with...
-
48
PING: physics informed graph neural network for advection-diffusion processes with applications to solar power forecasting
Jelena Simeunović, Baptiste Schubnel, Pierre-Jean Alet, Pascal Frossard, Rafael E. Carrillo
cs.LG · eess.SP
Advection-diffusion plays an essential role in transport phenomena such as atmospheric and cloud dynamics. Improved forecasting of cloud motion, in turn, enhances solar power production forecasts and facilitates the integration of additional photovoltaic (PV) sources into the power grid. However, both classical numerical models and emerging deep learning approaches face challenges when applied to spatially irregular sensor networks such as...
-
49
Optimal Online Regret for Context-Lumpable Bandits
Chenyu Lu, Zihan Zhang
cs.LG
We study stochastic contextual bandits with an unknown context-lumpable structure. The $S$ contexts are partitioned into $r$ blocks, and every arm has the same mean reward across contexts in the same block. Under arbitrary context distributions and without separation assumptions, we design an algorithm with expected regret $\widetilde{O}(\sqrt{r(S+K)T})$, matching the lower bound of Lee et al. (2023) up to logarithmic factors and improving...
-
50
Teach Me After You Learn It: Lookahead Self-Distillation for RLVR
Zepeng Zhai, Meilin Chen
cs.LG
On-policy self-distillation (OPSD) has emerged as a popular training paradigm for LLMs, providing dense, token-level supervision by aligning a model's own distribution with the distribution induced by privileged information. However, existing OPSD methods derive supervision solely from privileged information of the same instance, leaving inter-instance supervision largely unexplored. We propose Lookahead Self-Distillation (LEAD), which...
-
51
A Controlled Evaluation of Curvature Inspired Scalar Feature Scaling for Domain Generalization
Sunny Gupta, Amit Sethi
cs.LG · cs.AI · cs.CV
Domain-specific curvature is an appealing idea for domain generalization (DG), but curvature learning can denote substantially different computations. This paper evaluates one deliberately narrow implementation: the code calls it HyperDG, but it rescales Euclidean encoder features during training by learned positive scalars and uses an ordinary linear classifier on unscaled Euclidean features at inference. It does not implement a Lorentz or...
-
52
Operator Surrogates
Christoph Schwab, Josephine Westermann, Jakob Zech
cs.LG · stat.ML
This chapter provides a mathematical introduction to operator surrogates, which are finite-parametric maps approximating complex (typically nonlinear) operators between infinite-dimensional (function) spaces. A typical approach to constructing operator surrogates is the encoder-approximator-decoder paradigm, whereby infinite-dimensional inputs are mapped to a finite-dimensional latent space, processed, and then mapped back to the...
-
53
A Few More Tokens Ahead: Reinforcing Parallelism in Diffusion LLMs
Jiahao Kuang, Yangfan Gao, Tao Gui, Yuanbin Wu, Qi Zhang, Xuanjing Huang, Tao Ji
cs.LG · cs.AI
Parallel generation is widely regarded as an inherent advantage of diffusion large language models. Yet our fixed-quota evaluation of four representative models reveals severe degradation across mathematics and code, with all scores below 15% at 16 tokens per step. Token-level probes identify local predictive potential near the observed prefix, especially in mathematical text. We propose RePara, which reinforces short consecutive token blocks...
This edition is part of The Daily Abstract — cs.LG archive. Subscribe to receive these in your inbox each morning, automatically translated to Spanish, with reply-to-PDF: arxivdaily.ignorelist.com.
#D99C5E. Built and served on an always-free VM. The masthead is set 14% letterspaced because newspapers do that and it works.