stat.ML Statistics

Machine Learning papers on arXiv

Follow statistical learning methods and their inference properties in stat.ML. A statistical objective or assumption can separate the papers you need from a broad machine-learning feed.

1,164 papers indexed

Follow the whole category with the official stat.ML RSS feed, or use a personal filter to select keywords and authors. See the arXiv RSS feed guide for setup and the category guide for primary and cross-listed subjects.

Filter ideas for Statistical Learning

How filters work

Add cs.LG in an OR category group when your reading spans both categories. A narrower AND condition can then focus the combined feed on uncertainty, estimation, or another statistical question.

  • Uncertainty estimation

    Add calibration, prediction, or a model family to focus on the kind of uncertainty you study.

    Category stat.ML + abstract contains “uncertainty”
    Try filter
  • Bayesian learning

    Pair Bayesian with a task such as inference or optimization rather than treating all Bayesian methods as one topic.

    Category stat.ML + abstract contains “Bayesian”
    Try filter

Examples are starting points. Preview them before saving; keyword matches do not assess a paper’s quality. Recent papers below use the primary category; saved category filters also match cross-listed categories.

Recent Machine Learning papers

5 recent papers
  • David Köhler, Matthias Schmid

    Abstract

    Modern machine learning models, such as artificial neural networks, achieve high predictive performance through increasingly complex black-box architectures, limiting their use in settings where transparency, interpretability, and fairness are required. Here we present the R package ONAM, which provides a novel and practical implementation of functional decomposition, a technique to facilitate the interpretability of black-box models through structured effect representations. ONAM implements an algorithmic framework for the post-hoc decomposition of predictions into components associated with different input features. The package is designed for usability and efficiency, including functionalities for visualization as well as quantitative measures to assess interpretability. As a model-agnostic method, it can be applied to a wide range of prediction models. We describe the mathematical foundations, algorithmic design, and software architecture of ONAM. A central result of the proposed methodology is that the resulting decomposition satisfies the condition of hierarchical orthogonality, thereby guaranteeing that all components are identifiable and uniquely defined.

  • Daniel Kua, Yan Song

    Abstract

    Generative data augmentation adds samples from generative models to a classifier's training set. Many influential studies use generators pre-trained on large external datasets, and so mix the value of synthetic data with knowledge transferred through pretraining. We study the case without external data, where each generator learns only from the classifier's own training set, and ask when and why adding generated samples to that training set raises or lowers the classifier's test accuracy. We train classifiers with four backbones on six datasets, two of them medical, adding samples from four generators. We bound the excess risk of a classifier trained with synthetic data, that is, how far its population risk can lie above the minimum risk within its hypothesis class. The bound charges synthetic data for the total variation distance between the real and generated distributions and credits synthetic data for a larger training set. The gain from adding synthetic data, the fall in population risk, is at most the excess risk of the real-only classifier, which trains on the real images alone. Read through the bound, the experiments suggest that synthetic data help most when our classifiers detect only a small discrepancy between the generated and real data and the real-only classifier has room to improve, and that synthetic data help little or hurt when the detected discrepancy is large. Generators with a larger lower bound on the total variation distance, a bound that we estimate from our own classifiers, tend to help less on five of the six datasets, and on the CIFAR datasets the most accurate backbone gains the least. The bound gives these directions but not the size of the largest gains, most of which traditional augmentation also reaches at the same number of optimizer steps.

  • Jianbin Tan, Anru R. Zhang

    Abstract

    We discuss the Journal of the American Statistical Association (JASA) discussion paper "Deep Fréchet Regression" and explore an alternative perspective based on conditional flow matching. We propose learning covariate-dependent transport directly within the response space, avoiding separate manifold learning and reverse mapping. This perspective offers a potentially more flexible framework for incorporating geometric structure and modeling full conditional distributions.

  • Michael Wieck-Sosa, Cosma Rohilla Shalizi

    Abstract

    Neural posterior estimation (NPE) is a popular Bayesian simulation-based inference (SBI) approach that uses neural networks trained on simulated data--parameter pairs to approximate the posterior given a user-chosen prior. For high-dimensional observations, NPE methods typically approximate the conditional density of the parameter given a user-chosen or learned summary statistic, rather than the entire dataset, due to the curse of dimensionality. However, both approaches face challenges: hand-crafting good summaries for complex models requires considerable effort from scientists, while learning them from massive datasets can be prohibitively computationally expensive. To address these challenges, we propose approximating the conditional density of a parameter given low-dimensional sketches of the data. We show that, for a model with a pp-dimensional parameter, only 2p+12p+1 random sketches are needed for posterior consistency. We compare with state-of-the-art embedding networks for NPE, and we find that our method \emph{matches} the best embedding network's performance while \emph{increasing computational efficiency}.

  • Tobias Höppe, Stefan Bauer, Qiang Liu, Andrea Dittadi, Kirill Neklyudov

    Abstract

    Few-step generators via flow models require complicated technical solutions such as a pretrained flow model or a consistency loss. While rectified flows have gained popularity for their success in few-step sampling, they pay a cost of needing a pre-trained model and sampling a large number of couplings per dataset. Yet for a finite dataset and a Gaussian probability path, the probability-flow vector field of the empirical distribution is available in closed form, so one could use this field as the teacher and skip pretraining altogether. We demonstrate that, surprisingly, this approach is impossible for problems of practical relevance. In particular, the analytic solution of the diffusion process in high dimensions becomes almost a linear transformation. The integration returns the noise vector drawn at initialisation, plus a drift that is empirically a faint copy of the data point, almost independent of the noise. The coupling therefore differs from an independent pairing only through this drift. As a result, training a model on these couplings is almost equivalent to training the model on the independent draws of the noise and the data, which does not provide a useful teacher. We highlight that this is the direct consequence of the high-dimensionality of the data, and that for low enough dimensions or large enough datasets, analytic couplings become more efficient teachers. We quantify this effect by the fraction of endpoint energy carried by the drift, which scales as log⁡Nd\frac{\log N}{d} for a dataset of size NN in a dd-dimensional space. Finally, we provide an empirical study confirming all the theoretical results presented in the paper.

Frequently asked questions

How do I get alerts for new Machine Learning papers on arXiv?

Create a filter for the stat.ML category — add keywords or authors to narrow it further if you like. Matching papers are delivered to a personal RSS feed, and optionally by immediate, daily, or weekly email digest.

Where does this Machine Learning paper data come from?

Paper metadata comes directly from arXiv through its official OAI-PMH feed, checked roughly every 15 minutes for new submissions in stat.ML and every other arXiv category.