On cross validation for model selection.

I Rivals; L Personnaz

doi:10.1162/089976699300016476

Home/Archive/PubMed

PubMed · 10226186

On cross validation for model selection.

I Rivals·L Personnaz

View original source ↗Export metadata ↓

Abstract

In response to Zhu and Rower (1996), a recent communication (Goutte, 1997) established that leave-one-out cross validation is not subject to the "no-free-lunch" criticism. Despite this optimistic conclusion, we show here that cross validation has very poor performances for the selection of linear models as compared to classic statistical tests. We conclude that the statistical tests are preferable to cross validation for linear as well as for nonlinear model selection.

Explore related subjects

Linear Models Models, Statistical Nonlinear Dynamics Probability Reproducibility of Results Journal Article

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

I Rivals, L Personnaz. 1999-05-15. On cross validation for model selection.. https://doi.org/10.1162/089976699300016476

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related citations

PubMed2025

A robust transfer learning approach for high-dimensional linear regression to support integration of multi-source gene expression data.

Transfer learning aims to integrate useful information from multi-source datasets to improve the learning performance of target data. This can be effectively applied in genomics when we learn the gene associations in a target tissue, and data from other tissues can be integrated. However, heavy-tail distribution and outliers are common in genomics data, which poses challenges to the effectiveness of current transfer learning approaches. In this paper, we study the transfer learning problem under high-dimensional linear models with t-distributed error (Trans-PtLR), which aims to improve the estimation and prediction of target data by borrowing information from useful source data and offering robustness to accommodate complex data with heavy tails and outliers. In the oracle case with known transferable source datasets, a transfer learning algorithm based on penalized maximum likelihood and expectation-maximization algorithm is established. To avoid including non-informative sources, we propose to select the transferable sources based on cross-validation. Extensive simulation experiments as well as an application demonstrate that Trans-PtLR demonstrates robustness and better performance of estimation and prediction when heavy-tail and outliers exist compared to transfer learning for linear regression model with normal error distribution. Data integration, Variable selection, T distribution, Expectation maximization algorithm, Genotype-Tissue Expression, Cross validation.

Linear Models↗

PubMed2003

Multistability analysis for recurrent neural networks with unsaturating piecewise linear transfer functions.

Multistability is a property necessary in neural networks in order to enable certain applications (e.g., decision making), where monostable networks can be computationally restrictive. This article focuses on the analysis of multistability for a class of recurrent neural networks with unsaturating piecewise linear transfer functions. It deals fully with the three basic properties of a multistable network: boundedness, global attractivity, and complete convergence. This article makes the following contributions: conditions based on local inhibition are derived that guarantee boundedness of some multistable networks, conditions are established for global attractivity, bounds on global attractive sets are obtained, complete convergence conditions for the network are developed using novel energy-like functions, and simulation examples are employed to illustrate the theory thus developed.

Linear Models↗

PubMed2002

Non-linear mixed effects models for the evaluation of dissolution profiles.

The use of non-linear mixed effects models to describe dissolution data has been evaluated. A theoretical part is included to introduce this approach to scientists who are not familiar with this type of statistics. The standard settings of the statistical software package (S-plus) are used as much as possible. Several mathematical functions like the Weibull, logistic, first-order and Gompertz are employed as basis for the non-linear mixed effects models. Examples are given using dissolution data of immediate and extended release tablets. The results are compared with those obtained using linear mixed effects models.

Linear Models↗