Estimates the phenotypic covariance matrix P of one or more traits
after accounting for fixed effects, for data with no relatedness or genomic
information. It is the counterpart of lmm_animal without the
genetic component, and uses the same priors, scaling and outputs.
Usage
lm_phen_cov(
Y,
X = NULL,
lkj_prior = 2,
iter = 3000,
warmup = floor(iter/2),
chains = 4,
cores = chains,
control = list(adapt_delta = 0.9),
cmd_stan = TRUE,
model_file = NULL,
covariance_structure = c("unstructured", "factor"),
n_factors = NULL,
hs_global_scale = 1,
hs_slab_scale = 1,
column_decay = 0.8,
...
)Arguments
- Y
Numeric response vector or matrix, with individuals in rows. Every trait must vary: constant traits cannot be standardized and are an error. Missing values (
NA) are allowed and are not discarded: see Details.- X
Optional finite numeric fixed-effects matrix with the same rows as Y and full column rank. The default
NULLfits an intercept-only model: one column of ones, named"(Intercept)". A matrix must be able to represent a constant (an intercept column, or the 0/1 indicators of every level of a factor), so that the mean of the response can be reported. Continuous columns are centered and scaled internally; returned beta draws use original X units.- lkj_prior
Positive shape of the LKJ prior on the correlation matrix.
1is uniform over matrices, not individual correlations: pairwise marginals concentrate increasingly around zero as dimension increases. Values above1add shrinkage toward zero. Used only bycovariance_structure = "unstructured"; explicitly supplying it for the factor model warns that it is ignored.- iter
Number of MCMC iterations, a whole number.
- warmup
Number of warmup iterations, a whole number smaller than
iter.- chains
Number of chains to run, a whole number.
- cores
Number of chains to run in parallel (cmdstanr only), a whole number.
- control
Named list of sampler controls, passed to whichever engine runs:
adapt_delta,adapt_engaged,max_treedepth,step_size(rstan'sstepsizeis accepted),init_buffer,term_buffer,windowandmetric. Any other name is an error, so a control is never silently ignored.- cmd_stan
Logical, use cmdstanr. If FALSE, uses rstan. Both engines are suggested packages: when cmdstanr is not installed the fit falls back to rstan with a warning, and when neither is installed the fit errors.
- model_file
Optional path to a custom Stan file replacing the default model.
- covariance_structure
Covariance structure for the phenotypic model: "unstructured" (default, scales times an LKJ correlation matrix) or "factor", which uses \(P = \mathrm{diag}(\sigma_P)(\Lambda\Lambda^T + \Psi)\mathrm{diag}(\sigma_P)\) with regularized-horseshoe loadings, where \(\sigma_P\) are the residual standard deviations, estimated with the same half-normal prior as the unstructured model.
- n_factors
Number of factors when
covariance_structure = "factor", a whole number between 1 and the number of traits minus one. IfNULL, the model falls back to the same sublinear rule aslmm_animal()and warns with the value; that fallback depends only on the number of traits, so usesuggest_n_factorsto choose it from the data when the factors themselves are meant to be interpreted. Ignored otherwise.- hs_global_scale, hs_slab_scale
Positive scales of the regularized-horseshoe prior on the factor loadings, used only by the factor model. The slab limits the prior standard deviation, not the realized loading. Communality is \(q/(1+q)\), where \(q\) is the random sum of squared unnormalized loadings; its prior depends on both scales, local scales, factor count and column decay.
- column_decay
Standard-deviation multiplier for successive factor columns, between 0 and 1. The default 0.8 orders the columns so that the factor labels stay meaningful; with
column_decay = 1the columns are exchangeable a priori and their labels can swap between chains. Few factors need a stronger decay than many: with two columns of comparable size 0.8 is often too mild to separate them, while0.5^(m - 1)would shrink the fifth column of a many-factor model to a sixteenth of the first. Only used by the factor model, and explicitly supplying it otherwise warns that it is ignored.- ...
additional parameters for stan sampling.
Value
With cmd_stan = TRUE, a CmdStanMCMC object from cmdstanr, usable with posterior and bayesplot (see lmm_animal); use rename_beta for beta draws named after the columns of X, and fit_draws or draws_mean to get P, corrP and sigma_P as matrices. With cmd_stan = FALSE, a stanfit object.
The unstructured fit reports P (original trait scale), corrP,
sigma_P and the fixed effects beta[trait, j]. sigma_P are
the residual standard deviations, so P, corrP and sigma_P
describe the covariance of the residuals after the fixed effects, not the
marginal covariance of Y.
Factor fits add quantities that describe the decomposition. communality
and uniqueness do not depend on how the factors are oriented, so they
are identified by the data. loadings and factor_magnitude do:
two solutions with the same P can differ by a rotation, a sign flip or
a permutation of the columns, and nothing fixes that choice except the prior.
A poor Rhat on those two is therefore not evidence that the fit
failed, and their posterior means describe whichever orientations the chains
visited. Judge convergence on P, corrP, sigma_P,
communality and uniqueness, and pin the orientation down by
lowering column_decay, which orders the columns, or by asking for
fewer factors.
loadings: the trait by factor loading matrix \(\Lambda\), on the correlation scale.communality: for trait \(k\), \(c_k = \sum_m \lambda_{km}^2\), the part of the trait's variance explained by the common factors. On the correlation scale \(c_k + u_k = 1\); multiplying bysigma_P[k]^2gives the common phenotypic variance of trait \(k\).uniqueness: for trait \(k\), \(u_k = 1 - c_k\), the trait-specific variance left after the factors. Multiplying bysigma_P[k]^2gives the unique phenotypic variance.factor_magnitude: for factor \(m\), \(\sum_k \lambda_{km}^2\), the variance the factor contributes across traits. It is a factor-level summary, not a per-trait quantity, and its values only compare across columns once the orientation is fixed.
These are available from fit$summary() and from
fit_draws: loadings is a trait by factor by draw array,
factor_magnitude is a factor by draw matrix, and the uniqueness and
communality vectors are trait by draw matrices.
Details
The unstructured model estimates P as scales times an LKJ-distributed correlation matrix, with half-normal priors on the scales of the standardized responses. With covariance_structure = "factor", P is instead \(\mathrm{diag}(\sigma_P)(\Lambda\Lambda^T + \Psi)\mathrm{diag}(\sigma_P)\), with regularized-horseshoe loadings and deterministic column decay. The factor fit reports loadings, factor_magnitude, uniqueness and communality in addition to P, corrP, sigma_P and the fixed effects. Returned quantities are on the original response scale.
Incomplete records are used rather than dropped. Without relatedness the
records are independent, so a row that misses some traits is evaluated on the
traits it does have: its marginal density is the multivariate normal on that
subvector, with the matching submatrix of P. Rows are grouped by which
traits they observe, each group gets its own Cholesky factor, and the missing
values are integrated out analytically, so they cost no parameters. A record
with nothing observed carries no likelihood and is an error instead. The count
of groups grows with the number of distinct patterns, at most one per row.
The pointwise log_lik is the density the model block evaluates, on the
standardized response; the raw-scale density differs from it by a constant,
which does not affect comparisons between fits to the same data.
The likelihood is computed by whitening the residuals with the Cholesky factor of P in a single triangular solve, at O(N K^2 + K^3) per evaluation rather than a per-individual loop, so the model scales to dozens of traits.
Continuous predictors are centered and divided by their sample standard deviation inside Stan. Constant and 0/1 indicator columns are not centered or scaled. Standardized slopes have normal(0, 1) priors; the intercept contribution has a wider normal(0, 2.5) prior. Returned beta coefficients are backtransformed to original predictor and response units, including restoring the response mean to the intercept; beta_scaled draws are also available.
Examples
if (FALSE) { # \dontrun{
set.seed(1)
n <- 60
X <- cbind(Intercept = 1, sex = rbinom(n, 1, 0.5))
P <- random_covariance(3, min_var = 0.8, max_var = 1.2,
trait_names = c("t1", "t2", "t3"))
B <- rbind(c(10, 9, 11), c(1, -0.5, 0.3))
Y <- X %*% B + matrix(rnorm(n * 3), n) %*% chol(P)
colnames(Y) <- c("t1", "t2", "t3")
fit <- lm_phen_cov(Y, X, iter = 1000, warmup = 500,
chains = 2, cores = 2, refresh = 0)
fit$summary(c("P", "corrP", "sigma_P"))
round(draws_mean(fit, "P"), 2)
# Sparse factor P
factor_fit <- lm_phen_cov(Y, X, covariance_structure = "factor",
n_factors = 2, iter = 1000, warmup = 500,
chains = 2, cores = 2, refresh = 0)
factor_fit$summary(c("P", "loadings", "factor_magnitude"))
dim(fit_draws(factor_fit)$loadings)
} # }
