Adaptive Mixture Metropolis (AMM)

Implementation of the Roberts and Rosenthal [79] adaptive (multivariate) mixture Metropolis [45][46][63] sampler for simulating autocorrelated draws from a distribution that can be specified up to a constant of proportionality.

Model-Based Constructor

AMM(params::ElementOrVector{Symbol}, Sigma::Matrix{T<:Real}; adapt::Symbol=:all, args...)

Construct a Sampler object for AMM sampling. Parameters are assumed to be continuous, but may be constrained or unconstrained.

Arguments

  • params : stochastic node(s) to be updated with the sampler. Constrained parameters are mapped to unconstrained space according to transformations defined by the Stochastic unlist() function.
  • Sigma : covariance matrix for the non-adaptive multivariate normal proposal distribution. The covariance matrix is relative to the unconstrained parameter space, where candidate draws are generated.
  • adapt : type of adaptation phase. Options are
    • :all : adapt proposal during all iterations.
    • :burnin : adapt proposal during burn-in iterations.
    • :none : no adaptation (multivariate Metropolis sampling with fixed proposal).
  • args... : additional keyword arguments to be passed to the AMMVariate constructor.

Value

Returns a Sampler{AMMTune} type object.

Example

See the Seeds and other Examples.

Stand-Alone Function

sample!(v::AMMVariate; adapt::Bool=true)

Draw one sample from a target distribution using the AMM sampler. Parameters are assumed to be continuous and unconstrained.

Arguments

  • v : current state of parameters to be simulated. When running the sampler in adaptive mode, the v argument in a successive call to the function will contain the tune field returned by the previous call.
  • adapt : whether to adaptively update the proposal distribution.

Value

Returns v updated with simulated values and associated tuning parameters.

Example

The following example samples parameters in a simple linear regression model. Details of the model specification and posterior distribution can be found in the Supplement.

################################################################################
## Linear Regression
##   y ~ N(b0 + b1 * x, s2)
##   b0, b1 ~ N(0, 1000)
##   s2 ~ invgamma(0.001, 0.001)
################################################################################

using Mamba

## Data
data = Dict(
  :x => [1, 2, 3, 4, 5],
  :y => [1, 3, 3, 3, 5]
)

## Log-transformed Posterior(b0, b1, log(s2)) + Constant
logf = function(x::DenseVector)
   b0 = x[1]
   b1 = x[2]
   logs2 = x[3]
   r = data[:y] .- b0 .- b1 .* data[:x]
   (-0.5 * length(data[:y]) - 0.001) * logs2 -
     (0.5 * dot(r, r) + 0.001) / exp(logs2) -
     0.5 * b0^2 / 1000 - 0.5 * b1^2 / 1000
end

## MCMC Simulation with Adaptive Multivariate Metopolis Sampling
n = 5000
burnin = 1000
sim = Chains(n, 3, names = ["b0", "b1", "s2"])
theta = AMMVariate([0.0, 0.0, 0.0], Matrix{Float64}(I, 3, 3), logf)
for i in 1:n
  sample!(theta, adapt = (i <= burnin))
  sim[i, :, 1] = [theta[1:2]; exp(theta[3])]
end
describe(sim)

AMMVariate Type

Declaration

const AMMVariate = SamplerVariate{AMMTune}

Fields

  • value::Vector{Float64} : simulated values.
  • tune::AMMTune : tuning parameters for the sampling algorithm.

Constructor

AMMVariate(x::AbstractVector{T<:Real}, Sigma::Matrix{U<:Real}, logf::Function; beta::Real=0.05, scale::Real=2.38)

Construct an AMMVariate object that stores simulated values and tuning parameters for AMM sampling.

Arguments

  • x : initial values.
  • Sigma : covariance matrix for the non-adaptive multivariate normal proposal distribution. The covariance matrix is relative to the unconstrained parameter space, where candidate draws are generated.
  • logf : function that takes a single DenseVector argument of parameter values at which to compute the log-transformed density (up to a normalizing constant).
  • beta : proportion of weight given to draws from the non-adaptive proposal with covariance factorization SigmaL, relative to draws from the adaptively tuned proposal with covariance factorization SigmaLm, during adaptive updating.
  • scale : factor (scale^2 / length(x)) by which the adaptively updated covariance matrix is scaled—default value adopted from Gelman, Roberts, and Gilks [32].

Value

Returns an AMMVariate type object with fields set to the supplied x and tuning parameter values.

AMMTune Type

Declaration

type AMMTune <: SamplerTune

Fields

  • logf::Nullable{Function} : function supplied to the constructor to compute the log-transformed density, or null if not supplied.
  • adapt::Bool : whether the proposal distribution is being adaptively tuned.
  • beta::Float64 : proportion of weight given to draws from the non-adaptive proposal with covariance factorization SigmaL, relative to draws from the adaptively tuned proposal with covariance factorization SigmaLm, during adaptive updating.
  • m::Int : number of adaptive update iterations that have been performed.
  • Mv::Vector{Float64} : running mean of draws v during adaptive updating. Used in the calculation of SigmaLm.
  • Mvv::Matrix{Float64} : running mean of v * v' during adaptive updating. Used in the calculation of SigmaLm.
  • scale::Float64 : factor (scale^2 / length(v)) by which the adaptively updated covariance matrix is scaled.
  • SigmaL::LowerTriangular{Float64} : Cholesky factorization of the non-adaptive covariance matrix.
  • SigmaLm::Matrix{Float64} : pivoted factorization of the adaptively tuned covariance matrix.