Adaptive Metropolis within Gibbs (AMWG)

Implementation of a Metropolis-within-Gibbs sampler [60][75][91] for iteratively simulating autocorrelated draws from a distribution that can be specified up to a constant of proportionality.

Model-Based Constructor

AMWG(params::ElementOrVector{Symbol}, sigma::ElementOrVector{T<:Real}; adapt::Symbol=:all, batchsize::Integer=50, target::Real=0.44)

Construct a Sampler object for adaptive Metropolis-within-Gibbs sampling. Parameters are assumed to be continuous, but may be constrained or unconstrained.

Arguments

  • params : stochastic node(s) to be updated with the sampler. Constrained parameters are mapped to unconstrained space according to transformations defined by the Stochastic unlist() function.

  • sigma : scaling value or vector of the same length as the combined elements of nodes params, defining initial standard deviations for univariate normal proposal distributions. Standard deviations are relative to the unconstrained parameter space, where candidate draws are generated.

  • adapt : type of adaptation phase. Options are
    • :all : adapt proposals during all iterations.
    • :burnin : adapt proposals during burn-in iterations.
    • :none : no adaptation (Metropolis-within-Gibbs sampling with fixed proposals).

  • batchsize : number of samples that must be accumulated before applying an adaptive update to the proposal distributions.

  • target : target acceptance rate for the algorithm.

Value

Returns a Sampler{AMWGTune} type object.

Example

See the Birats, Blocker, and other Examples.

Stand-Alone Function

amwg!(v::AMWGVariate, sigma::Vector{T<:Real}, logf::Function; adapt::Bool=true, batchsize::Integer=50, target::Real=0.44)

Simulate one draw from a target distribution using an adaptive Metropolis-within-Gibbs sampler. Parameters are assumed to be continuous and unconstrained.

Arguments

  • v : current state of parameters to be simulated. When running the sampler in adaptive mode, the v argument in a successive call to the function should contain the tune field returned by the previous call.
  • sigma : vector of the same length as v, defining initial standard deviations for univariate normal proposal distributions.
  • logf : function that takes a single DenseVector argument of parameter values at which to compute the log-transformed density (up to a normalizing constant).
  • adapt : whether to adaptively update the proposal distribution.
  • batchsize : number of samples that must be newly accumulated before applying an adaptive update to the proposal distributions.
  • target : target acceptance rate for the adaptive algorithm.

Value

Returns v updated with simulated values and associated tuning parameters.

Example

The following example samples parameters in a simple linear regression model. Details of the model specification and posterior distribution can be found in the Supplement. Also, see the Line: Block-Specific Sampling with AMWG and Slice example.

################################################################################
## Linear Regression
##   y ~ N(b0 + b1 * x, s2)
##   b0, b1 ~ N(0, 1000)
##   s2 ~ invgamma(0.001, 0.001)
################################################################################

using Mamba

## Data
data = Dict(
  :x => [1, 2, 3, 4, 5],
  :y => [1, 3, 3, 3, 5]
)

## Log-transformed Posterior(b0, b1, log(s2)) + Constant
logf = function(x::DenseVector)
   b0 = x[1]
   b1 = x[2]
   logs2 = x[3]
   r = data[:y] - b0 - b1 * data[:x]
   (-0.5 * length(data[:y]) - 0.001) * logs2 -
     (0.5 * dot(r, r) + 0.001) / exp(logs2) -
     0.5 * b0^2 / 1000 - 0.5 * b1^2 / 1000
end

## MCMC Simulation with Adaptive Metopolis-within-Gibbs Sampling
n = 5000
burnin = 1000
sim = Chains(n, 3, names = ["b0", "b1", "s2"])
theta = AMWGVariate([0.0, 0.0, 0.0])
sigma = ones(3)
for i in 1:n
  amwg!(theta, sigma, logf, adapt = (i <= burnin))
  sim[i, :, 1] = [theta[1:2]; exp(theta[3])]
end
describe(sim)

AMWGVariate Type

Declaration

typealias AMWGVariate SamplerVariate{AMWGTune}

Fields

  • value::Vector{Float64} : simulated values.
  • tune::AMWGTune : tuning parameters for the sampling algorithm.

Constructors

AMWGVariate(x::AbstractVector{T<:Real})
AMWGVariate(x::AbstractVector{T<:Real}, tune::AMWGTune)

Construct a AMWGVariate object that stores simulated values and tuning parameters for adaptive Metropolis-within-Gibbs sampling.

Arguments

  • x : simulated values.
  • tune : tuning parameters for the sampling algorithm. If not supplied, parameters are set to their defaults.

Value

Returns a AMWGVariate type object with fields set to the values supplied to arguments x and tune.

AMWGTune Type

Declaration

type AMWGTune <: SamplerTune

Fields

  • adapt::Bool : whether the proposal distribution has been adaptively tuned.
  • accept::Vector{Int} : number of accepted candidate draws generated for each element of the parameter vector during adaptive updating.
  • batchsize::Int : number of samples that must be accumulated before applying an adaptive update to the proposal distributions.
  • m::Int : number of adaptive update iterations that have been performed.
  • sigma::Vector{Float64} : updated values of the proposal standard deviations if adapt = true, and the user-defined values otherwise.
  • target::Real : target acceptance rate for the adaptive algorithm.