Adaptive Metropolis within Gibbs (AMWG)

Implementation of a Metropolis-within-Gibbs sampler [63][79][95] for iteratively simulating autocorrelated draws from a distribution that can be specified up to a constant of proportionality.

Model-Based Constructor

AMWG(params::ElementOrVector{Symbol}, sigma::ElementOrVector{T<:Real}; adapt::Symbol=:all, args...)

Construct a Sampler object for AMWG sampling. Parameters are assumed to be continuous, but may be constrained or unconstrained.

Arguments

  • params : stochastic node(s) to be updated with the sampler. Constrained parameters are mapped to unconstrained space according to transformations defined by the Stochastic unlist() function.
  • sigma : scaling value or vector of the same length as the combined elements of nodes params, defining initial standard deviations for univariate normal proposal distributions. Standard deviations are relative to the unconstrained parameter space, where candidate draws are generated.
  • adapt : type of adaptation phase. Options are
    • :all : adapt proposals during all iterations.
    • :burnin : adapt proposals during burn-in iterations.
    • :none : no adaptation (Metropolis-within-Gibbs sampling with fixed proposals).
  • args... : additional keyword arguments to be passed to the AMWGVariate constructor.

Value

Returns a Sampler{AMWGTune} type object.

Example

See the Birats, Blocker, and other Examples.

Stand-Alone Function

sample!(v::AMWGVariate; adapt::Bool=true)

Draw one sample from a target distribution using the AMWG sampler. Parameters are assumed to be continuous and unconstrained.

Arguments

  • v : current state of parameters to be simulated. When running the sampler in adaptive mode, the v argument in a successive call to the function will contain the tune field returned by the previous call.
  • adapt : whether to adaptively update the proposal distribution.

Value

Returns v updated with simulated values and associated tuning parameters.

Example

The following example samples parameters in a simple linear regression model. Details of the model specification and posterior distribution can be found in the Supplement. Also, see the Line: Block-Specific Sampling with AMWG and Slice example.

################################################################################
## Linear Regression
##   y ~ N(b0 + b1 * x, s2)
##   b0, b1 ~ N(0, 1000)
##   s2 ~ invgamma(0.001, 0.001)
################################################################################

using Mamba

## Data
data = Dict(
  :x => [1, 2, 3, 4, 5],
  :y => [1, 3, 3, 3, 5]
)

## Log-transformed Posterior(b0, b1, log(s2)) + Constant
logf = function(x::DenseVector)
   b0 = x[1]
   b1 = x[2]
   logs2 = x[3]
   r = data[:y] .- b0 .- b1 .* data[:x]
   (-0.5 * length(data[:y]) - 0.001) * logs2 -
     (0.5 * dot(r, r) + 0.001) / exp(logs2) -
     0.5 * b0^2 / 1000 - 0.5 * b1^2 / 1000
end

## MCMC Simulation with Adaptive Metopolis-within-Gibbs Sampling
n = 5000
burnin = 1000
sim = Chains(n, 3, names = ["b0", "b1", "s2"])
theta = AMWGVariate([0.0, 0.0, 0.0], 1.0, logf)
for i in 1:n
  sample!(theta, adapt = (i <= burnin))
  sim[i, :, 1] = [theta[1:2]; exp(theta[3])]
end
describe(sim)

AMWGVariate Type

Declaration

const AMWGVariate = SamplerVariate{AMWGTune}

Fields

  • value::Vector{Float64} : simulated values.
  • tune::AMWGTune : tuning parameters for the sampling algorithm.

Constructor

AMWGVariate(x::AbstractVector{T<:Real}, sigma::ElementOrVector{U<:Real}, logf::Function; batchsize::Integer=50, target::Real=0.44)

Construct an AMWGVariate object that stores simulated values and tuning parameters for AMWG sampling.

Arguments

  • x : initial values.
  • sigma : scaling value or vector of the same length as the combined elements of nodes params, defining initial standard deviations for univariate normal proposal distributions. Standard deviations are relative to the unconstrained parameter space, where candidate draws are generated.
  • logf : function that takes a single DenseVector argument of parameter values at which to compute the log-transformed density (up to a normalizing constant).
  • batchsize : number of samples that must be accumulated before applying an adaptive update to the proposal distributions.
  • target : target acceptance rate for the algorithm.

Value

Returns an AMWGVariate type object with fields set to the supplied x and tuning parameter values.

AMWGTune Type

Declaration

type AMWGTune <: SamplerTune

Fields

  • logf::Nullable{Function} : function supplied to the constructor to compute the log-transformed density, or null if not supplied.
  • adapt::Bool : whether the proposal distribution is being adaptively tuned.
  • accept::Vector{Int} : number of accepted candidate draws generated for each element of the parameter vector during adaptive updating.
  • batchsize::Int : number of samples that must be accumulated before applying an adaptive update to the proposal distributions.
  • m::Int : number of adaptive update iterations that have been performed.
  • sigma::Vector{Float64} : updated values of the proposal standard deviations if m > 0, and user-supplied values otherwise.
  • target::Float64 : target acceptance rate for the adaptive algorithm.