Some of the fundamental errors that occur in probabilistic programs are due

to incorrect statistical models, ignoring dependence between random variables,

or, using wrong hyper-parameters (such as sample size) for inference. To prevent these errors, the solution most probabilistic programming languages adopt is to keep inference external to the core language semantics. While this limitation avoids potential pitfalls, it prevents programmers

from invoking inference at arbitrary points in the program and composing the

results.

However, designers of large systems often need to use inference for practical

reasons, such as, to efficiently transmit a probability distribution across the

network or to decompose an intractable inference problem into

smaller pieces.

We present a programming model, FLEXI, that extends a typical probabilistic programming

language with new constructs for invoking inference. To debug problems that arise from the use of inference, we use a Decorated Bayesian network to represent the program’s distribution and its inference operations. The representation lets us detect and explain bugs from two categories: approximation and dependence.

List of Authors: Chandrakana Nandi, Adrian Sampson, Dan Grossman, Todd Mytkowicz, Kathryn S. McKinley

Camera ready extended abstract: