Variational Importance Sampling

 Posted on June 29, 2019  |  10 minutes  |  Chad Scherrer

Lots of distributions are easy to evaluate (the density), but hard to sample. So when we need to sample such a distribution, we need to use some tricks. We'll see connections between two of these: importance sampling and variational inference, and see a way to use them together for fast inference. Importance sampling Importance sampling aims to make it easy to compute expected values. Say we have a distribution \(p\), and we'd like to compute the average of some function \(f\) of the distribution (or equivalently, the expected value of a "push-forward along \(f\)"). [Read More]
bayes  Soss  julia 

Soss.jl: Design Plans for Spring 2019

 Posted on January 27, 2019  |  7 minutes  |  Chad Scherrer

If you've followed my work recently, you've probably heard of my probabilistic programming system Soss.jl. I recently had the pleasure of presenting these ideas at PyData Miami: [N.B. Above is supposed to be an embedded copy of my slides from PyData Miami. I can see it from Chrome, but not Firefox. Very weird. ] In April I'll begin another "passion quarter" (essentially a sabbatical) and hope to really push this work forward. [Read More]
julia  bayes  Soss 

Julia for Probabilistic Metaprogramming

 Posted on September 11, 2018  |  6 minutes  |  Chad Scherrer

Since around 2010, I've been involved with using and developing probabilistic programming languages. So when I learn about new language, one of my first questions is whether it's a good fit for this kind of development. In this post, I'll talk a bit about working in this area with Julia, to motivate my Soss project. Domain-Specific Languages At a high level, a probabilistic programming languages is a kind of domain-specific language, or DSL. [Read More]
julia  bayes 

A Prelude to Pyro

 Posted on August 21, 2018  |  8 minutes  |  Chad Scherrer

Lately I've been exploring Pyro, a recent development in probabilistic programming from Uber AI Labs. It's an exciting development that has a huge potential for large-scale applications. In any technical writing, it's common (at least for me) to realize I need to add some introductory material before moving on. In writing about Pyro, this happened quite a bit, to the point that it warranted this post as a kind of warm-up. [Read More]
bayes  python  variational inference  pyro 

Bayesian Optimal Pricing, Part 2

 Posted on June 3, 2018  |  6 minutes  |  Chad Scherrer

This is Part 2 in a series on Bayesian optimal pricing. Part 1 is here. Introduction In Part 1 we used PyMC3 to build a Bayesian model for sales. By the end we had this result: A common advantage of Bayesian analysis is the understanding it gives us of the distribution of a given result. For example, we very easily analyze a sample from the posterior distribution of profit for a given price. [Read More]
bayes  python  pymc3 

Bayesian Optimal Pricing, Part 1

 Posted on May 6, 2018  |  9 minutes  |  Chad Scherrer

Pricing is a common problem faced by businesses, and one that can be addressed effectively by Bayesian statistical methods. We'll step through a simple example and build the background necessary to extend get involved with this approach. Let's start with some hypothetical data. A small company has tried a few different price points (say, one week each) and recorded the demand at each price. We'll abstract away some economic issues in order to focus on the statistical approach. [Read More]
bayes  python  pymc3 

Bayesian Changepoint Detection with PyMC3

 Posted on February 25, 2018  |  5 minutes  |  Chad Scherrer

A client comes to you with this problem: The coal company I work for is trying to make mining safer. We made some change around 1900 that seemed to improve things, but the records are all archived. Tracking down such old records can be expensive, and it would help a lot if we could narrow the search. Can you tell us what year we should focus on? Also, it would really help to know this is a real effect, and not just due to random variability - we don't want to waste resources digging up the records if there's not really anything there. [Read More]
bayes  python  pymc3