Collapsed Gibbs Sampling for LDA

[jegonzal]

This is a work in progress implementation of the collapsed Gibbs sampler for the LDA model using the GraphX abstraction primitives. While this is based on the (non-ergodic) bulk synchronous Gibbs sampler, we do exploit local parameter sharing and if document vertex partitioning is used we recover the Newman et al. style sampler.

Remaining tasks:

• Unite tests
• Raw document processing
• Likelihood calculation (requires log gamma transcendental functions)

[AmplabJenkins]

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/GraphXPullRequestBuilder/5934/

[dcrankshaw]

Can you make this a separate named function?

[AmplabJenkins]

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/GraphXPullRequestBuilder/9536/

[AmplabJenkins]

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/GraphXPullRequestBuilder/9695/

[AmplabJenkins]

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/GraphXPullRequestBuilder/10213/

[AmplabJenkins]

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/GraphXPullRequestBuilder/10215/