Saibo Geng
If you are familiar with constrained decoding, you might have occasionally heard people mentioning the word “posterior inference” in the context of constrained decoding. In this post, I will explain why constrained decoding can(should) be seen as a form of posterior inference. What is Posterior Inference? Posterior inference simply means inferring the posterior distribution of a model…
Speculative Sampling The idea of speculative sampling is to use a draft sampling to achieve the same sampling result as the target sampling. We have a target sampling distribution $p(x)$ and a draft sampling distribution $q(x)$. For each token $x_i$, we have two probabilities: $p(x_i)$ and $q(x_i)$, and we have: either $p(x_i) > q(x_i)$, or $p(x_i) \leq q(x_i)$, If we directly sample from $q(x)$,…
Perplexity Definition 1 Given a sequence $X = x_1, x_2, \dots, x_T$ of words and a language model $p$, the perplexity of the sequence can is defined as follows: $$ \text{PPL}(X) = p(x_1, x_2, \dots, x_T)^{-\frac{1}{T}} $$ where $p(x_1, x_2, \dots, x_T)$ is the probability of the sequence $X$ under the language model $p$(also called the likelihood of the sequence). With language models, the…
WordPiece WordPiece tokenization is a subword tokenization algorithm. It is used in BERT, which is a work of 2018. Example: from transformers import BertTokenizer tokenizer = BertTokenizer . from_pretrained ( "bert-base-uncased" ) tokenizer . tokenize ( "I have a new GPU!" ) [ "i" , "have" , "a" , "new" , "gp" , "##u" , "!" ] The tokenizer splits “gpu” into known subwords:…
Create your slides in Markdown - click the Slides button to check out the example. Add the publication’s full text or supplementary notes here. You can use rich formatting such as including code, math, and images .
Create slides in Markdown with Hugo Blox Builder Hugo Blox Builder | Documentation Features Efficiently write slides in Markdown 3-in-1: Create, Present, and Publish your slides Supports speaker notes Mobile friendly slides Controls Next: Right Arrow or Space Previous: Left Arrow Start: Home Finish: End Overview: Esc Speaker notes: S Fullscreen: F Zoom: Alt + Click PDF Export Code Highlighting…
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis posuere tellus ac convallis placerat. Proin tincidunt magna sed ex sollicitudin condimentum. Sed ac faucibus dolor, scelerisque sollicitudin nisi. Cras purus urna, suscipit quis sapien eu, pulvinar tempor diam. Quisque risus orci, mollis id ante sit amet, gravida egestas nisl. Sed ac tempus magna. Proin in dui enim. Donec condimentum,…
Click the Cite button above to demo the feature to enable visitors to import publication metadata into their reference management software. Create your slides in Markdown - click the Slides button to check out the example. Add the publication’s full text or supplementary notes here. You can use rich formatting such as including code, math, and images .
Click the Cite button above to demo the feature to enable visitors to import publication metadata into their reference management software. Create your slides in Markdown - click the Slides button to check out the example. Add the publication’s full text or supplementary notes here. You can use rich formatting such as including code, math, and images .