First of all, perplexity has nothing to do with characterizing how often you guess something right. It has more to do with characterizing the complexity of a stochastic sequence. Or in a variety of other ways, shrekcoin and this should make it even clearer where « log-average inverse probability » comes from.
- First of all, perplexity has nothing to do with characterizing how often you guess something right.
- So as you make make rolling one side of the die increasingly unlikely, the perplexity ends up looking as though the side doesn’t exist.
- So in this sense, perplexity is infinitely more unique/less arbitrary than entropy as a measurement.
- So perplexity represents the number of sides of a fair die that when rolled, produces a sequence with the same entropy as your given probability distribution.
Sign up or log in
- Wikipedia article on perplexity does not give an intuitive meaning for the same.
- I came across term perplexity which refers to the log-averaged inverse probability on unseen data.
- Or in a variety of other ways, and this should make it even clearer where « log-average inverse probability » comes from.
- It has more to do with characterizing the complexity of a stochastic sequence.
- So in this sense, perplexity is infinitely more unique/less arbitrary than entropy as a measurement.
So perplexity represents the number of sides of a fair die that when rolled, produces a sequence with the same entropy as your given probability distribution. I came across term perplexity which refers to the log-averaged inverse probability on unseen data. Wikipedia article on perplexity does not give an intuitive meaning for the same. So as you make make rolling one side of the die increasingly unlikely, the perplexity ends up looking as though the side doesn’t exist. I think it’s worth pointing out that perplexity is invariant with the base you use to define entropy. So in this sense, perplexity is infinitely more unique/less arbitrary than entropy as a measurement.