The first time you see a species, word, bug, password, merchant, or customer behavior, it feels like a discovery.

The second time, it starts to feel like a member of the world.

The awkward question is what to do with the things seen exactly once.

They are not reliable representatives. A singleton might be noise. It might be a typo. It might be a real but rare type. It might also be the visible edge of a large unseen tail.

singletons are not clutter
they are a measurement instrument

That is the core Good-Turing intuition. If many observed types appeared only once, then the sample is still discovering new things. The sample is telling you, indirectly, that a nontrivial amount of probability mass remains outside the observed set.

Claim: the rarest observed counts are often the only evidence you have about the unobserved population. Throwing away singletons can make a dataset look cleaner while destroying the instrument that estimates what is missing.

The Count of Counts

Good’s 1953 paper, building on Turing’s wartime ideas, studied the “population frequencies of species” problem.1 The setup is wonderfully general: draw a random sample from a population with many possible species. A species could be a biological taxon, a word type, a phrase, a user action, a malware family, or any category in a long-tailed distribution.

Let \(n_r\) be the number of species observed exactly \(r\) times. The sample size is

\[N = \sum_{r \ge 1} r n_r.\]

The ordinary estimate for the probability of a seen species is its empirical frequency. But for low counts, especially \(r=1\), that estimate is too literal. A type seen once is not merely a type with probability \(1/N\). It is also a signal that the sampling frontier is still active.

Good-Turing’s most famous first-order missing-mass estimate is:

\[\widehat{P}(\text{next type is unseen}) \approx \frac{n_1}{N}.\]

The formula is almost suspiciously small. Count the singletons. Divide by the sample size. That is the estimated probability that the next draw belongs to a type you have not observed at all.

It is not magic. It is a leave-one-out thought experiment. The types that would have been unseen if one occurrence were removed are exactly the types that appear once now.

Richness Is Not Missing Mass

There are two related but different questions:

how much probability is unseen?
how many types are unseen?

Good-Turing’s \(n_1/N\) answers the first question: missing probability mass. Chao’s 1984 estimator addresses the second question: a lower-bound style estimate of total richness using singletons and doubletons.2

The common Chao1 form is

\[\widehat{S}_{\text{Chao1}} = S_{\text{obs}} + \frac{n_1^2}{2n_2},\]

where \(S_{\text{obs}}\) is the number of observed species and \(n_2\) is the number of doubletons. If there are many singletons and few doubletons, the bound rises. The sample has many things barely touched and few things that have started repeating.

Those two estimates answer different operational questions:

  • Missing mass asks how likely the next case is to surprise you.
  • Richness asks how many different surprises may exist.

A corpus can have modest missing mass but enormous unseen richness if the tail is very broad and individually tiny. That distinction is why word vocabularies, ecological surveys, security logs, and product event taxonomies all resist being summarized by “we saw 90% of the common stuff.”

Shakespeare’s Invisible Vocabulary

Efron and Thisted’s famous Shakespeare paper asked how many words Shakespeare knew but did not use in the surviving canon.3 They had a count-of-counts table: many words used once, fewer used twice, and so on. From that, they compared models for the unseen vocabulary.

The literal topic is charming. The statistical point is larger. The observed text is not the author’s vocabulary. It is a finite sample from a vocabulary through a writing process. Rare words are evidence about the unobserved lexicon.

Modern versions of this problem appear everywhere:

  • How many microbial taxa remain unseen after sequencing?
  • How many rare queries will hit search next week?
  • How many customer intents are absent from a support dataset?
  • How many security event types have not appeared in a training period?
  • How much probability mass is outside an LLM evaluation set?

The point is not that Good-Turing or Chao1 solves all of these. The point is that the naive observed set is almost never the population.

A Long-Tail Sampling Notebook

The lab below creates a synthetic long-tailed population. It samples from that population, records the frequency-of-frequencies table, and compares:

  • the true missing mass, which the simulator knows but a real analyst would not;
  • the Good-Turing missing-mass estimate \(n_1/N\);
  • the Chao1 richness estimate;
  • a Good-Toulmin forecast for how many new types appear in a future sample up to the same order as the original sample.

Default run:

  • true population size: 900 types;
  • sample size: 420;
  • observed types: 179;
  • singletons: 121;
  • true missing mass: about 26.9%;
  • Good-Turing missing mass: about 28.8%;
  • Chao1 richness estimate: about 415;
  • in the future holdout of 231 draws, the sample sees 53 new types while Good-Toulmin predicts about 58.
Observed types Singletons Good-Turing / Good-Toulmin Chao1 richness True unseen quantity

Deterministic toy model. The audit covers 28 settings for finite estimates, valid probability bounds, Chao1 not below observed richness, and a default Good-Turing missing-mass error below eight percentage points.

Try increasing Sample size. The observed set grows, but the singleton count does not necessarily vanish quickly. In a long tail, more sampling can reveal new singleton types at the same time it turns old singletons into doubletons.

Lower Tail skew. Probability spreads across many types; missing mass and Chao1 rise. Raise Tail skew. The head dominates; the sample repeats the same types more often, and the singleton instrument weakens.

Increase Future sample toward 100%. The Good-Toulmin forecast is still in its safer region here because the future sample is no larger than the original sample. Orlitsky, Suresh, and Wu study the deeper problem of predicting unseen species as this extrapolation becomes harder and the classical series becomes unstable.4

The Practical Lesson

The rare tail is where product metrics and scientific surveys often lie to us. If a dashboard says “we saw 179 types,” the next question is not only how many types were observed. It is how the observed types were distributed:

  • How many were seen once?
  • How many were seen twice?
  • How quickly is the discovery curve flattening?
  • Is missing probability mass operationally expensive?
  • Are singletons real events, data-quality artifacts, or both?
  • Does the sample design make rare types discoverable?

Deleting singletons may be reasonable after investigation. But deleting them because they are awkward is statistical self-harm. They are the only part of the sample that can point cleanly into the dark.

The singleton is not the tail.

It is the tail’s shadow on the wall.

  1. I. J. Good, “The Population Frequencies of Species and the Estimation of Population Parameters”, Biometrika, 1953. ↩

  2. Anne Chao, “Nonparametric Estimation of the Number of Classes in a Population”, Scandinavian Journal of Statistics, 1984. ↩

  3. Bradley Efron and Ronald Thisted, “Estimating the Number of Unseen Species: How Many Words Did Shakespeare Know?”, Biometrika, 1976. ↩

  4. Alon Orlitsky, Ananda Theertha Suresh, and Yihong Wu, “Optimal Prediction of the Number of Unseen Species”, PNAS, 2016. ↩