Imagine a record with no fields.

No pointer to the agent.

No slot for the object.

No little dictionary saying where the place lives.

Instead, every role and every item is a wide random vector. To store the fact, the system multiplies role vectors by item vectors, adds the products together, and keeps the sum.

That sounds like vandalism until the dimension gets large enough.

The trick has several names and several concrete models: hyperdimensional computing, vector symbolic architectures, binary spatter codes, holographic reduced representations, semantic pointers, random indexing. The family reappears because it offers a strange compromise. It keeps symbolic operations like binding and composition, but the objects being manipulated are distributed vectors that tolerate noise and superposition.12

This note uses one small member of the family: bipolar Multiply-Add-Permute style vectors. It is not the whole field. It is the smallest version I know that makes the core bargain visible.

The Three Moves

Start with random bipolar vectors:

\[R_i, V_j \in \{-1,+1\}^D.\]

Here R might be a role such as agent, and V might be an item such as item-040. If the entries are independent signs, two unrelated vectors have cosine close to zero. More precisely, their normalized dot product has standard deviation about:

\[\frac{1}{\sqrt{D}}.\]

That is the first move: random symbols. A new symbol can be minted by a pseudorandom seed instead of a learned embedding table.

The second move is binding. In this lab, binding is componentwise multiplication:

\[R \odot V.\]

For bipolar vectors this operation is its own inverse, coordinate by coordinate:

\[R \odot (R \odot V) = V.\]

The third move is bundling. To store several role-value pairs in one memory vector:

\[M = \sum_i R_i \odot V_i.\]

To ask for the value bound to role \(R_k\):

\[R_k \odot M = V_k + \sum_{i\ne k} R_k \odot R_i \odot V_i.\]

The first term is the desired item. The other terms are crosstalk. In high dimension, they behave like noise. Cleanup memory then compares the noisy query against the known item vectors and returns the nearest one.

So the whole record query is:

unbind with the role
then clean up to the nearest known item

That is not a parser. It is a noisy associative memory.

A Lab: Binding Without Fields

The lab below creates random role and item hypervectors, bundles several role-value bindings into one record vector, corrupts the bundled memory by flipping coordinate signs, and then tries to recover each item by unbinding with its role.

The default uses D=2048, 128 item symbols, 10 role-value bindings, and 6% coordinate sign noise. The audit recovers every default binding. The mean cleanup margin is 0.2272, the weakest margin is 0.2098, and the measured standard deviation of unrelated item cosines is 0.0244, close to the theoretical 1/sqrt(D)=0.0221.

That margin is the receipt. It is the space between “the right item was nearest” and “an impostor was nearly as good.”

Deterministic browser experiment using bipolar MAP-I style hypervectors. Binding is componentwise multiplication; bundling is integer addition; cleanup is nearest-neighbor search over the item vocabulary.

Lower Dimension to 256, raise Bindings, or raise Sign noise. The record does not fail all at once. The first warning is the margin shrinking. Eventually a random item becomes closer than the correct value.

That is the practical shape of the method:

capacity is not a field count
capacity is a margin budget

Why the Nearest Impostor Matters

If a query vector is compared against one unrelated item, the accidental cosine is usually near zero. If it is compared against hundreds or millions of items, the nearest accidental match is not near zero anymore.

A crude scale for the largest random cosine among \(V\) candidates is:

\[\sqrt{\frac{2\log V}{D}}.\]

The lab calls this the impostor floor. It is not an exact theorem for every cleanup memory, but it is the right warning label. More vocabulary items mean more chances for a random item to look plausible. More bindings mean more crosstalk in the query. More coordinate noise means less signal survives.

Those three pressures meet at the cleanup decision.

In the default lab, the impostor floor is about 0.0688, while the weakest observed cleanup margin is about 0.2098. That is comfortable. In a stress case with D=512, 48 bindings, 256 item symbols, and 18% sign noise, the same audit recovers only 27.1% of the role-value pairs.

The algebra did not change. The margin budget did.

This Is Not an Embedding in the Usual Sense

It is tempting to call every vector an embedding and move on. That loses the interesting part.

A learned embedding usually asks nearby vectors to mean nearby things. A random hypervector begins with the opposite policy: unrelated atomic symbols are nearly orthogonal. Similarity is not assigned at birth. It is created by operations.

Bundling creates set-like similarity:

A + B is partly similar to A
A + B is partly similar to B

Binding creates a new vector deliberately dissimilar to its arguments:

role * item is not close to role
role * item is not close to item

That dissimilarity is useful. If agent * mary were still close to mary, then every role-value pair containing Mary would blur together. Binding moves the item into a role-specific subspace-like disguise. Unbinding removes the disguise.

Plate’s holographic reduced representations used circular convolution for the same broad purpose: represent compositional structure in fixed-width distributed vectors, then recover noisy parts through cleanup memory.2 Kanerva’s binary spatter-code line pushed the bit-vector version.3 Gayler’s VSA framing emphasized the cognitive problem: how can a connectionist system rapidly build and transform compositional structures without falling back to ordinary symbolic pointers?4

The answer is not “make vectors magic.” The answer is:

make the vector space wide enough
that random accidents are measurable

The Fine Print

There are several traps.

First, cleanup memory is doing work. The query vector is not self-describing. If the dictionary of possible items is wrong, incomplete, or too large for the margin, the nearest neighbor can be confidently wrong.

Second, superposition is lossy. Bundling ten role-value pairs into one vector is not like storing ten fields in a struct. The crosstalk is a real term in the equation. More dimensions buy more room, but they do not repeal the budget.

Third, there is no single VSA. The Kleyko, Rachkovskij, Osipov, and Rahimi survey is useful partly because it refuses to collapse the field into one operator set. Tensor products, HRR, MAP, binary spatter codes, sparse binary distributed representations, and related models make different tradeoffs.5 The lab uses MAP-I: random sign vectors, addition for bundling, and elementwise multiplication for binding. Clarkson, Ubaru, and Yang analyze capacity for several VSA families and explicitly connect some of the behavior to sketching and Bloom-filter-like data structures.6

Fourth, a beautiful vector record is still a data structure. It needs tests:

  1. What dictionary is cleanup allowed to search?
  2. How many bindings are bundled before retrieval?
  3. How many dimensions are used?
  4. How large is the nearest-impostor margin?
  5. What happens when coordinates are corrupted?
  6. Are the symbols random, learned, or reused from another system?

Those questions sound ordinary because they are. Hyperdimensional computing is not an escape from engineering. It is a different place to spend the engineering budget.

The Part I Like

The part I like is that the record does not contain a field table.

It contains evidence.

Ask with the right role and one item rises out of the superposition. Ask with a wrong role and you get noise. Add too many facts and the cleanup margin thins. Make the dimension larger and the random vectors move farther apart. Corrupt coordinates and the system degrades gradually until the impostor wins.

That is a lovely little model of a broader computational mood:

structure can be exact locally
while storage stays statistical

The record has no fields. It has enough dimensions for the fields to be recoverable.

That distinction is the whole machine.

Paper Trail

  1. Pentti Kanerva, “Hyperdimensional Computing: An Introduction to Computing in Distributed Representation with High-Dimensional Random Vectors”, Cognitive Computation 1(2), 2009. ↩

  2. Tony A. Plate, “Holographic Reduced Representations”, IEEE Transactions on Neural Networks 6(3), 1995. ↩ ↩2

  3. Pentti Kanerva, “Computing with 10,000-Bit Words”, 52nd Annual Allerton Conference on Communication, Control, and Computing, 2014. ↩

  4. Ross W. Gayler, “Vector Symbolic Architectures Answer Jackendoff’s Challenges for Cognitive Neuroscience”, arXiv, 2004; originally ICCS/ASCS International Conference on Cognitive Science, 2003. ↩

  5. Denis Kleyko, Dmitri A. Rachkovskij, Evgeny Osipov, and Abbas Rahimi, “A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part I: Models and Data Transformations”, ACM Computing Surveys 55(6), 2022. ↩

  6. Kenneth L. Clarkson, Shashanka Ubaru, and Elizabeth Yang, “Capacity Analysis of Vector Symbolic Architectures”, arXiv, 2023. ↩