On The Mathematics of Meaning

I have worked with co-occurrence models of semantics for a long time. These computational models try to bootstrap word meaning from analysis of patterns of word co-occurrence in large corpora of text. Recently, Google released a set of tools (word2vec) and associated materials for a new, and very good, kind of co-occurrence model that they have built. There is a nice explanation of the model here.

One of the things you can do with co-occurrence model word representations is subtract or add them, to see what the resultant word representation 'means' (I skip over the mathematical details since we are just here for fun). For example, in word2vec space:

king - man + woman = queen

The equality sign here has to be taken with a grain of salt; it really means 'is similar to'.

My colleague Geoff Hollis and I have been working with the word2vec model (using a smaller dictionary and a slightly different representation and similarity measure than Google). I added the ability to add fractions of representations instead of just adding or subtracting each word representation as a whole, and have spent some time looking for interesting semantic math results. I have defined '=' here as 'being in the top ten closest results' (and also restricted myself by requiring that the final result on the right of the '=' sign cannot be among the top ten closest neighbors of any the input words on the left of that sign). This human flexibility (and the fact that I have deliberately searched for interesting results) means that this math is really a human-computer collaboration rather than a purely computational result. 

Here are some of my most interesting results. Enjoy.

love + 0.4 * sex = friendship
love + sex = infidelity
love + 3 * sex = monogamy

murder + fun = gunplay

apple + pig = potato

cat + 0.7 * dog = poodle

despair + 0.5 * hope = frustration

wealth + 0.2 * dream + 3 * selfish = elitist

courage + 2 * stupidity - incompetence = audacity

hope + time = opportunity

logic + hope = principle

man - 2 * education = snake

tiger - cat = rhino

sex + drunken = debauchery

love + dream = passion

[Image from: Alfred Bray Kempe (1886) A Memoir of the Theory of Mathematical Form.]

On Attaining Our Goals

 Approach the goal. 

         It is difficult to attain 

what is not there.

The words above are one of my favorite JanusNode productions. I like the idea that life is all about striving to attain goals that are really just figments of our imagination. We make up goals, and then our goals make us up.

In my history of psychology course at the University of Alberta we discuss Carl Jung, whose work addresses the weird question that has to be asked: Who made up the process by which we make up goals? Whoever or whatever controls our goal-making algorithm controls us. Jung had a labyrinthine answer to the question of where that algorithm comes from.

 Jordan Peterson's (1999) book 'Maps Of Meaning: The Architecture of Belief' and Elizabeth and Paul Barbers' (2004) book 'When They Severed The Earth From The Sky: How the Human Mind Shapes Myth' both discuss Jung's answer, more or less, from different perspectives. The discussions they each offer are also complex, but include noting that:

• Humans are not very good storage devices so information gets distorted when it passes into our heads. What is incidental fades away. What is important is magnified.
• One way to safeguard what is important when it has to be stored in a leaky human mind is to store it more than once.
• The unknown is frightening and has to be made comprehensible, predictable, and  approachable by speaking of it using analogies to what we understand for sure, notably human needs and desires.
• Analogies using human needs and desires require that the story be 'fleshed out a little', with the analogy-maker adding elements to make the story coherent.
• Similar stories from different sources can be merged into a new meta-representation that can encode the gist of the similarity, giving us recognizable, stable and versatile mythic elements are useful for coherently representing and thinking about what is important.    

When we invent the goals that make us up, we use the cognitive tools that we have. Those tools include some that have been passed down to us by natural selection and others that have been passed down encoded as mythic elements in tales, poems, songs, and legends. 

Who programmed the algorithm by which we make up the figments of imagination that make us up? All of history did.