Towards a formal model of qualia

I am sure that this is not the first attempt at it, but I wish to give a try at constructing a formal model of qualia. For starters, we first need to define what "qualia" means. Most of us know that this is the plural of "quale". But what is a quale? Is it the color red? Or is it the perception of some shape of the color red? (There is a difference of formal interest between the two.)

In this blog, I make the latter assumption. I call the color itself the "hue". And I term the gamut of all colors that can be perceived by a particular consciousness "qualette" (taken after palette, the range of colors). In terms of sound, a particular pitch is the "hue". An actual sound of the pitch is the quale corresponding to the pitch. And the set of all pitches that can be perceived by a consciousness is the qualette corresponding to sound. And I call different senses (hearing, vision, smell, and so on) just "senses". And I call the collection of senses at any given time along with the emotional state at that time, the "state". A non-zero trajectory of states is an "experience".

I want to clarify what I mean by "perceive". This is the equivalent of the term "access" I have used in the previous posts.

Now, the idea that a conscious state can be modelled as a point in some space is certainly nothing original. Let's not jump ahead and assume that this space is a Hilbert space now. But let me give my own view points of what this space is going to be like. I would also try to address the binding problem in the same breath.

Senses like sight, hearing, and touch have a 3-D "space" associated with them. We see objects in 3-D, because of our two eyes. Same holds for hearing (we can roughly determine the direction from where medium and high pitched sounds emanate. This is the reason we have surround sound systems. Of course the spatial resolution is much worse than that of sight.). And of course, if a portion of our skin itches, we know where exactly to scratch. The key point is, since the perceptions themselves are qualia, the aforementioned spaces are the "qualia spaces" corresponding to those senses. The binding problem (actually not a problem as much as an observation) states that the spaces corresponding to the different senses are "bound" together, so that the overall experience is consistent. For example, if we see a songbird in a particular direction, but its singing comes from a different direction, we would get confused. In the case of "out of body" experiences, the senses are bound to each other in a consistent way, but the "mind" moves around. This implies that the mind (or emotional qualia) also have a spatial relation in the same sense as sensory qualia but this might be a "spot" instead of a non-zero volume. I call this "spot bound". We can take the emotional qualia's center to be the coordinate reference, since this is the center of vision of the "mind's eye". I call the space the "common binding space" or CBS.

I think the senses of smell and hunger/nausea are also spot-bound, although the spots are obviously at different locations of the CBS. Please note that by "spot" I mean exactly that - I don't mean a very small area that can be approximately considered to be a spot from everyday experience. An actual way of confirming whether a sense is spot bound or not is to see if spatial information of the particular sense organ is processed/routed in the brain. In the case of smell, I don't think spatial information generated by the olfactory bulb is processed by the brain anyway. So we cannot really figure out from where a particular odor is coming unless we move around. I am not so sure about hunger or nausea. For example, if cells corresponding to hunger are activated in the lower region of the gut as opposed to the upper region, does the qualia of hunger also shift vertically?

So we can construct a space, and the whole experience at any time can be given as a point in this space. This point would move with respect to the "proper" time of the particular consciousness, which may be different from the physical time stream (and not to be confused with the proper time of the theory of relativity). Let us be clear on this. Let's assume two people A and B who both have the same qualia till they pass out (due to administration of general anesthesia) which puts them into deep dreamless state of unconsciousness. If A is revived after 5 minutes, but B after one hour, and there is no clock in the room, how exactly do their respective qualia points move in time? If after waking up, their experiences are exactly the same, they should not be able to tell for how long they were under the anesthesia. Their qualia trajectories should be identical. But this would not happen so simply if physical time is used to mark the points. If we parametrize the qualia trajectory as a function of the "proper time", it should be noted that in periods of total unconsciousnesses, proper time itself is undefined. Hence if proper time is used as the parameter, blackouts would correspond to the zero state of the qualia space, and every conscious stream of experience begins and ends at this state. In the example of the two people A and B, the trajectories would be identical in both cases for both before and after the period of anesthesia, in other words the total conscious experience. The "real time" spent at the zero point is meaningless when one uses the proper time for parametrization. Any attempt to relate "real" time to proper time suffers discontinuities at the zero point. Of course, this does not mean that we have to use proper time only. We can use real time, provided we accept that different qualia trajectories may correspond to the same experience if there are blackouts. Since the mapping of real time to proper time is one to one or many to one, never one to many, any meaningful trajectory with proper time as the parameter would lead to a continuous trajectory in real time without any stagnation points (multiple qualia states at the same real time). Also note that there is nothing that prevents a qualia trajectory (with whichever time parameter) from intersecting and coinciding with either itself or with others.

So how do we go about constructing this space? Since the CBS seems to be common to all senses (including emotional ones), we can annotate sight, hearing, touch, etc. data to each point of this space and construct one giant string. Note that we collapse the data over the entire CBS to a single point of our qualia space in doing so. For senses that are spot bound, we need to add their data along with only one coordinate in relation to the origin of the CBS. This would take care of even out-of-body experiences.

A word on emotional qualia states. While physical states can affect emotional states, it might seem that emotional states don't spill over into physical states. But there is an exception. This is the case of "imagination states". For example, A and B can be undergoing exactly the same physical qualia trajectory, but A can also be imagining a physical state different from B's at the same time. When A and B are looking at a green background, A can be imagining a big red disc in front of him while B can be imagining a blue square. These are different from hallucinating the same. But the imagined qualia can be as good as real ones when one performs operations in qualia space (see previous post) on them. For example, one could remember one's favorite song and start playing it in one's head. But when actually playing it on the CD player, one frequently realizes that the "imagined" version was pitch-shifted by one or two keys. What the person succeed in doing was apply an operator (comparing pitches) operating on two quales in qualia space (one corresponding to the physical state, and the other corresponding to the imagination state). One could also perform the same operations on quales existing solely in the imagination state. For example, I could determine which of the three spots I imagined was the brightest, or imagine a color between two colors which also I am imagining. Note that in the first instance the function result is a non-quale (a logic value) while in the second instance the output is a quale itself.

I wish to reiterate that imagination is different from hallucinations or dreams. If I imagine a red disc on a green background and actually end up seeing one, I would run scared! That would be a hallucination. The physical states of both hallucinations and dreams are indistinguishable from the "normal" physical states, and therefore considered as one. The imagination states are something that need to be included in the state vector, but separately from the classical state. So the entire "normal" physical state space is duplicated for the "imagination" state space. Also it should be noted that imagination states are bound to the CBS just like their physical counterparts. I can imagine only qualia, not the hues themselves (see the beginning of the post).

Does there exist an imagination counterpart of mental qualia? Speaking for myself, I couldn't come up with a decisive answer. I can imagine that I am angry, but I am not sure whether I really get angry at the moment! That would be similar to imagination turning into a hallucination in the physical realm.

Now coming to the representation of actual data. I would deal with only two, sight and sound, in this post. Although this is for the "normal" physical states, it applies equally well to the imaginary states.

Sight

For sight, for us humans with trichromatic vision, each point in the CBS has 3 independent quantities, namely the intensities of the eigenhues. This is all the information that is needed. It should be noted that this is a 3 dimensional space, and the operations the mind is able to perform on quales of sight are mixing operations only. In other words, the resultant quale would belong to a subspace of the input qualia. If someone has access to only two of the three eigenhues (due to a "back end" color blindness), then there is no way the person can come up with the missing eigenhue.

Sound

This is more complicated. At least theoretically, no two sounds are subjectively the same unless the complete Fourier spectrums of them are the same. So, in theory, we need an infinite dimensional vector space to represent the sound of a single point in the CBS!

There is one more complication, which I have discussed in the last post. This is, even though the pitches of all frequencies are orthogonal to each other (since the Fourier spectrum has to match for every frequency), there exists operators in qualia space where a single pitch can generate the entire gamut of pitches. Examples of such operators are "move up by a note" or "move down by a half note". So even though we have an infinite number of basis vectors in this space, fixing a single vector in this space fixes all of them! (One might wonder if two distinct pitches are required - after all, what one individual means by "moving up a note" might actually be the equivalent of "moving up by two notes" to another individual. But since pitches an octave apart have to sound the same, while pitches any closer should't do so, an individual would detect an inconsistency if "moving up by a note" meant a different ratio in qualia space- he would determine the matches come sooner or later than they are supposed to.)

Take the case of the inverted spectrum problem. In this case, even if we are sure that one of the eigenhues are the same for both individuals, that doesn't automatically imply that the other two are not swapped. But in the case of hearing, if we are certain that the pitches corresponding to a single tone are the same for two individuals, then we can be certain (at least theoretically) that all pitches between the two individuals match.

Access vs. platonic qualia spaces

As I had discussed in the previous post, the platonic qualia space for sound would have an infinite number of octaves. For sight, I am not sure if it is also infinite dimensional, or if its dimension is limited (certainly 3 or more though). Birds are tetrachromatic, so possibly the dimension is at least 4. It is possible that each species, even if only trichromatic, has totally different eigenspaces from humans, so one wonders what the dimensions of the platonic color space is. Note that the platonic space is at least as great as the combination of all access spaces (of whatever species). Note that the space refers to a single point of a particular sense bound to the CBS.

Of course, access spaces are restricted versions of platonic spaces, and in humans, for sound, access is limited to between roughly 20Hz and 20KHz. Same with the intensities (for both sight and hearing, and with all other qualia spaces too).

Now if we were to consider the echolocation of bats, it might form a new sense that is also bound to the CBS. So unless we are allowed access to this new sense, we would never know what it is like to be a bat. Of course, the platonic qualia space includes everything, including those of the bat or of any other creature in any galaxy far, far away.

Operators in qualia space

Perception of qualia would be of no use if operations on them don't exist. For example, one can perform relational operators on qualia - for example, one could decide which light is brighter of two, or which sound is louder. One could also perform things like ordering pitches or hues. A tribal who has never come across printing or painting would still instinctively classify orange as being "between" red and yellow. What he does is apply an averaging function in the qualia space of hues.

However there is a fundamental limitation to operations on qualia whose result is qualia. This limitation is what I term "access limitation". This essentially states that the resultant qualia can sometimes be "out of bounds" for the access of a particular brain.

I will give two examples. The first one has to do with intensity. Intensity is the magnitude of different senses (vision, hearing, smell, pain, etc). At lower intensities, one can perform relational operations quite easily. (By performing operations, I mean, ordering sources according to different intensities, or even imagining or dreaming two sources of different intensities.) But at higher and higher intensities, the sense organs become saturated that further relational operations is not possible. Now one can dismiss this as just the limitation of the physical sense organs. But according to my previous post the limitation is built into the access pathway in the brain itself, and with a different brain architecture, a potentially infinite dynamic range can be realized for most, if not all, senses. This is the reason why we cannot even imagine or dream or hallucinate a greater dynamic range than our everyday experience.

The second limitation is even more illustrative. When I mentioned an averaging operator in the space of hues, the assumption was that the individual is not color blind. If an individual cannot experience an eigenhue (see this post for what I mean by it), then no relational or any other operation on the hues he perceives would give rise to that eigenhue. This is why soemtimes color blind people cannot even imagine the color they are missing. This is again an example of access limitation. This is not about intensity, but about the hue.

I will give another example of the second limitation that affects everyone. Take the sense of pitch. Now pitches corresponding to two frequencies which differ from each other by the twelfth root of two are adjacent notes. In fact, one can easily figure out the succeeding note from the current one. This is the basis of music theory. What we do is keep applying the same operator in qualia space ("one note above") starting from a low base note and we very soon find ourselves at the upper end of the hearing range. We can also go down as "one note below", and would then find ourselves at the lower end. In fact, when one crosses the limits (either lower or upper), one cannot even imagine notes beyond these. This is quite unusual because in pitch qualia space where neighboring notes are so beautifully related over several octaves with great consistency, the breakdown seems almost implausible. For in an abstract qualia space of pitches, it would seem that we should be able to carry out the determination of successive notes (either ascending or descending) ad nauseum. A platonic model of pitches should correspond to an infinite number of octaves. Logical reasoning dictates that. And yet that isn't the case. We should blame it on the access part of our brains. Our access is closed to pitches below 20Hz or beyond 20KHz (roughly).

I wish to reiterate that when I mean "access" I don't mean the ability to just sense such qualia (which can be tied to the ability of the sense organs). It means the ability of the mind to construct the qualia under question as a function of some operation in qualia space. In the pitch example given above, the ability of people to determine a pitch which is an octave higher than a pitch sounded to them is the access of the higher pitch. Most people would succeed if the sounded pitch were 1KHz. They would fail when the sounded pitch is 15KHz.

As I had mentioned earlier, the platonic space of qualia (which is post Cartesian theater) is ultralinear (infinite dynamic range), and the vagaries of the access limitations are due to the physical workings of the brain. It is possible that the brains of some other species of animals have different access limitations and may or may not overlap the regions reachable by the human brain.

On the linearity of qualia

I am wondering if qualia would lend themselves to a Hilbert space kind of formalism. But one of the main requirements is linearity. In our everyday experience, qualia are not very linear. If a spot of light is kept on made brighter, it would saturate our senses after a certain threshold. In fact, we cannot even imagine a brightness beyond a certain limit.

But I think that a fundamental distinction needs to be made between phenomenal and access consciousnesses. The latter is a result of processes in the brain (pre-Cartesian theater), and it doesn't surprise anyone that there are fundamental non-linearities built in the chain (which can be explained physically).

Imagining (as well as dreaming or hallucinating) anything is also subject to the same access chain, so this is the reason we cannot even imagine a bright light or a loud sound which is 10x what our sense organs are designed to withstand maximally.

But I think that p-consciousness (which is post-Cartesian theater) should be super-linear. This doesn't hold just true of physical qualia but also emotional ones. While emotional qualia (like anger, empathy, or even sex drive) may seem to be purely access driven and not as straightforward as physical qualia (like colors, pitches, and so on), I think they are as elegant and straightforward as the physical ones. A theory of qualia is incomplete if it leaves out emotional qualia solely to the physical workings of the brain. While the hook-ups to the emotional qualia are again access-driven and hence non-linear, the "other side" should be ultra linear. In fact, I think meta ethics should be of much importance as the color inversion problem. Some entheogenic drugs like Ecstasy in fact cause a heightened state of empathy. There is no reason to rule out, in theory, a super-high empathy state (> 1000x), which would solve the world's political problems! And imagine an orgasmic state which is 10000x the regular one!

Of course, one problem is how to quantify emotional qualia to begin with. What does it mean to say some one is having 453x the empathy of another person?

If we assume linearity of the p-qualia space, then the brain maps processes to a certain subspace of this space. The complementary space would have qualia which cannot even be felt or imagined or dreamed of. For example, we do not know what it is to be like a bat, whether they "hear" or "see" or do something else with their echolocation. If they use the Doppler effect to determine the speed of insects, how does this info get mapped? Would it be analogous to color of "our" vision?

I think we should assume linearity of p-consciousness as a first step towards a formal model of qualia.