Yasuhiro Suzuki, an associate professor at Nagoya University, Japan, explains time related variations in tactile sensations and tactile melodies.
Tactile science has not dealt with time-varying tactile sensations like melodies but have dealt with tactile sensations that do not change over time. We proposed a ‘tactile score’ that allows you to describe, edit, and distribute tactile sensations. It can handle both time-independent and time-varying tactile sensations.
Tactile score
The Tactile score describes changes in tactile strength. The Music score intentionally ignores various elements of music and describes only the change in pitch over time. While the Tactile score also deliberately ignores various tactile elements and describes only changes in pressure over time (Fig.1).
The Tactile score uses a staff notation like the music score. The middle line (third line) of the staff represents normal pressure, below the middle represents strong pressure and above it represents weak pressure. Then, the notes are diverted to the notation of the rhythm.
In these days, people who should be called ‘tactile composers’ compose tactile scores for massages. A tactile composer collects favourite tactile sensations by transcribing to tactile scores and composes novel tactile scores; there emerge markets of tactile scores and the size of market is expanding.
Some beauticians can create beauty massages with the tactile score. They can transcribe and collect their favourite tactile sensations with the tactile score and sell composed tactile score as a new beauty massage.
The benefits of tactile melodies
Marshall McLuhan, the founder of media theory, pointed out that written language and letterpress invented by Gutenberg completely changed culture and civilisation. Tactile scores are written language and letterpress for tactile sensation.
As his insight suggests, Tactile scores soon became used in many disciplines. So, I collaborate with people in many fields. One music producer kept telling me, “No matter how much I hear the explanation, I don’t know what the tactile score is.” As I asked myself, I noticed that sensitivity resides in changes in strength, and the tactile score describes the changes in strength, which delivers sensibility.
Why do you feel the energy and lack of energy from the greeting of “good morning”? The meaning of a language is the same, whether greetings are energetic or not. I translated the voices of greetings into the tactile score and confirmed that tactile rhythms are different. Tactile score of energetic “good morning” changes significantly, while less-energetic ones have little change.
Tactile melodies conveys sensibility. Musical melodies also convey sensibilities such as sadness and happiness, and melodies with different sensibilities have rational differences such as minor and major.
Rational difference of tactile melodies
The quality of tactile sensation is called tactile quality, and it is scientifically known that it is crucial in the order of hardness, roughness, and temperature. Tactile qualities give a rational difference in tactile melody. We have used hardness and roughness for indicating tactile quality.
Hardness is how much it dents when pressed. For example, if it is soft like a sponge, the unevenness will increase concerning the change in pressure, but if it is hard as an iron plate, even if the pressure is changed, the unevenness will hardly occur.
Roughness is the homogeneity of hardness. Walking on well-maintained turf or paved roads feels smooth with no difference in hardness while walking on unmaintained gravel roads feels inhomogeneous and coarse.
In the tactile score, hardness is the magnitude of change, soft is the big change, and hard, is a small change. Roughness is the complexity of the note pattern, smooth is the same pattern repeated, and rough if the pattern is intricate. The tactile quality of the energetic “good morning” is soft and rough, and the tactile quality of the less-energetic “good morning” is hard and smooth.
Melody of tactile in music?
Since volume is the pressure of sound, the tactile score of music is a volume change. The piano is a tactile instrument that converts the touch of a pianist into sound. Unlike string and wind instrument players, pianists cannot later modify the sounds they make. Hence the difference in how to play the piano is the difference in the time change of tactile, that is tactile melody. Therefore, we converted the performances of different pianists of the same music piece into tactile scores and examined the tactile qualities. We compared Frederic Chopin’s “Heroic Polonaise” performances by S Bunion and A Rubinstein by tactile score.
The sound sources we used were Bunin’s incredible performance at the 1985 Chopin Competition and Rubinstein’s artisan performance (recorded in 1965). We separated the sounds every 0.01 seconds, measured Hardness and Roughness, and created a two-dimensional plot with Hardness on the vertical axis and Roughness on the horizontal axis. The difference in the change in Tactile quality (melody of tactile) was evident. The pattern of Tactile quality in Bunin’s performance was almost opposite of Rubinstein.
Melody of tactile in daily life
We created a tactile map using audio data such as outdoor or indoor sounds, whale songs, music, speeches, Kabuki, Sutra chanting and so on. We got exciting results such as Barack Obama’s speech in Hiroshima and whale’s singing voice are similar in tactile quality. Hillary Clinton’s speech and Japanese singer Mika Nakashima’s song “Yuki no Hana” is almost Same Tactile quality (Fig.2).
We can freely choose the quantity corresponding to “strength” in the tactile score. The tactile score can describe any changing quantity, and the tactile quality can be extracted. If the changing quantity is the stock price, we can extract the tactile quality of the stock price.
I extracted the tactile qualities from the time series of daily closing prices of the Nikkei 225 Index Fund from 2006 to 2010. The tactile quality of 2007 showed a big difference from other years. As you know, the Lehman shock occurred the following year. The Tactile qualities of 2018 were very similar to the previous year of Lehman Shook. I joked in the lecture, “We may have a Lehman shock class shock in 2020.” I do not think a causal relationship between the tactile analysis of the stock market and the epidemic of the new coronavirus.
Fine art of melody of tactile
The melody of tactile creates a new art of time, different from music. The change in quantity is the melody of tactile. The amount may be a visual quantity such as brightness, and auditory quantity such as volume, or a physical quantity such as speed. Tactile unifies any amount of change, and the Tactile Score is an intermediate language for all senses.
For example, we extracted the changes in emotional strength embedded in poetry and created a tactile score. How to extract the change in emotion strength from a poet? The strength of emotions is the length of words of each semantic unit; I first divided the poem into semantic units; in natural language processing, this process called morphological analysis. I counted the number of vowels in the semantic unit (morpheme); then calculated the average number of vowels. I defined semantic units less than average as strong, and more, as weak emotions.
We used the tactile score as a dance score, so poetry created the dance. We also motion-captured the dance; then extracted the speed of physical activity and converted it into a tactile score. So, a dance created acoustic and vibrating tactile sensations.
Tactile melody player
Since we transform different sensibilities or tactile sense into tactile score, we want to touch them. We have created tactile melody players; that converts a tactile score into a vibrotactile and reproduces it as a tactile sensation.
We created a face mask type device named Haptic Vibration Display (HVD). HVD does not use a power supply or amplifier and can present sufficient tactile sensation by merely connecting a device such as a smartphone. Tactile Melody Players (TMPs), such as HVD is crucial, creates a new world of tactile content. TMPs gave birth to a world of tactile content, which is just born; no one knows it yet. Only humans in nature do not know the universe of tactile content.
Tactile melody is a common language in nature
The tactile is a common language in nature. Nature designs and interacts with tactile by using various ‘tactile design material and method’ such as DNA, molecular structures, body surface materials and so on.
However, humankind has not developed design methods of tactile communications. The Tactile score gives us a method to create tactile interactions. At last, human will beings to interact with the natural world by tactile.
Post singularity points of tactile sense
History can be used as a reference to predict the future. Futurist Ray Kurzweil, along with data and history, shows that technology advances on an exponential time scale. The tactile score was born almost 1000 years after Guido’s proposal for the music score. As soon as the tactile score was born, tactile score composers appeared. Within ten years of the birth of the tactile score, the tactile sensation has converted into electronic information, and the tactile sensation distributed on the Internet can be enjoyed by Tactile Melody Players, such as HVD.
Tactile score was a singularity point of tactile sense world of humankind. Marshall McLuhan, the founder of media theory, said, “Letterpress is like an atomic bomb”; he suggested that “written language and typography would completely change culture and civilisation.” The tactile score is a tactile “written language and letterpress”. His suggestion was correct, and the Tactile score, a method of describing massage, soon became used in many fields. Therefore, it will fundamentally change the tactile culture and civilisation.
History has shown that the pace of this progress is accelerating. Until now, information technology, such as Artificial Intelligence has dealt only with logic and data. Soon, the Tactile score will extract information and data sensitivity information (Tactile bit, T-bit). Then, T-bit will integrate with conventional information technology, and IoT technology will process information processing of logic, data, and sensibility.
The tactile score and T-bit are essential to sensory information processing. Researchers in cell and biotechnology have confirmed that simple tactile sensations activate biomolecule or cell-cell interactions; however, the response to time-varying tactile stimuli is unknown. In recent years, somatosensory stimulation in the γ-wave frequency band of EEG has reduced plaque in the brain of mice with Alzheimer’s disease, and medical experts expect to treat dementia with tactile stimulation. In medicine and biology, the designed tactile melodies will be important in tactile notation.
Tactile sensation is also useful as an inclusive technology. Very few people have tactile disabilities, and tactile function typically works even with severe dementia. Therefore, the tactile sensation is an inclusive medium of information and communication (Fig.4).
Since ancient times, the earth has been full of tactile melodies. Magmatic activity, meteorology, fireballs, and meteorites have produced powerful infrasound and have massaged the earth with sound and vibration. We have existed with tactile stimuli since ancient times. It is rational for natural systems to make tactile interactions useful. Ancient humans would have made tactile interactions useful. We may have rediscovered the tactile technology that was once used by humankind. This technology must develop because nature is already using it. We launched the academic society of science and art based on Tactile score, Tactileology Society & Association since 2019.
