The Innovation Platform speaks to Yasuhiro Suzuki of Nagoya University about their proposed method of converting the Tactile Score to measure vibrotactile sensations.
Humans have five senses (visual, auditory, tactile, gustatory, and olfactory). The tactile (touch) sensation is not only the first sense to manifest in the womb, it is also the least recognised in terms of technology and culture. If we compare current tactile science with auditory (sound) science, auditory research typically only focuses on the structure of the cochlea and forgets to research the science of music. The cochlea is a nerve organ in the inner ear which converts air vibration into sound. If we consider that music is essentially the variance of sound over a period of time, then no matter how deeply we investigate the cochlea, this will not aid us in understanding music.
Using staff notation, we have invented the Tactile Score in order to describe time-varying tactile sense. Musical scores delineate music in terms of changes in pitch over time, whereas the Tactile Score describes the tactile sensation as changes in pressure over time. The middle line of the staff illustrates standard strength; the ascending lines are weaker pressure and descending stronger; and the musical notes are used to describe the length of a period of pressure.
Science has shown that softness, roughness and temperature are the three major components of tactile sensation. Therefore, the Tactile Score can visually describe both softness and roughness – for example, when you hug someone and the pressure changes from weak to vigorous. Additionally, roughness can temper the complexity of softness. When we walk on well-maintained grass, the tactile sensation of softness we feel by walking is constant and smooth. However, if we walk on unmaintained grass, the softness and roughness varies depending on the location.
Tactile Score vibrations
The Tactile Score was originally invented for describing the skills of hand massage of a skilled technician: a person who can read the Tactile Score can give a massage with a qualified technique. Moreover, the Tactile Score has also been used for the education of beauty technicians and will soon be applied to massages in nursing and healthcare. However, even if a well-skilled technician uses the Tactile Score, it is impossible to provide massages to many people at the same time in the same way that a musician performs in a concert using a music score.
As a result, we have proposed a method to convert the Tactile Score to measure vibrotactile sensations. Since the Tactile Score describes changes in strength over time, it can also be used to depict changes in vibrotactile intensity over time – enter ‘TS Vibrations’, which has enabled the digitalisation of tactile sensation.
Using an age estimation system equipped with Artificial Intelligence (AI), we investigated changes to the physical age of patients’ skin before and after massages conducted using TS Vibration. As a result, an average of 3.6 years of rejuvenation was confirmed in the treatment of women in their 70s. Furthermore, we performed experiments on skin physiology and confirmed that TS Vibration aggregates collagen into the epidermis: we believe this to be the cause of the rejuvenation effect. We started a trial operation of the beauty TS Vibration Internet distribution business in 2019, and the full scale operation will begin in 2020 due to the rapid increase in the number of users.
Deep Micro Vibrotactile (DMV)
Goro Noguchi, a national singer and musician, believes that the secret to delivering music which leaves an impression on people is the use of sound with a very low frequency. Through discussions with Noguchi, Suzuki noticed that there are two types of tactile sensations: direct and inter-contact. Extremely low sound frequencies, such as the sound of fireworks, can transmit tactile sensations to distant places.
In music composition and production, very low frequencies are typically avoided because they can distort the sound. Suzuki and Noguchi conducted joint research and established a method for embedding inter-contact sensation (very low frequency sound) in music. This ultra-low frequency is referred to as ‘Deep Micro Vibrotactile’ (DMV). During an experiment conducted at several concerts performed by Noguchi concerts in different music halls, Suzuki analysed the sound effects of the DMV at each venue, using a unique method to quantify the tactile sensation of sound. As a result, it was confirmed that DMV significantly changed the tactile quality and impression of sound.
Despite various efforts made, there is still no effective treatment for dementia or Alzheimer’s disease. However, medical experts expect DMV can apply to the treatment of dementia. For example, reminiscing (autobiographical memories) is known to be effective in treating dementia. A study into ‘music evoked autobiographical memories’ (MEAMs), published by the University of California Davis group in 2009, showed that music heard at adolescence evoked autobiographical memories. MEAMs have also been confirmed to contribute significantly to the improvement of dementia. In April 2019, an MIT group announced surprising results, with 40Hz flashing light and sound suppressing the progression of Alzheimer’s disease. In experiments using rats, researchers found that amyloid-beta, a causative substance, decreases, and atrophied brain enlarges. DMV can merge MIT results with UC Davis results. Expectations are growing for the novel treatment by using DMV for dementia and Alzheimer’s disease. Clinical research on dementia will start in 2020 with the cooperation of the Japan Medical Association.
Braille Tactile Score for job creation and social participation of people with deaf-blindness
The results from the researchers of Tactile Score can apply to job creation and social participation for people with disabilities. It is difficult to communicate with deaf-blind people because they have overlapping impairments to both their vision and hearing. Therefore, we have developed the Braille Tactile Score, which expresses pressure, length and contact area by numbers.
One of the founders of the Tactile Score, Rieko Suzuki, started the haptic project with deaf-blind people, and noticed a great misunderstanding: the erroneous belief that they were tactile superiors and those without deaf-blindness were tactile inferiors. We opened a beauty salon in Kyoto and Tokyo with a deaf-blind person, Mr Norimasa Kawasora, as Chief Technical Officer (CTO). He takes responsibility for designing, marketing, and teaching massages.
Beyond tactile sense to sensibility
The Tactile Score can also describe sensibility. Sensibility depends on changes in either power or size. For example, if you say ‘good morning’ while raising the volume of your voice in a crescendo, you will be able to convey a feeling of energy. On the other hand, while reducing the volume of your voice, a sense of no energy is conveyed in the diminuendo. The meaning of the language is the same, but the sentiment is different depending on how you say it. In addition to this, any time-series data can be converted to the Tactile Score (referred to as a Tactile bit); and anyone can extract ‘softness’ or ‘roughness’. In other words, any data can be converted into sensibility information.
We are collaborating with artists to expand the possibilities of the Tactile bit (T-bit). At SHOKKAKU 2019 held at Kyoto University in June, the motion of a dancer was converted into time-series data by a motion capture system and turned to a T-bit. A composer performed a real time sound conversion and produced vibrotactile data automatically. The audience enjoys the performance while feeling the sensation of sound on the body from four huge thin metal sheets (thundersheets) and tactile sensation from HDV to the face.
Most conventional AI and information technology rely on logical thinking. T-bit can extract sensitivity from data. Traditional information technology corresponds to the left brain, whereas the T-bit technology corresponds to emotional thinking performed by the right brain. T-bit makes it possible to make information systems more sensitive. Shortly, the system will have a left and a right brain. Why has Tactileology evolved so rapidly? The reason is that we succeeded in the unification of tactile and emotional information.
Yasuhiro Suzuki
Associate Professor
Yasuhiro Suzuki laboratory
Department of Complex Systems Science Graduate School of Informatics
Nagoya University
+81 52 7894783
ysuzuki@nagoya-u.jp
www.ysuzuki.info
Please note, this article will also appear in the second edition of our new quarterly publication.
