...professor of food development at the University of Turku and welcome you all to our webinar 'Do science art and technology spice up our future food experiences.' This question was raised by a group of researchers and artists from different disciplines of food science, technology and art, as we were discussing the future visions of food experience and dining. Technology is an increasingly diverse part of our lives and the dining experience is no exception.
However the essence of eating remains unchanged when we eat we share a common experience and we are together present in a unique moment. The moment of dining is the core element of our community and this is also the case in the technological world. We raised the question how to harness technology to serve that age-old meaning of communal dining.
Could the combination of technology and art provide an experience of a shared meal with a friend even if we were sitting alone in our kitchen on the other side of the world. How could we enhance the feeling of communality and commensality by combination of technology and art? Maybe transferring the movement of the fellow diner's spoon into shared soundscape and then further into the tactile sensation of your neighbor's body? And we should also learn how do these new technologies shape the sensory and emotional experience of eating and dining. For example; how does cheese and wine flavored with virtual technology taste like? And during the autumn of 2021 our group has been working on these new spices of future food experiences, focusing on communality of dining and its importance in the digital world. And in this December webinar we sum up our results so far. We have chosen fermentation for our artistic forum and virtual and mixed reality dining examples, because fermentation is the technology, that humans have used to produce foodstuffs and varieties since the neolithic age. And we want to acknowledge this ancient technology and connect it with the most recent ones.
You can see here the program of the webinar. The webinar is divided into two sessions which both are divided into presentations on science and then presentation on art and technology. And after each session there will be time for questions and discussions and for the webinar participants the chat is open all the time for questions and comments, and they will be shared during the discussion. This first session is entitled 'From neolithic times to virtual and mixed reality dining' and it takes us from fermentation and the world of microbes via human sensory experience into the possibilities of virtual and mixed reality dining.
And by these words I welcome our first speaker, professor Lucrecia Delfederico from National University of Quilmes, Argentina, to take us to the world of microbes, Lucrecia, please. Thank you very much for having me here today. We are talking about fermented foods, the other domestication, wine and cheese. The domestication of plants and animals during the neolithic allow humans to organize certain societies and develop technological innovations for the treatment and conservation of food, cooking, drying, salting, smoking and of course fermentation. The fermentation provided new foods with a wide range of flavors smells and textures, and in many cases more digestible than starting materials.
There is evidence of great wine and viniculture, that cames from georgia, in the south caucasus regions, dated around 6 000 years before Christ. It is the earliest biomolecular archaeological evidence for great wine and viniculture, as we know it in the west, and it is crucial to the later history of wine in Europe and the rest of the world. The oldest known evidence of cheese making are pieces of pottery full of holes discovered in Poland, of about seven thousand years, with remains of dairy fats in them. The first farmers have used this object to separate the curd from the liquid whey. Simultaneously and most likely by chance the domestication of bacteria, yeast and molds also took place, mainly through artisanal food production practices, such as the cereal reinoculation of new foods with materials from previous food.
These practices led to the selection of microbial communities adapted to the characteristics of the food to be fermented. Neolithic humans so harnessed the metabolic capabilities of microbes while they were trying to improve the digestibility, palatability and self-life of their newly abundant food. So fermentation is one of the oldest biotechnologies applied to meat, fruit meat and grains to transform and preserve them by microbes. Despite the fact that wine and cheese are such different food matrices they both depend on interaction between microbial communities for their development. The structure or compositions of these microbial communities differ between both fermented products, but a group of micro-organisms plays a fundamental role, lactic acid bacteria.
The basic manufacturing process is the same for all cheese, actually it's a dehydration process of milk. Rennet and starter cultures are added to the milk, the milk coagulates, the coagulum is cut and the whey drained. The remaining curd is pressed in the form and allowed to ripen.
For fresh cheeses the curd is cut and drained, but not pressed, and they are not allowed to ripen. What actually differentiate cheeses and strongly influences the final products, is the use of different starter cultures of micro-organisms. The presence or absence of mold, and the method of weight separation and ripening methodology.
Also variations in size, shape and all other ingredients added lead to the local production peculiarities and almost infinite variety of cheeses. According to their role the microbiota involved in the process of cheese manufacture and ripening are starting lactic acid bacteria, such as mesophilic Lactococci or Leuconostoc, thermophilic strains of Streptococcus thermophilus, which are typically used for the production of semi hard and hard cheeses, and adventitious non-starter lactic acid bacteria, mainly Lactobacilli. The starters are responsible for the acid development during cheese production and contribute to the initial ripening process. The non-starters have been shown to play a contradictory role during ripening by enhancing flavor and sometimes deteriorating cheese quality. Flavor develops in cheese by the combining metabolic activities of microbial communities on milk fat, milk proteins and milk carbohydrates.
And the main kind of carbohydrate is the lactose. The depletion of lactose in cheese curd is essential to avoid the development of undecidable secondary micro-organisms which will negatively affect cheese quality. The citrate present in milk constitute a secondary energy source for some LAB (lactose acid bacteria) and the co-metabolism of lactose and citrate leads to production of carbon dioxide and aroma compounds such as the acetylene. The major LAB metabolic pathways involved in cheese flavor formation are metabolism of lactose via glycolysis, metabolism of citrate and metabolism of the fat.
For lipolysis and proteolysis and the subsequent metabolism of amino acids. In some cheeses secondary microbiota is added to enhance proteolytic activity during ripening. They are molds like Penicillium camemberty, Penicillium roqueforti and Geotrichum candidum, and also bacteria as Brevibacterium and Propionibacterium. Moving on to the wine, it consists of around 86 percent water, 15 percent alcohol and 1 percent secondary metabolites.
This complex mixture of chemical compounds define the wine appearance, the wine color, aroma, flavor and mouth-feel properties. Sensorially active compounds originate from the grapes, from microbes which come from the soil of the vineyard and from the winery and, when it is used, from oak. Grapes and grape must naturally contain mixture of microbes. During fermentation there is a sequential development of some yeast and bacteria. And once anaerobic conditions are established in the vat or tank, nutrients become depleted and the concentration of ethanol increases. Such conditions favor Saccharomyces cerevisiae with the most efficient fermentative metabolism that are able to withstand low PH, high sulfide and high ethanol environments.
A widely used practice is to inoculate the grape must with fast fermented strains to ensure a rapid and reliable fermentation. However by doing that, the desirable yeasts originating from the grapes, have limited opportunity to contribute to the complexity of wine Thus to preserve the regional characteristics of the wine some wine makers prefer to conduct spontaneous fermentations comprising mixed and sequential dominance of indigenous non-Saccharomyces and Saccharomyces yeasts. A wine yeast converts grain nutrients into ethanol, carbon dioxide and secondary metabolites so it's glycolytic metabolism, which impact aroma, that can enhance the quality of wine. Some grape derived compounds, such as acids, including L-malic acid, and polyphenols, remain largely untouched by the yeast cells while other compounds are fully generated. A secondary process that can improve the quality of many red wines and some white and sparkling wines, is malolactic fermentation, carried out by lactic acid bacteria during or after alcoholic fermentation. Malolactic fermentation is in fact a decarboxylation process, in which malic acid is converted into carbon dioxide and lactic acid, resulting in the acidification, flavor modification and microbial stability of wine. Oenococcus oeni and Lactiplantibacillus plantarum
are the main responsibles for this process, since they are the best adapted species to the harsh conditions of wine. The biosynthesis of aroma compounds during malolactic fermentation includes the activity of various enzymes present in LAB, such as beta-glucosidase. Malolactic fermentation improves wine flavor by reducing herbaceous notes due to six carbon alcohol content and enhance the fruity aroma increasing ester contents. Also malolactic fermentation often occurs spontaneously by action of lactic acid bacteria from grapes and cellar, it implies fruits such as consumption of residual sugar and formation of undesirable biogenic amines. The use of indigenous starter cultures best adapted to
a specific wine production area is recommended to maintain the wine regional characteristics. In the wine industry the growing demand for distinctive regional flavors has then led to the concept of the one defining that the set of physical and biological characteristics as well as local viticulture and winemaking techniques, which together determine the sensory attributes of the wine. There is evidence that microbial activity is associated with wine terroir. Nowadays we know that the bacterial and fungal communities of wine are shaped by the grape variety, by geographical and climatic factors, and by wine resources. Thanks to next generation sequencing techniques, that have been used to monitoring seasonal change in the microbiota present in vineyards and wine, this technique allow to prove that the consortium of fungi and bacteria involved in wine fermentation correlates with the chemical composition of the finished wine. And the last word can be applied to wine, to cheese and other fermented food because it is a collaboration. The collaboration between nature and culture express
the characteristics of the region and the history and culture of the people who make it. Thank you. Thank you very much Lucrecia for introducing us to these fellow microbes, that make the food delicious food for us. And now we go into the deliciousness from here, and I invite professor Mari Sandell from University of Helsinki to share her experience on multi-sensory experience and especially cross modality. Mari, please. Thank you.
So hello everyone. I'm professor in sensory food science and then also in sensory perception, both at the University of Helsinki and University of Turku, and I've been dealing, my sensors are like human senses, so I have been dealing with my group called Senses and Food in this area already quite many years, and it's very very fascinating. And I try to activate you very different ways now in this next 10 minutes that I do have my speak. When we are talking about multi-sensory perception, this is how I understand it, so that we do have five senses, which are activated different ways, and the objects or the stimuli, stimulus, it may be food, it may be both, in case of food, it may be also eating environments but it may be any other object that our senses are perceiving. And we may have five senses or then we we may have less, so it depends on the person, and this is very individual. Our senses are working very different way, but in many cases, especially in food there are several senses that are activated together and they are really absorbing the things from the food. The perception part is pretty complex, and for different senses
there are different ways how they are processing the information from the object, and how they are processing the information in our brains, so it is very complex. And maybe more complex is the way that how our experiences are really result. And maybe in case of food quite often the wish is that food will create such kind of woe effect or at least the pleasant effect something that I really remember for a long time as a very good experience. But it may happen that the experience is also negative or it it may be even neutral. and it depends on the person as well and this is very complex there are very different kind of emotional factors that are also activated and we can create also the emotional profiles for different experiences.
Now I'm activating you with one photo, which is one of my favorites, some people scare it some people think that it's very ugly, some people think that that's something that I would like to try, so let's see that's how you are now perceiving this visual stimuli. Think about this nice plate of vegetables. There are different kind of colors you can see, very different kind of plants and it's very healthy, it looks that it's something that we are supposed to eat. Some people may see here a happy face, some people may see here that there is
something very weird going on. The question is that to eat or not to eat, and people are reacting in very different ways to this photo. It would be nice to discuss with you about your reactions as well. Flavor is something - the word flavor is missing here - here i'm explaining that how the flavors are actually formed, so the flavor which in many cases is confused with the name of taste it is actually the combination of retronasal smell, taste and chemesthesis. I have been studying strawberry flavor quite a lot, and if we think about it from those factors that are contributing to flavor, there are different sugars and different acids and very different kind of volatile aroma compounds and also polyphenolic compounds, and there are different kind of smells and and tastes and chemesthesis when we do have the strawberry in our mouth, so flavor is truly the multi-sensory perception, it's an experience.
We have studied, in the project funded by Academy of Finland, in my group the sensory perception with Finns, and this is an example where we ask people to look at different kind of colored bottles and we ask them different kind of questions, for example put them to order, how sweet you think that the samples are, how sour they are, and this publication is a demonstration about the fact, one of those publications, that how by our sense of sight and different kind of colors are activating also the taste in our mind. And one point in this publication is also that those reactions are different, so we are not similar as human beings. Another publication, which was published last year, was dealing with food consumption and salad buffet, and we applied here the multisensory environment so we had the control environment, control space, and then we did have a forest kind of environment, where people were selecting and eating the food. And actually we didn't see that great impact on the environment in this food choice and the buffet choice, you can see the buffet there in the photo, there were 14 different kind of food items that people were allowed to select, but what was interesting that when we asked also the emotional profiles for these different conditions actually in the case of the forest landscape, people felt that they were more happy while they were also eating and selecting and eating the food, so even more complex combination of both the food and then the environment. Then into the end of this talk I selected several publications, that have been published this year, concerning the multi-sensory perception and different kind of food items. For example there is one publication that's
study topic was beer, very fascinating, quite different, by several researchers around the world and then another publication from Denmark how the visual attention and sounds are contributing to the consumer food choice. Then cookies, and dining-out during corona conditions. These are all public publications, so just go ahead and read more about them. I'm not going
to the details, but showing you that what kind of different things are published during this year. Sense of smell, so olfactory censory, and then the audio visual integration, it's very fascinating area for researchers and there is one example of those. Then one example of the odor, taste and texture interaction, and how this can be connected to the health issues, such as overweight. Both the children and adults, they have been studied because children and adults may also differ a little bit, and then let's remember that there are also individual differences. Then the taste and smell, and how they are contributing to the combined flavor, which was published just a couple of months ago. Then the third slide still about the visual and taste, their connections. Sweetness is always pretty, pretty interesting. How to enhance the sweetness without adding sugar
And here we're also a little bit going to the virtual reality world, that we will hear in the next presentations much more, so I'm not going into details in that. These are very nice examples, all of these, that what all kind of activity we have in the area of the science concerning the multi-sensory perception and also related to food. That was shortly my presentation, my 10 minutes. Thank you for your multi-sensory attention and I wish I activated you also with this photo from the real life, a lot of food, a lot of colors, and I wish that there's at least something for everyone. Thank you. Thank you very much again for your inspiring presentation on multi-sensory perception in a sort of, in this traditional world. It was interesting to hear that multi-sensory perception and cross-modality take place also in the real world, and now we move on to doctor Jean-Christophe Sakdavong's presentation from University of Toulouse and we hear state of VR and AR and so forth. Please go on.
Thank you. So I have a PhD in Computer Science, but I'm researching cognitive psychology and ergonomics, so it's a specific profile. And then we try to make a fast state of art part of what we can do with virtual reality, augmented reality, mixed reality, for future food experiences. So I will start first by speaking about mixed reality. Then the factors which have been identified. I will do as my colleague, you'll see a little spectrum of publications and the researches which have been done. Unfortunately the only one she selected is the one I was not using,
but I know it and we'll see a bit what we can do in the future. So first mixed reality. About, first of all virtual reality, I think everyone knows what it is now. It's usually a computer generated simulation, it's in three dimensions and you have most of the time a headset.
You can include in virtual reality use case watching 360 degrees videos, and it's quite used in the sensory researches, so that's why I speak about it. Augmented reality, at the beginning it was simply adding information to the real world, the same as your car display the speed you are driving for example. This was the beginning. After we have started to add 3D virtual objects, and what we call currently mixed reality, we could call it augmented reality, but with evolution of technologies people have tarted to speak about mixed reality, which is a mixed and an interaction between virtual reality and augmented reality. And it has come with the most modern technologies as you can see. Here for example with Microsoft Hololens, where you have glasses, which are adding informations.
But you can also have it in a simple smartphone using the pass-through camera and watching the real world with other things, and also with virtual reality, when we have a pass-through camera. So, if I try to use a reference, I can speak about the virtuality continuum. and if you see how the things are coming virtual, you can start by the real environment and the more you become virtual, it's like this, so you are in reality, you are with a virtual headset and you can augment the reality with information, three objects. You can also see it in the other direction by augmenting the virtuality by the real world. So for example a virtual world with a small window like this one and it will be a kind of augmented virtuality.
This is the usual use, and usually people see this with only two senses, the visual and the audio. So you have a real environment with everything but they take care more of visual and audio elements and you add visual and audio elements. The first augmented reality and the beginning of mixed reality it was the museums with these headsets, which are telling you the story without the guide, but it can be also visual. So you see it's about visual and audio. In our case we are a bit more interested in real physical elements or reality, it could be food or dreams. So we can augment the real world with our virtual elements or we can augment the virtual world with food elements, with drinks. And it is more what are doing in one side, and the other side it can be like this for example. Here you can change the appearance of food.
Or here you can make a virtual world, where you are navigating and grab some fruits in the trees, for example. And this is a bit where we are going, what is interesting to me at least. And so now I will speak about more cognitive psychology factors that I have studied and which I think are important for sensory elements. The first one is immersion. And immersion is, to understand it, I mean the proper way, the right way, which is as an objective factor, which means it's not depending on the person. It's depending on what you are using; a big headset, a small phone - this is how you can describe the degree of immersion - and a speaker, but I can hear everyone, or a noise cancellation headset, it will change my degree of immersion. And so we can do it well, it's only depending on technology .
And many researchers are speaking about it, mixing it up with the feeling of presence. The feeling of presence you can describe it as a sense of being here. If you feel you are in this world and it's not depending on how much the technology is good, it's a bit more complex and it's a more individual. It's the same as for food enjoyment, the emotions that we can get. And the most recognized researcher in cognitive psychology about it, Lee, has described with three main dimensions, spacial presence, the social one and the self one. For us the social is not so important here, except if you want to enjoy food with other persons, but it's not the main.
But the special, and the self is most interesting, the special one it's about the physical world, if you feel it's real, I mean feel it's real, you can be in the cartoon, but you really feel you are in a cartoon and that is not fake. And the self dimension, it's about feeling my hands are my real ones, for example, so it's about being here. Also it's about you. So I think it's important to have this in mind. The technical dimension, the acceptation of technology. If you put a headset on the head of someone, but it's two kilos and there are wires everywhere, it will not work.
And it's important to measure it in order to know if it's your experience which is good, bad or if it's simply some technology, which is not accepted because people don't like it. I will not enter deep in it, but you have models to check it. And then I will tell something which is obvious for multi-sensory researchers like most of you, but which is not for everyone, and especially for cognitive psychology researchers. It's to take into account the idea of embodied cognition, but you are not a computer with a brain and the brain is all, but you think, you learn, you feel all the things with your whole body. And I know for most of you it's obvious, but it's not for the people, and maybe for computer science researchers it's not also because they see people as computers. So this is to take into account also. So for us, I would tell if you want to build a
sensory experiment with mixed reality, keep in mind to take your self-presence, feeling that it's really you, your hand, but the thing around you seems realist, at least real for you [missing audio] accept the technology, the acceptance of technology and about the embodied cognition to have congruent moves when you eat, but it seems that you are grabbing the real things, but you have your tactile sensations which are real, because there are a lot of experiments which can be done, and if you train for example people to grab things in virtual reality, but there's nothing, and after you put them outside the box and they are not able to open the door to grab a glass because they have learned to do it in virtual and we don't know more how to do it really, so it has to match. And that's the difficult thing. Here the VR headset can also be tricky because of access to mouth is difficult, so many people are using straws for example, but the straw, you know it's cutting a part of the sense and so you don't feel, maybe for Coca-Cola it's okay, but for a nice wine you will lose some parts of the sensation. So I think it's important. The same, you have to know where is the food and so it's difficult to know where are the cups and people who will see it, or experiment this afternoon, will see that with cups it's not yet perfect. There are tricks to put trackers under for, from the virtuality headset
on the plates, on the dishes and so on, but if you have a tracker on a fork, the tracker is much bigger than the fork, so it's not very convenient. And if you use a mixed reality headset but you can see through, like Microsoft Hololens, it's good, but you cannot change the environment. You have a very small field of view and so you can change the appearance of the food you are eating, but not the environment. So it can do some things but not everything. Here you have a long list of the studies I've found the most interesting, and what you can remark for it - and I was not searching to have recent studies - is that the oldest interesting I found has three years. So it's all very recent. If we focus a bit on the topics
they were doing experiments with blue cheese, with wine and beer, with wine, coffee, lemon cake, juice, chocolate. So you see a bit about what we are studying. They have changed environment, and half was to see if it was changing the perceptions, and the other half was to see if we could replace sensory booths by virtual realities to have lower cost during researches. Another thing that you can remark, is that they are most of them with quite old headsets, which are no more used, no more sold since years now, so I will tell you in the next slide, but yes, many nice things are to come because all these studies are with, not all, sorry, but most of them are with old headsets. And a lot of them were using not a digital environment but 360 degree videos, which is nice, but you are not interacting with anything, you are immersed in a movie, but it's not giving all the perceptions and I think about presence, it's not so high. And there are a few modern headsets even if the year is not so modern. So the main conclusions from this is that the replacement for sensory booths are really working, changing the environment of the food seems to have significant effect, but there's not so many studies currently.
And this I already told you, so I think many interesting things will come this year, next year, and we have nice things to see and to experiment ourselves. I will finish by what we can do. We can, I would like to improve food enjoyment by changing environment. If you can eat some, I don't know, some, the example is always bad, but I always have this one, you can eat some sushi, maybe in Japan or with an environment, a congruent environment it can enhance this pleasure and if it's working, why not to do it. It has been told, the idea of changing the sugar perception to help people to have a better diet. Can be really interesting you can do the same with the fat, with the salt, maybe we can help the environment by making people to eat less meat so there will be less dioxide carbon.
So there's a lot of things that we can do with this. And the coming things are kind of experimental headsets, so I don't think they will come for everyone but to experiment it's interesting. There are virtual headsets which can give you sense, which can warm you, send you cold air, so this this can be quite interesting.
Yeah, thank you. Yes, thank you Jean-Christophe. And I think now we move on into art and technology part of this. Jean-Christophe: That's why I stayed. Anu: Yes exactly, and that is why you you stay and we invite also Bruno Mesz, from National University of Tres de Febrero, Argentina.
So please, now we sort of combine all that theory into art and technology. The floor is yours. Thank you. So thanks for having us, and especially thanks to professor Anu Hopia and Seppo Salminen and Kone Foundation, that made this possible.
And I will start with a little story of how we started to collaborate with Jean-Christophe and how we designed the sort of experiments in mixed reality learning. And previously, before I met Jean-Christophe, I was investigating the phenomenon known as sonic seasoning. A little bit in the line of what Mari Sandell told before, that is how music influences the taste of food, in particular of wine and coffee. And we did some studies using temporal methods because music changes in time and wine changes in taste in the mouth and time also Using temporal methods from sensory analysis. And we found interesting things, the ideas, for instance
you taste a wine and you select an attribute, which is the dominant attribute of the wine for instance for a red wine we have this kind of attributes, and according to what music was played to the people in the experiment or if they were in silence, the proportions and the timing of the emergence of different tastes of the wine was really quite different So we did something similar with coffee afterwards, that is drinking coffee with different kinds of music, that was thought to be congruent, cross modally congruent to bitter taste or sweet taste. And we saw that there wasn't not much sensation transference from, that is the sensation of the music, didn't go too much, it didn't affect too much the combined sensation of coffee plus music but there was a very important emotion transference that is the emotion of the music was really impacting the emotion of the combined experience of drinking a coffee listening to that music. So when we design some devices for doing this kind of experiments like interacting with glasses, the sound when you drink from them. So you synchronize the taste with the sound or a smell organ that you can play and launch smells and because we also investigated smell and music experiences. So because I work
in an art and science center so we are always trying to combine both kinds of investigation, research and artistic experiences. So when I met Jean-Christophe, virtually because of the pandemic, I was thinking of this impact of the atmospheres on taste and in general the impact of for instance visual atmospheres on musical experience, so we thought of a study in virtual reality playing the same music in different environments, for different seasons, the summer or the winter, and having people to write the emotions of the music and we haven't done this yet, we want to know if this impacts their experience of the musical emotion or something like this, people will can I enlarge, no, but you will see anyway, no problem, so you play music in a summer environment. Write the musical emotions, as you see, there is some evaluations of emotions. And then we thought of combining this also with food, so what happens if we have contrasting atmospheres and there is some cross-modal congruence between the sounds the visual environment and food, what will people, how will people experience the same food or experience a food in this kind of environment. So this is the experiment that we will present today, and for the, let me play, Jean-Christophe: You want to play it? Bruno: Yes. Like this.
Jean-Christophe: Do you want the first one? Bruno: The first one please. This is one kind of environment and the other one They are contrasting. Let's go further okay. And in the meantime there was this experiment that we did with Sami Silén and Gabriel Vinderola on what happens if you play sound to kefir during the process of fermentation.
This sound has some effect on bacteria, so also in the experiment today we will have environments related to bacterial, microbial environments like these ones, one for wine and another for cheese, so to play here This is for cheese. Okay, so these sounds were designed by Sami Silén and myself. We have some semantic relations with the food, I don't want to anticipate too much because there's an experiment involved in this, and also the videos were made by a Dutch artist Rob Verf, they are called 'being inside a piece of cheese' and 'the expressive life inside my plastic body', with a 360 camera, and they try to express the sensation of being inside, the like assuming into the bacterial world.
And with this I give the word to Jean-Christophe. So now I will tell a bit more about the technicals, but the same as Bruno we will not tell things which can bias the measures that we'll do this afternoon for the presencial participants who will do it this afternoon. So I'm on the technical dimensions. There are, we will think of this mixed reality, so we synchronized two worlds, the real one and the virtual one. First of all there are two parts an artistic part and another part which is not artistic. The artistic part is with red wine
so here you can see real water because Anu was not drinking wine at this time and you have real cheese and the wine was chosen by Delfederico who spoke just before and we were having sort of a virtual reality part with realistic appearance with the cube of cheese and the glass of wine. And then when the people and when you will pick up cheese or take a glass then you will be immersed in the 360 degrees videos, that Bruno has described and shown to you. So here you are in the one from the cheese's video made by Rob Verf and you are also immersed in a sound, in an audio environment which has been chosen by Sami. And also, so this is the artistic part. On the second part of the experience you also have food it's
a bit small in the picture, but you have here food balls, I will not tell you about what it is, these have been made this morning by, I guess or yesterday, by Nanna Rintala and so this is a reward and again we have a digital virtual reality environment, we have shown it a bit, I will not tell so much, but it's looking like as if you are in the space and you are completely immersed, you can see planets and so on. With also sounds, which should be congruent with it. And we have a synthetic appearance of the food ball. So in contrary to the, from the previous part, it's not realistic. It looks like something else. Here it looked like little asteroids for example. The difficulties of doing something like this is, that you have a real world, you have a digital world.
here you can see what's inside the headset for the participant and it has too much at a millimeter position. And this is really hard, current headsets are doing it, but it's not so accurate so when you will do it, maybe you will have to search a bit your food, be aware about it. In our setup we are doing something that, for example we are using Meta Oculus headsets and our programming setup, but I do, yesterday an experimental one has been released by Facebook to experiment for developers only, so we are in advance from them, so that's why it's still a bit tricky and a bit fragile solution. But it's coming, it's coming fast, but that was the main difficulty, and to have markers and really to make this synchronization between the two worlds. Here you can see better, my finger, the cheese I'm sorry, in fact the people, the remote people don't see the marker, so here you have my finger over cheese and here you have the same thing so you see it's matching very well but it has for you when you press the cheese you have to feel it at once so it's at one millimeter maximum but it has to be accurate, and this is not so, this is really a difficult thing For the the glass and the food to be grabbed, so you have to take care and the same when you pick a glass you pick it in real, but you want that in virtually it's big, so it's difficult to detect and sometimes it's difficult to release. So for the people who will do it
think when you grab your glass to open widely your fingers, else you will have released the glass but it will stay with you in Virtual Reality, and you will not know really what to do. The same for orientation in the experiments I've described, there are a few ones who have sensors, a bit like in the one Bruno has shown about in the video and these sensors can help to know the orientation of the glass. We are not doing like this because we don't have this kind of sensors with the Oculus Meta headset, and so we do with the fingers but your fingers are moving near you the less the heads that see them, and so you have impression the glasses will be shaking, but nothing is, no illness, neurological illness is coming to you, it's only the headset. So just trust your fingers, you know how to deal with a small small glass. So yeah, I'll show this here it's perfect, but it's not always perfect as in this photo. So this cool experience experiment has been designed by me, Bruno and Sami. This afternoon we will do a kind of preliminary experiment where you will experiment
it as you have not been biased too much by us, I hope. We will do some measure, there will be a few questionnaires to answer. Anu: Question time, yes thank you very much, Bruno and Jean-Christophe. These are really for the webinar audience. You have learned these experiments now by this presentation and
and those who are present here will have the opportunity to experience those here on site, and in real life virtual reality in real life. Now we have a few minutes time to discuss about thoughts and impressions on about this session number one. I would like to hear your opinion on the, you know, the near future what is the near future, you said in your presentations, Jean-Christophe, that the development of technology is really, really fast. How do you see, what kind of, now we probably - at least what I know - the applications in this VR, mixed reality applications, they are a lot in entertainment. They are probably a lot also in the business world, in which we do not now go, maybe you can share that too. How widely these technologies are applied now in the different sort of platforms, whether it is, well industrial development work or health technologies, what about this idea now, when we talk about this VR dining, and you for example mentioned that might be a good tool to help people to improve their diet, for example or that can be used for people to sort of enhance motivation for example for sustainable choices and so forth, how do you see that, you know, future? How close is that kind of or future at the moment? You can come here, yes, and you don't need to know but it would be nice if you guess.
Anu: Because you probably have the best guess. Jean-Christophe: First of all, it's really very used now in industry, in avionics, in medical, they are using virtual reality, mixed reality, but it's where it's important, where we have money, so we have very expensive headsets. It's also very used in entertainment with cheaper headsets, like the one we were using in our experiment, but the market is mature enough to make light headsets, like a bit big glasses, that will not disturb so much. So it's really coming fast. And for example
one month ago a very big Virtual Reality society, HTC has made a new headset which is for entertainment, meditation, it's very light and so on, but unfortunately they have used all technology and it will not be accepted because it's too expensive for something to all. But they are ready, next year maybe we will have a later one, and so I think it's coming fast. And in the same idea we are, there's a lot of how to do sport, to meditation and I think to have nice dinner in another environment, it's also coming far if we develop it. So I think it will come fast. Okay, so you think like next April we will all be dining together in virtual reality? Not next April, but maybe it will be feasible in two, three years. In two, three years it's really possible because it's going fast and it needs something acceptable. That's why I was
speaking about acceptance of technology, which is light, which is not disturbing you, would have no wires because you don't want to eat with a wire... Anu: Yeah, surely. And at the moment really the food experience in virtual and mixed reality world, it is very different from from our conventional dining so we probably need to, I don't know we should be ready to accept new experiences. We probably should not, this is my feeling that we should not try to mimic the conventional dining experience, but rather take it as a new one, extension or expansion of our experience.
Jean-Christophe: I think I have forgotten to tell something very important. Anu: Yeah, do you want to come here? Jean-Christophe: Yes, maybe I have forgotten something very important, everyone is speaking about metaverse currently, and this should be an important part of the metaverse, if the people will join in virtual reality in all the world's environments and if they can also have dinner together. It will be kind of a, sense more and wonderful experience, if you can have a dinner with your friends, which are at the opposite side of the world. So we move on to session number two, which is on commensality, and actually we go back to the world of microbes to learn about commentary of microbes and the senses that they do on commensality, which we don't. And we have in our artistic artistic examples, we have them try to learn from microbes. But before that I'd like to invite professor Seppo Salminen
to tell us about what is quorum sensing of microbes. Seppo Salminen from University of Turku, please. Thank you Anu. Thank you for the introduction and now we go from dining of humans, to dining of microbes. And that's actually telling you something about the two and a half kilos at most, one and a half to two and a half kilos of microbes every one of us is carrying in our gut. And those are the microbes that are discussing with us, and we are discussing, we are responding. And if they are the right microbes we don't have to think about them,
if they are the wrong microbes, we feel quite quickly that something is wrong. So, quorum sensing in the way is the process of cell to cell and cell to human communication in bacteria. As said, bacteria will talk with each other, that's the quorum sensing part. But they will also talk to us, through our immune system, through our gut, through epithelial surfaces, and we respond. And it's quite common to forget that we also respond to those questions. So for a long time we believed that the bacteria are just quiet, silent and not doing anything. But in actual fact the more we have, the more refined methodologies we have, the more we understand that there is intercommunication, high level of intercommunication between the two.
They produce signaling molecules and these actually help us to understand the communication, but they also you define how the bacteria act. And this system is called quorum sensing. So, regulation allows bacteria to have the so-called social life based on the control of the component secrete. So, all of this is actually taking place by signaling molecules. And what we are trying to do we are trying to understand the change in concentration of signaling molecules and how they interact with our genetic transcripts, and the transcription of other microbes. This interaction is actually seen in many gut bacteria. It's also seen in bacteria that we use in foods, and if we think about quorum sensing in bacteria, low cell density is something that we should perhaps avoid. And high cell density is something
which makes the group behave in a certain manner. And we can modulate that manner by eating. So, dining with each of us, we are actually dining with our bacteria at the same time and the bacteria are dining together. So it's a real community working on dining and nutrients that we get. Then fermentation. One of the important things, that we forget today is that fermentation has been most important for the survival of mankind. If we go back in time, about 60 percent of our food supply has been fermented. So we have had a huge amount of bacteria ingested every day. And now we are doing our best to actually make our food sterile. We pasteurize, we UHT-treat, we even sterilize food.
And we think about for instance the first food we have, with this mother's milk. Mother's milk is full of bacteria, viable bacteria, dead bacterial cells, bacterial metabolites, signaling molecules for the microbial community. And if we compare it to infant formula, practically no bacteria, no metabolites, no bacterial cells and it tells you already that that kind of introduction to our food system is going to have a profound effect. If we lose the fermented foods like we have lost them in the western diet it's also important that we lose the bacterial input, and so the quorum sensing part is going to be quite different as well. Fermentation gives us dynamic stresses, and lactic acid bacteria were taken out just as examples by Lucrecia, and when we do fermentation, there are several different steps that actually influence the bacterial signal. Fermentation makes the bacteria produce not only aroma
compounds, but these signaling molecules, that influence the transcription that we see in the genes and the way they behave. Of course at the same time you can produce different flavor profiles. You can produce different industrial properties, but the main thing is that fermentation is something that we perhaps need more and more in the future again. This has been noticed by the European Commission, and they are now actually funding a lot of research to fermentation and especially plant product based fermentations. It is in the European Union goals for 2030 to have a significant amount of plant-based fermented foods in our food supply. So, what does it mean? It means that the bacteria actually have
a significant role in fermentation control. We have seen it in our experiments ourselves. Bruno was mentioning the kefir. And the South-American sweet kefir that Gabriel Vinderola has been studying with us, and later on short-term fermentation with yogurts. And this is
something where we need to continue, and also understand that if we have simple things like sound or music, how they influence the fermentation control. And perhaps they are also important in producing specific sensory profiles and at the same time they could be an ease for the industrial process. We have seen this for instance in beer production, if we give specific sounds to beer, when it's maturing we can provide different flavor profiles and different sensory experiments have proven, that the modification of beer flavor is actually quite possible just by the simple use of different sounds or different types of music.
So, what do we know about it now? If we look at early human diet, there were high numbers of microbes in the diet, and they produce the high level of interaction with the immune system. Priming the immune system towards health. So what we have done? We have industrialized the human diet. Very little, very few microbes and no levels of interaction with immune system. So we are priming us for the situation where the simple pathogen can cause a lot of harm because there's no resistance in the normal microbiota. So what do we do? Microbes supplemented diet as
even the European Commission is suggesting. More fermented foods, new types of fermentations, and going into actually plant-based fermentations. Just recently published in Cell, one of the top journals in this area, indicating that if we do replace even high fiber diets with fermented food diets, we are actually improving health and well-being quite significantly. So this all in the face of bacteria discussing with each other and discussing with us. So perhaps that also gives us some future directions for nice research. Thank you. Thank you, Seppo. This quorum sensing of bacteria, at least for me, has been really
one fascinating new finding. I have understood, I have learned new things during this project. And now I would like to move onwards in this session number two, into art and technology where our researchers and artists together have been inspired by, for example this quorum sensing microbes, and built two different types of VR, I don't know, how would you call them? Are they VR, are they mixed reality? How would you call this quorum sensing and tweeting the feelings examples that I hope we will hear now presented by Bruno Mesz and then Sami Silén.
Okay, thank you, Anu. Another classification of this experience, I really asked Jean-Christophe, he's the expert, it's not virtual reality for sure, so this experience that we are going to present today after the webinar, is is inspired by this, also these experiments that we have been doing with Gabriel and Seppo and Sami, on fermentation of bacteria, the effect of sound of fermentation, so we were inspired by this special bacterial sense by which they detect the presence of others, and they change their behavior, turn in to a collective way of behaving And to the metaphor of this in the situation of commensality, in the situation where we have dinner together this part, yes so the idea is that to metaphorize this special sense at the human level by invoking the sense of touch, of human touch, because we reach and contact the others by this kind of, by this sense, and also in this installation we try to increase the sense of remote presence of the other people that are dining with you so how do we conceive this? Well, there are many layers involved in this installation. So first let me skip this and go to the scheme of the installation and then I will go back to this. So the idea is that you have a special soup bowl.
And the bowl has an embedded tablet in it. And I'm going to return to this. And you eat the soup using a spoon, that has an accelerometer. You see the basis of the spoon, the second picture, you have this capsule and there there is an accelerometer that measures the force, that you apply to this console. You eat and this force is detected, and the signal of the accelerometer is controlling sound, is controlling the factors in a computer, that regulate sound intensity. But you don't perceive it as a sound, but as a tactile stimuli, because we send the sound to these special belts that have transducers that vibrate in response to the sound. So when when some somebody eats at the table with this spoon, the other people feeling their backs the result of the activity of eating, of this person.
And it's more intense as it depends on the intensity of the activity of eating of this person, so you can eat in darkness but nevertheless feel the other people eating with you via this extended tactile sensitivity metaphor of quorum sensing the others dining with you. But I want to return to the soup bowl because there is another layer of meaning or another idea in this that this using tablets or using digital devices at the table also like contemporary art or art in general maybe have different layers of meaning that are not strictly related or strictly consistent, but they're like different concepts that mix in some way in a piece, in an installation. So, the the soup bowl is based in an experiment of on cross model correspondences and I want to show you a little video on this. Video: we present conceptual design of tableware that explores the use of digital tablets embedded in greyware for the creation of multi-sensory gastronomic atmospheres.
And it was conceived from the results of a pretest on crossmodal associations of shapes, colors smells and materials with four emotions induced by music. Energetic, tender in awe, "sobrecogido" in Spanish, as in the experience of listening to pipe organ music in a temple, for example, and the emotion of being agitated. Each video is accompanied by a good smells presented in on perfumery strips. Gas and dollar bill smell for energy, pear and apple for tenderness, absolute roles for awe, and sea smell for agitation.
We are currently analyzing the results of a new test and plan to use them for the design of real plate wear or for designing the environment at dinners in virtual reality. So that was the experiment so people listen to different musics and they draw shapes inspired by the music, they selected materials from at list, for instance is this music related to wood, to metal, to stone, to water, to liquid, and also colors. And we are now, this was an installation in a museum, and we are now here, we are going to present the actual plateware inspired by a musical agitation. So this is
combined with the quorum sensing. We present like two ideas, more than two in the in the same installation Another thing is that we want to reflect on the use of digital devices at the table which is generally regarded as negative because we are concentrated in the tablet and not interacting with other people, so they are like interrupting social interaction and commensality, but here we embed the tablets in the plateware. They are used to generate a multi-sensory atmosphere with sounds and colors that are shared by everyone in the table. This is some photos, Sami has nicer photos I think, and he is going to talk a little bit about the technical implementation of this quorum sensing installation. Thank you. Okay, I'm Sami Silén, and I'm a PhD student here in the University of Turku.
Bruno already told some technical things about our quorum sensing system, but I'm gonna give some glimpse and maybe tell more about the process how this thing has been evolving. The system is based on basically three objects which are the spoon with the motion sensor attached, and then we have a vibrotactile transducer or actually two of those inside the cloth belt that you can see on the on the left, and then we have this glass bowl and a plate on top of that and we call it, like Bruno told, agitated plate. In the center of this system is a motion sensor that is attached to the spoon and it holds an accelerometer, a microcontroller and a battery. And this thing tracks the movement of our hand and converts it to a sound and vibration. But the system is not that simple. We need something else also. And it's based, like Bruno told, on a code on a computer, but
we need several kinds of things and programs. In this system we are using four different programs, or should I say programming tools which are called Mosquito, Arduino, Processing and Ableton Live. And all of these required some programming and making some codes how these motion sensors will react with the system. And then of course we are in a wireless world with the motion sensors, so we need a wi-fi router for having this network because the devices are of course talking to each other wirelessly. And then from the computer we go out with the audio device that leads the sound to amplifiers which supports the sound and amplifies the signal so that it can be used by tactile transducers, which looks like that. It's a kind of special speaker which doesn't have this basic system that the usual speak
2022-01-24