Case Study: Leveraging AR/VR Type Technologies to Enhance Customer Experience, presented at ACBD23

Case Study: Leveraging AR/VR Type Technologies to Enhance Customer Experience, presented at ACBD23

Show Video

[Applause] Hi. Good afternoon everyone.  I’m Tsukasa from Tokyo.  Actually this is my second time to  be a part of this speaker faculty   here, I'm so excited to be here again. But  I'm actually a little bit nervous to speak in   front of this international audience because I'm  basically all educated in Japan and I know very   little about the building and construction market  internationally. But fortunately today I speak  

about the case studies so hopefully I can leave  some examples for you to take away. So today,   my topic is about the VR / AR / metaverse things  to enhance the customer experience, but before   I go into the main part, let me recapture  this: Why are they actually important for us? I think they are important for  the stakeholders who are keeping   on requiring better visualization for  what to decide. They are important for   the project because they help complete the  project model life cycle. And, they will   be important for our future because I think the  metaverse can extend the common data environment. Let me explain about those topics, but  first of all, let me introduce about myself.   I'm Tsukasa Ishizawa, I’m a PhD,  I'm a registered architect in Japan,   and I'm a LEED AP. I'm a group lead for  Computational Design Group, Takenaka Corporation,  

and I'm a project associate professor in  The University of Tokyo. I'm a YouTuber,   I have a channel with a little bit more than a  thousand subscriber where I'm talking about tips   and tricks for the modeling software, and  a little bit about the life of architects. About takenaka; actually Takenaka Corporation  is one of the top five general contractors in   Japan. On top of the corporate scale, it will be  characterized by its history, whose origin can   date back to the year of 1610. After the company  transformed into the modern corporate style,  

and since then, we were engaged with numbers  of the landmark projects like Tokyo Tower,   Tokyo Dome, cultural facilities and mixed complex…  basically we do everything around the buildings,   but we don’t do the civil works. We are  fully engaged with the building projects. Actually we are a design-build contractor.  In fact, 70 percent of Takenaka‘s projects   are designed-and-built. Simply it  means that, in our Takenaka project,   first our design team starts working usually  from the predesign phase and we run the rough   cost estimation at the end of BD phase; and this  is when our construction team embarks and start   talking about the design also. sometimes they come  to me and advise that “Hey maybe you will need  

to have some more clearance from the neighbor  building because we need to do the scaffolding.”   It's not really competing; it will give us some  good constraint for the design thinking. And once   it goes harmoniously then we can deliver  the project on time, within the budget. This became possible in Japan mainly because of  its cultural background. This picture is of the   middle of 13th century and the person  circled at the center of the picture   is a master builder called toryo. This  toryo is said to master five different   skills in one personality; namely, the building  design, estimation, carpentry work, drawing,   and detailing. It means that, in Japan, it was  quite a norm to come to a single personality  

to talk about the building project regardless  if it’s design or construction. The separated   concept of architect and builder was actually  imported at the middle of 19th century when the   Japan was in the process of the modernization,  but somewhat that process was fusioned into this   existing Japanese culture; and this is how the  modern Japanese general contractor was formed. Okay, so let's start talking about AR /  VR thing. You will use either a head mount  

display or a mobile device like smartphone or  tablet. In the Virtual Reality (VR) you'll be   fully presented within the digital environment,  so you will have the immersive experience in the   digital build environment, whereas in the  Augmented Reality (AR), still you see the   actual real world and some information like the  geometry or attribute will be overlaid onto the   real world object. So on the site the designed  building will be popping up in front of you, and   in the Pokemon Go the Pikachu will appear in front  of you. That's how the augmented reality works.  

And in the Mixed Reality (MR) it somehow goes  beyond. In this explanation it will allow you   to do the manipulation or interaction with  the object whether it's digital or real, but   simply this mixed reality is a fusion  of these two technologies. Overall,   we might call them as an extended reality (xR)  and today we mostly focus on the VR / AR things.

Looking at the hardware market growth,  in terms of the shipment it became   four times more in the last five years.  It's gaining the popularity so rapidly. One more important aspect is about the  Building Information Modeling (BIM).   Actually I'm a BIM guy and I'm not supposed to  be here to talk about the VR / AR things but   it is a very good opportunity for me.  This is an interesting survey done by   Dodge Data and Analytics. They say that  BIM users, compared to the non-BIM users,   have a great advancement that advantage in terms  of the adoption of digital design intelligence   tool like VR / AR, reality capture like LiDAR,  mobile devices like wearables, and the digital   twins. So now the BIM is an infrastructure  to adopt the further digital transformation.

That is why we started focusing on taking the  project model directly into the extended reality   presentation. We collaborated with the Unity  specialist and internally developed some kind of   add-on that allows to import the 3D models which  is sometimes from Revit, sometimes from ArchiCAD,   or Rhino, or SketchUp it doesn't matter. Just we  take the model into the VR or AR presentation.   One more important aspect is that this  environment allows the telepresence or   telexistence. In the demo on the left hand side  you see the different architects from different   cities are joining into the same environment to  talk about the design simultaneously. This is  

what we can do thanks to this  kind of digital technology. One more thing. We can visualize what’s  basically invisible. In this example we   are visualizing the airflow by the arrow or  bubble. Or we simulate the direct sunlight  

intake. Sometimes we can do very simple  acoustic reflection simulation. Those   are so essential for the environmental  performances and building sustainability,   so these are so helpful to align the understanding  of the stakeholders for the decision-making. Let's go into the examples. The first one  is about the San’ei headquarter office.  

The building’s overall shape looks boxy but  actually the internal view is a little bit   complexed. This is because this office is  for a steel fabricator located just next to   their steel shop, they wanted to have this  office as their technology showcase. So we   designed a building with the slanted wall and the  Voronoi-like volumes which was quite difficult   to understand in the conventional forms. So  just we had the CEO and we gave a head mount   display to him and he tried to manipulate the  design to understand what's going to be built. Another project is about Chuo University.  This building was characterized by the  

huge atrium space at the center. Now  you see the rooftop design changed;   and actually the design of rooftop and material  selection has a great impact to the building   sustainability. So we prepared several design  options and we ran the environmental simulations   in advance. We took those design options and  simulation results into the VR environment   for the client to choose a specific design  options to see the impact to the thermal   comfort. So in the video you will see the flying  arrows and actually this show the airflow with the  

temperature. Once they change the design it will  be reflected back to the arrows and they will see   the differences. They don't actually feel that  but they are visualized for the thermal comfort. This is an AR project and this is the use case  for the inspection. This building is called EQ   house for Mercedes. They wanted to have this  finely detailed building, and this was possible   because of the well-made design model. We took  this model for the AR inspection. Because the  

building has quite complicated shape, we wanted  to simplify the thing for the auditor to focus   on. You can see at the right bottom picture  that this is highlighting the sprinkler heads   with the dimensions so that for any people that  once they wear a head mount display they can   focus onto the items that they need to inspect.  Rather than adding the things to the real world,   this is to filter down the information to focus  on to the things that we really need to discuss. One more thing, This is the PoC project for  the device called Atom by XYZ Reality. This   is an AR device but this is integrated with a  safety helmet called smart safety helmet. This   obviously allows the on-site AR. This is durable,  and this provides hands-free environment for the  

site foreperson to see the items to be built.  For example, on top you see the MEP services,   and on the middle you see the drywall  installation. In terms of the accuracy   there's a comparison at the bottom, which is  quite satisfactory. Only the thing is about  

the affordability; at the moment, we can't  afford this device for every site person,   but if it's getting the popularity then we can  step forward to the paperless construction. So it's going, advancing, but I have some  complaint about the use of the model. These   pictures are from my past project where I was  playing a role of the BIM manager. I might say   that the people including the clients, the  corporate members were so collaborative for   completing the project model. But actually  how they use it in the project was always  

like that. They prepared a projector or monitor  and they show up the model and they manipulate   it for three to five minutes — and they come  back to the paper agenda and paper drawings,   and keep on talking inside this. Nothing’s  wrong in this conventional working style but   it obviously looked like something additional or  something top-up to the existing workflow. Once we  

illustrate it's gonna be like this. Once it goes  very successfully, the building information model   can spare some effort to create 2D drawings or  the quantity take-offs: but sometimes that might   be the best. Of course we can derive out from  the model in terms of the VR / AR presentation,   but it still remains as the position of an  additional tool-set to the conventional things. One day I started thinking like this. In last ten  years or even in last three to four years, we got  

so familiarized to have a meeting online, like  Zoom or Microsoft Teams, whatever you name it,   but still to date we prefer to have  the meeting face-to-face basis inside   a physical meeting room, which is completely  fine. But those spaces, whether it's virtual   or real, have nothing to do with these spaces  we are going to create. Of course. It's not   built yet. But how about virtually? It's somewhat  there already because someone designed something.   Then why don't we host the  meeting inside the project model   and track the project activity all the  time inside the project environment? The reason why I started to think like this  was because I have some bitter experience. In  

one project, the client required me to update the  models to the cloud environment every week. I did   so. But what I found was that nobody opened the  model for a week; and almost forever. What was my   effort overall? Yeah so the client wanted to make  sure the things are updated and that’s the client   job I know. but I wasn't so satisfactory about  those kind of things; then the idea was that   clients need to be more exposed to the model we've  created. I know that sometimes it's too much to   ask a client to open up the model to manipulate  around and see inside, because they have their own   job. Then why don't we originally have the meeting  inside the digital venue that we've already  

designed? You can be inside (or above, beneath,  nearby or wherever you want) then we start   talking about the project. And the things will  be digitally recorded, time-stamped and sometimes   transcribed, tagged and stored in the Common Data  Environment (CDE). It's going to be possible. Then a concept will be like this; originally  we imagine the project metaverse space and we   locate the project model inside and all  the project model or project activity   will be inside this environment. I named  this concept as the metaverse as common   data environment; and actually I I'm going to  publish a paper in coming academic conference   so if you want to know more about this concept  please find it in the conference proceedings. So now is a time to talk about the metaverse! Wikipedia says that in the colloquial usage  a metaverse is a network of 3D virtual worlds   focused on the social connection. Yes, VR and  AR are the essential technology to enable this  

kind of virtual environment experiences, but  in the metaverse we are more focusing on the   communication, interaction, transaction  happening inside this digital workspace. This is very useful framework called  digital twins-native continuum.   We, as a building specialists, tend to think  that the metaverse is all about the digital   twins which corresponds to the real world. But  actually it's not. We can imagine the worlds like   Fortnite, worlds like Minecraft,  world like Super Mario, whatever.   This digital native space is all possible and  people can communicate inside those spaces.   Actually this is spectrum and we can imagine  the blend of these two different elements   so this allows us to think about the new potential  or possibility of the data we already have.

But let's start from very straightforward  example. This is our first metaverse project   called Takenaka Dorm Reunion. This  is our Takenaka dormitory. This is   our design-build project, and this is  exclusively for company’s first-years.   And now we have the design model we took it to  the AR environment and we invited the company   people to exchange their latest issues. This was  quiteactively done because due to the pandemic  

the people's behavior was quite restricted. We  realized that making a conversation inside some   spaces related to building environment, it's  quite valuable. So we extended a little bit. This is something different. Recently  we expanded the office a little bit,   but this was not a design-build; this was actually  a retrofit. So we didn't have the detailed design  

model. But instead, what we did is that we  took the 360 video device called Ricoh Theta,   we walked around the building and we took it to  the simple metaverse platform called Buralit.   We call for the people and they join to  this virtual tour. Still this is a tour   along this pre-recorded video, they can spot  on and they can focus on the specific point   they wanted to see because it's 360 video. And  we can communicate over voice chat, text chat,   or even with the stickers just like the social  networking tools. This activates the conversation   about the build environment, so why don't  we all the time take this 360 video tool   and share the building experiences with our  teammates? It will activate the discussion   about what we build and what's good what’s  bad for the designs. It's going to be useful.

If it's useful for the internal people it's  gonna be true for the rest of us. Then, this is a   proposal for a property called Chochikukyo (聴竹居).  Chochikukyo was originally designed in 1928 by an   ex-Takanaka architect. This is a housing property.  Recently Takenaka acquired this facility and now  

it's designated as a national important cultural  property. Because this property is located in very   quiet residential area near Kyoto City, it is  so challenging to host a big group of visitors   to be inside here, physically. But digitally,  it's going to be possible. Just we scan around   the building and we open up the space for  the virtual building tour then it's going   to be meaningful for those who are interested.  Even we can assign one building specialist to  

do the building tour and answer to the Q&As. If  it makes sense and we can find a way to monetize   this tiny business, then we can think about the  better management for this cultural heritage. This is the last example and this is tiny, tiny  one. Well, you can see a girl inside and this   is my four-year daughter. She loves arts and  crafts and she always creates something new  

and bring it to me. At first I did some pin-ups  so happily, but the space is limited anyway. So   I took it into the box, but nothing happens  after all. So I imagine that what if I can do   the quick scan of the drawing or works, and I  can store it in the digital environment.I can   design a tiny virtual museum and I can locate  her works inside. Then I can do anything! I   can open up the rooftop, I can enlarge the  pictures, I can invite a group of people,   and even I can invite the TV crew like this!  So this costs very little, but this is actually   expandable. This is not only for the children  but maybe for everybody who loves the art. If   this makes sense Takanaka will be so happy to  design the digital-native museum space. Actually  

I talked about this idea to friend of mine, he  said “oh… interesting” but nothing happened after   all. So if you find it interesting please talk  to me I will take it so serious (just joking). Like this so we've seen four different metaverse  examples. The first two were under the concept   called “Metaverse as a tool.” We were actually  focusing on the communication happening inside   the metaverse, so metaverse was just a digital  venue to host those communications, whereas in   the last two examples it will be aligned to this  “Metaverse as a target.” There'll be something   that they may want to visit because there might be  something that only there you can see. So this is  

quite useful framework again to think about our  purpose to look at the metaverse’s possibility. One more thing. It reminds me about this famous  cartoon that a client asks you to create a swings   and somewhat your engineer designed something on  the left hand side but actually overall what you   needed was just like on the right. So we, as the  building specialist again, tend to think that the   model deliverables should be always like something  on the left. It's so detailed, nicely coordinated,  

and this is a building actually. But for  a layperson, for example to my mother,   how my mother recognize about the building  is that “here's the door, here's the window,   here's the wall.” That's it. And nothing’s wrong  with that. In most cases this is good enough.   Then we can expand our way of thinking just a  little bit; we could make use of the project   model but sometimes just a reality capture  or even just 360 photo will be good enough   to provoke some communication over the build  environment then it's going to be cheaper,   more feasible, and more activating  for the people to communicate. But I know that there's doubts and risks.  Some company answered investing in a metaverse   is risky because the trend will fade, or  there will be issues around cyber security.   Will be. And the digital life takes over  real life. Hmm. It will more make sense  

to me if it's gonna be rephrased like this. “Stop the metaverse, save the real world.” Mm-hmm.  But it might be true. The metaverse, VR and AR,   all these things look playful and somewhat it's  on the buzz, and in terms of the expectation,   it's a little bit overheated. In the world  around you, there exist tons of issues to   be solved. Some people may question that is that  the very thing to be prioritized in the real life.

But I might say that maybe it is. VR, AR  and metaverse help save the architecture   and construction when it is used to preserve  the building conditions, or if it's used for   education, training or knowledge sharing. Or  if it contributes in creating the better city   data or it's used for the smart city where  it serves for the citizens’ better life. At the first time our company, including  myself, expected metaverse to make profit.   And that is the top expectation of the companies.  But we soon after realize that the expected amount   of the actual revenue that we can expect from  the metaverse is not directly comparable to   the one from our main domain. Then, why don't  we think about making the better ecosystem,  

or better communication channel for the building  lovers, to create a better communication among the   buildings specialists or building lovers? That's  one potential role of this kind of environments. This is the conclusion and takeaways. The VR /  AR things will be important for the stakeholders   because better visualization matters. Sometimes  the cinematic realism is important but just a   visualization “for the rest of us” will just  work nice. They are important for the projects  

because the metaverse can become a platform. But  we all know that not everybody is a metaverse   lover so it has to be comfortable, it has to be  inclusive, yet non-geeky environment is needed.   At the last, for our future; it’s not just for  fun but for fans. So let’s think about what we   can enjoy only there. Thank you so much. [Applause]

2023-03-02 01:55

Show Video

Other news