Tech Talk #5

e e e e e e e e e we'll wait for two more minutes so that others can join then we start yeah e get e e let's start so very good evening everyone I am sm s from sc9 University and a very warm welcome Today Show and today we are thrilled to have you all join here it is my absolute honor to introduce Our Guest B kurana the director of engineering at gamecraft one of the India's fastest growing technology company and with an over with an over 16 years of experience in software development ver has consistently been at the Forefront of business carable B2B and b2c and D Innovations across diverse Technologies leading high performance teams also today he uh he will share his expertise or a topic that is crucial for every inspiring technologist high level design 101 design scalable and robust applications this session will offer uh invaluable insights into how to create system that are not only resilient but also capable of handling the complexity of model deck applications Beyond The Talk this is also an incredible opportunity to connect with a leader who has a a wealth of knowledge to share not only about technology also about leadership Innovation and career growth without any further Ado please join me to Our Guest hello everyone all of you should just mute yourself once huh cool so thanks am for a really warm welcome and really thanks for inviting me uh for for the discussion I'll try to keep it casual um by the way I'm sitting outside so in case uh I'm not audible let me know of course right uh um yes I think Samar gave a I think very flattering uh introduction yes I've been in the industry from quite some time now uh worked with different industry including quick Commerce gaming education right and when I was asked to uh uh talk here I could have chosen a lot of different topics right data structures or what are the things AI or anything else but I thought this uh this one is something which this exact topic something where I can share better right I thought from my point of view that what are the things where I can explain it in a way which is slightly better than know any uh any other form there are a lot of ways to do this there are a lot of courses available but I'll try to keep it very very short and crisp and very clear um easiest one right if you don't know anything about high level design this will still help I'll also I also see a lot of good questions from you keep them coming I'll keep a lot of time at the end for answering hopefully all of them right so um I'll just start and uh Samar let me know if there is any anytime when you want to interrupt me for anything right I'll just start sharing my screen let me know when you can see it I hope you can right yes we can see cool uh so when you talk about high level design essentially what does it mean right how do you say that something is a good design right and uh I I've got a lot of questions around it so I'm assuming a lot of you are aware of how do you evaluate a good design one of the crucial metrics of course is skillability when you say scalability of course it means ability to handle the scale when you say that ability to handle the scale it should mean that the applications keeps working right within the whatever is expected right so if let's say you expect something to work in 20 millisecond and it is okay if it goes up to 50 millisecond right and uh let's say you know you want to be able to handle a scale of a million request in an hour then it should not go above 50 if it goes then it's not skillable right it's not just about that you able to handle the request but you able to do it within the contract or within the metrics you have already defined for yourself right second thing is of course High availability uh when you say talk about high availability that means your app keeps on working without any interruptions what does it mean right let's say if there is it can be because of scale let's say you know a machines go down or something or it can be because one of the one of the infra had a failure right because of that your whole application should not go down right um so that's what high availability means of course there are more uh performance which means optimal utilization of resources uh to optimize on cost and user experience so essentially uh you can have a great design but let's say you put very very costly infrastructure pieces and you're not even worried that how much I'm paying you just just make a great design it's still not great if if it is costing you way too much or if you know it's actually underutilized all that stuff right so performance is again another thing resilience is essentially one more metric which we don't talk about often is resilience which essentially means that how soon you can get back to the original let's say some in some cases infra infra level failures or uh um you know let's say if there was actually a big spike which happened which caused your application to go down but how well it can recover either manually or automatically that's what is a living uh now when I talk about this steps to design architecture this is when you talk about 101 of course over the period of time you start getting better and you start designing from the scratch itself really good design but if you if you are starting soon starting you should start with the simplest design you can think of figure out what are the limitation single point of failures spfs Bott leg Etc and then improve it right this is uh like yeah this is like trying to uh come up with the D um we talk about no brute for solution and then we get better and better and better think of it that way so what I'll do I'll start with a simple example in this example what we are doing is we are in an application which has a server so it should have apis Etc and it talks to DB very simplest right now how I could have designed the simplest architecture will be um ah sorry uh these are the limitation which we can talk about scalability bottleneck downtime Impact Zone single point of failures which I mentioned in the last slide also high cost less maintainability and less performance now let's talk about the example I talk about right so there's a application server which takes the user request and it's off to DB great all great right now this is in one machine right uh somebody has raised their hand do you want to ask a question or how is it if if if it's okay we will uh give time for the questions or there's a document you can write there we can have question afterward yeah okay good so uh now when you look at this design the simplest design easy to understand but you will need to identify the limitation in this now easy thing when you think talk about it it's not scalable right so it the number of requests it can handle will be dependent on how big the machine is and there's a limitation to that how high downtime impact which essentially means if this machine goes down everything is gone right and of course single point of there in fact there's only one point here there is one box if it fails the whole thing goes down how do you fix it what you can do is vertical scaling which essentially you can get bigger and better and better machine more RAM more CPU but these kind of solutions are always very costly and they are not scalable uh because Beyond a point you won't be able to get that machine and it will be super costly and uh then what you are saying is even in even in low scale you are H you giving the cost for the very big machine and it will still not solve the problem of single poter failure so the other option is vertical partitioning you want to have a DB you want to have a application server have two machines right so this is the best first fix you get have a database server box application server box how does this design how does this design also help is now you can have specific machines machines which are designed to be you know holding a database which has probably more memory right uh can have more I Ops on application server s side you probably can think about uh it should have more RAM right all that stuff now now the machines can be specific to the use case you want so this is slightly better design than the first one what are the limitations still not scalable we haven't done great job there still hide down in Impact single point of failure now there are two points like this database can fill or application can fill so how do we fix it again not now we can try to do horizontal scaling so what you can do is instead of having one application server you can have multiple application server right so and uh you can distribute the load between them right I'm pretty sure most of you would have heard of what how this is how the load balancer works right there's a load balancer and there are multiple servers which has your same applications uh code right and uh the load B does a job of Distributing the traffic and when you do that you can always increase the number of application servers or decrease it in the run time right so uh this is the architecture which will look like when you talk about load balancer it is not one load balancer again it is multiple like there's a master and then there is a slave load balancer because um if one of them goes down what happens right I think I think that also I'll cover in the next architecture so this is the architecture we have come up with now there is a database server multiple application servers a load balancer user request coming to it what are the limitation database server is not scalable high downt impact if database server goes down and single point of failure which is database server and load balancers so uh another thing which I mentioned is that when you have multiple application servers on a load Balan so you would in in lot of cases you would still want the same user to request to go to the same server if it is available uh then you can use sticky sessions or uh you can use cookies possible fixes you can have multiple load balancers so essentially there will always be a master and then active passes basically or active active there are two or more load balances if one of them goes down the other one handles the load for some time and the other one comes up in between and in DB you can have something like Master Slave right so Master Slave is essentially uh multiple replicas of the same DB one of them is used for edit the other one is red uh is replicates the data and it's used only for read right so very simple if Master goes down for any reason the slave becomes a master node and then there is a one more slave which comes up right so this is how it will look like uh so if you see there's the applications servers load balancer with active passive or active active whatever you want to use uh they are writing the application servers are writing to the master DB and it getting replicated to the slave databases critical read let's say you don't so there'll be a little lag between the master DB data to come to the slave DBS so if let's say you want to read something very critically like you it cannot have little even little bit of stale data so you might want to read from Master DP right there are designs around it that how do you make sure that the data is fresh but this is a much much more scalable design if you think about it right so essentially what we are saying if load balancer goes down the other one will take up its place and uh then there'll be one more which gets spawned um we can even talk about how do they get spawn but asally think of it is that there is no component if you see which is a single point of failure if one of them goes down the other one will come up application servers are multiple and then there are DB which has multiple slaves right so there's no single point of failure here right now uh from the looks of it um but yeah there are still lot of issues with this design still what are the issues database scalability is limited right at some point you will say that oh my company is growing we have more data more traffic but uh the DB we have is you know can scale up to certain point only let's say you're hosting it your so for some maintainability on um is is tough here because what will happen What happens if uh if database failure happens then getting back that data is is not not that trivial cost again uh we haven't talked about Auto scaling so here the cost can be very high if you put lot of servers and you don't increase or decrease the number of server based on traffic so what we can do is um vertical partitioning in database so uh it essentially means that you go ahead and take different tables and put them in different databases so let's say you have one table for for Simplicity let's say employee and employee details right so you put one of them in one DP the other one in another DP sounds very stupid if even if you think about it right because there are a lot of issues uh it's still not scalable you can scale as much as the number of tables but uh too complex not great performance but the only way which works um in database is horizontal partitioning which means you can say all the tables are there in the DB but after some time right or based on let's say shr some logic it gets distributed across different data uh datab let's say uh it can be based on something random right every one first record goes here second record goes here third record goes there and there'll be a central uh node which will remember where the record went right and uh it can be based on um user ID let's say for some users all the data goes into this for some users all the goes into this depends on your application depends on what is the use case on an application so when you're designing you need to figure out how do I want to partition my database right if you're having multiple shs if if not a lot of cases even the random one works where uh it gets depend uh basically it gets evenly distributed as much as possible right that also works so now we have multiple databases with their own slave DBS and now you can keep on adding right uh there is small issue here because if you have decided that if it is random adding a new one will be easy but let's say if you have decided that from user a one to every even let's say every even user goes to this one odd user goes to this one then you add a third one it becomes a challenge so you need to think in the in advance that uh that the partitioning logic which you're using should be in a way that in case you want to add more you should be able to add right or there are techniques also if you have if you stuck with this also you can change it the data gets R return return and everything so it'll be a bigger exess but now if you look at this design now we what we are saying is that you know it's quite scalable still the issue is u maintainability a little and the cost when you say talk about cost essentially in the application servers what you can do you can add that let's say if the CPU goes beyond a certain point I will add one more note if the number of or you can say number of request overall When you're counting at the uh load balancer level if they go beyond a certain point I will add one more let's say you want to say that my application server can handle 10,000 request in an hour or let's say 1,000 request in a in a minute so uh so let's say that is example 1,000 minutes uh 1,000 per minute now if you're getting number of request which is beyond 3 33,000 then you can add a fourth one and if you're getting less than 3,000 you can move to two again right you will not want to move to one because that will make it single point of failure so whenever there are autoscaling if it's a critical system you will never want to have one server at a time because even if in the lowest of the lowest of the traffic you would still want at least two to be there because if one of them goes down then the other one can handle the traffic for some time till the first one comes back right so why is that because we don't want to introduce single point of failures so that's why it's important to choose what kind of machine it is there also so uh you can't choose a very big machine if you do that then you are taking the cost of two machines all the time at least and you can't choose too small a machine because if it's too small a machine uh the operating system and all also takes away lot of ram right so you might not be able to get the um performance you expect right so you have to you have to have that tradeoff you need to figure out what is the right machine because you know that you have to scale between two2 whatever the number is maximum so this architecture gives you a feeling of that okay uh I can I am able to scale right essentially Auto scale the auto application servers m DBS I have already defined I cannot do it in runtime but I if I see that the DBS are filling up and I need to add one more shr which shr essentially is a um think of it as as a the way to divide database data across multiple uh databases right so possible fixes that already covered apply Auto scaling to application server based on or other factors other factors being CPU utilization um disk in some cases number of connections right all of that can be a factor CPU being the most popular one because you know that whenever CPU is going high let's say it reaches 70 then you know that it's if it reaches 100 then it can actually uh application can go down and everything right uh for DBS of course you can use manage services so that's why manage services are very popular because they they give you out of the box all lot of uh scalability can get taken care of right if you want to uh increase the size it's very easy right and uh even even when you talk about load balances right making a load balancer yourself in itself will be a big deal you'll have to write lot of scripts health checks uh so as a load balancer you need to keep checking every few seconds that the all the machines are up if they're not up then then you need to you know create one more machine so all of these guests um you get this solution out of the box if you take something like AWS or gcp or digital lotion or you know any cloud provider so so these are the architecture thing um now when you talk about architecture and this will be crucial a lot of time whatever architecture you make will have a issue I'll give you example let's say you're designing something which um converts a word to PDS now when you do that the user is uploading it there's a request to your application server but the conversion actually is a very time consuming task right so would you want want to user to see a page which is loading or something or you would want ke no at the back end I can I can take this request and process it in a que queed fashion right so uh asynchronous task or any task which you think cannot be horizontally scaled uh having them as asynchronous task is a great idea like let's say and if you think about lot of designs which happen right now across uh if you talk about very highly scalable applications even um that's a Twitter right a lot of things happen in a way this where you say okay I have um most of most of them use microservices and then they event driven architectures so essentially what is happening there multiple Services one service takes the request and says okay fine I've done this acknowledges is that okay I've taken the request and then s went to the other service okay you do the next part right or you save it or you use it in your report all the services talk to each other through events right so uh if this service the first service which is talking to the second service to do something is not really waiting for the service to finish it just saying do it and you know give back the response to user okay your file is processing then it is asynchronous event because now the users is are is not user action or whatever you're showing to user or is not part of that one Atomic operation it is I've done this part and then next part is happening once it happens I'll get a event back or something some way to note it and I'll give it back to the user when you're implementing such uh asynchronous task what happens there are a couple of approaches um one of them is push base so I know that there are these 10 machines which are available I will keep on sending the uh request to one of them round robin or something this is how load balancer Works load balancer says when the request comes to me because although that's not asynchronous but you know uh I will give it to different servers I know that they are healthy because I checked it just now I know that they can handle because they're healthy and everything but if you if you actually see it in action it will be like this application service are assigning some task to the workers and workers are doing them and the other one is pool based where I say okay these are the task I keep on putting it somewhere and the workers take it from there right so this is s similar to anything you see in real life also right I get request there is um let's say there is somebody let's say there is any office where you you're putting the form and then somebody actually based on that form uh does something right so there'll be somebody who's actually accepting the form putting it in some CU and then there is somebody who's actually processing them right so here also it's happening the same way application service are putting task in the queue and workers are picking it up now Advantage here of course is if um you can even autoscale this workers if the queue if you see too many um things in the queue right then you can say okay I will add one more worker so that it get processed faster even in real life what will you do you see if there's a lot of backlog you will ask one more person to come quickly you know help me because there's too much backlog so pbas is much much better if you talk about asynchronous task right there is hardly very very less use cases where you do a push based for asynchronous task because they're easier to implement since there is a queue involved you can always assign um so multiple workers can keep on asking for the task and Q can take care of it that know this task I already given to the first worker I'll give the next one so if you if you know there is Kafka open source queue which can which does it beautifully very very very popular similarly there is manual services like sqs uh so um lot of different cues are available and they are very heavily used right why they are popular is for the because of third point also there are spikes at certain points let's say you know first service instead of getting 1,000 request per minute now it's getting 5,000 now you added more servers to handle the request but the next service cannot immediately scale up right so what will happen you will put that in the skew and then the next service is processing it it on its own time right but at the same time because it's processing it own time it's not actually increasing the CPU utilization or anything because you have already defined the Pace that okay this is the maximum Pace will go we already calculated that you know this this is the load I can handle so Spike can be handled better right what will happen probably the processing will get delayed a little let's say when the first event was coming it was getting process within 10 millisecond now since there are more events coming every event will have to wait for more time right uh think of it as a traffic signal right if there's more traffic you'll get you'll take more time to reach uh uh after the red light so essentially uh Spike can be handled better because of adding Cube but how do you avoid data loss so essentially it can happen I told you I put it in some Q the other guy did not pick it up what happens so uh if there's a q involved of course what you can do that Q can retain it till the till the ACT system says yes I'm done with it acknowledges it and then Q can remove it or Q can just retain it for a day for a little longer time so that if there are multiple workers who are checking it they can actually do it so this is very very popular so Kafka works on time based retention so essentially you'll have to Define that my data stays for X hours minutes whatever you want to Define and the other systems responsibility is to finish the processes within that time so if a event comes within that the time starts like you know if you have 24 hours it should get finished within 24 hours typically all the systems works very very much faster than this it gets happened in seconds minutes but uh you keep the retention High because let's say the workers are down for some reason right something went wrong then you have some time to fix them and the data is still not lost right so um there's a lot of I know there's a lot of uh uh things I shared just now uh but in any system even if you're making a small project for your college or doing something internship in in internship it is important to think from this point of view that if this goes wrong if there's a issue here I am not best case will be that my application doesn't suffer it works beautifully if not at least when I get it back my data doesn't get lost right so it's very important to think it that way if you are thinking let a very simple application like a lot of very very popular project uh uh ticketing system let's say now somebody's put a request for Ticket something goes down that ticket cannot get deleted even that request cannot get deleted right obviously right because that means uh the new system is not at so very important concept always think that if lot of things goes wrong at what point uh my data also start getting lose of course there can be cases where you still lose the data but your point should be your system should be that it cannot happen unless something really drastically happen that all of the system go down at all the at the same time and probably in that case also you should be able to recover it probably from the files cool so uh that's the last part um now of course questions uh people can ask it uh directly right now or uh I can go through the te talk the document which was created yesterday um yeah um we have some questions for you students also ask some questions so start with one of the good questions there which is what are some key lessons you have learned in your 16 years of experience that have shaped your approach to engineering leadership I think that was in one of the document also right yes it uh so um I'll try to answer one question uh which was in the in the document also and try to connect it with what I've learned so one of the question which was there was how do you approach tradeoffs between scalability man maintainability and performance in the high level design right and a lot of time uh when you become a people management people ask how do you maintain um performance uh quality and at the same time you want to make sure that the you know people aspect of his is correct right um so what I have learned is that you have to always strike a balance you have to understand what are your Gils and when I say Gil essentially means that what are the things you know that this should not go beyond this point right let's say you talk about scalability you know that your scalability has to you need to start from the beginning that what is the scal looking for right what is the performance you're looking for and then you design the best system yeah right let's say you are looking for a performance of 50 millisecond yeah yeah you did a great job and now it becomes 20 even better than what you were expected but you see that the cost is high now you'll think now since I can go up to 50 can I decrease my cost and make it 40 45 so what are you what what are you trying to do here because you know what is your performance is not that the objective is not to lean towards one thing that I will make it the best performance thing okay instead of 50 I do it 1 millisecond if the require if you know that you can go up to 50 and you're saving some cost here you'll have to strike that balance similarly um when I talk about when I think I had a post also one time that as an engineering manager when I started what is the most important and I mentioned that you will always need you you can always lean towards one thing you can always say Okay um my I will deliver the projects fastest you know I focus on delivery I will put more people and try to I'll say that okay you know everybody works really hard put in time extra hours and finish it or I can say I will focus a lot on quality right even if the time suffer but I will focus on quality that okay you know everything should have proper design test cases proper testing uh great maintainability right or I can think about that you know no I have to make sure that my people are not getting burnt uh right they are not yeah definitely you know people aspect of this you can probably if there's a critical thing probably you will put in extra hours and be done with it but you can't do that every day right it's not sustainable so I always mention that you always have to strike a balance that's what I've learned you need to know that what are the guardrails what are you trying to what are you trying to optimize and what are the guardrails guardrails is that it cannot go beyond this or below this or something like that right so I'll give you example let's say you know you need to set your standards and quality you say okay you know uh we should have these reviews we should follow these guidelines no negot non-negotiable I cannot make a system which doesn't ensure quality right but when I say quality I'm I'm saying okay this is a standard you need to do you can do better than this but I will I will be okay with you know this quality right yeah then you say okay um I am okay with the these timelines right I you will of course estimate and you'll be like okay these LEL are okay and you'll give it so you have to know what are your uh uh what are your parameters you know and then design a system or design or distribute task or figure out how how are you going to do the project I think that's what I have learned that you cannot lean towards one thing because that will make other things in a in a worse shape right wor that was very insightful and I think you will just answer one of the most important questions which we have and like also want to ask you like what was your what was your one of the most challenging project which you have overcome in your career and how did you overcome the obstacles with that great question I think that was also in the document a great question um I would have would have wanted to say something when I was starting my career early that you know um I didn't know much and I was trying to learn and I failed and then I retried and then did better took help ask a lot of question research and all um so that was my first project right I I was like yeah really somebody who was trying to put in a lot of effort but still making lot of mistakes um that was one option but I think I would still choose something which which happened quite not that recent but I think four four years back or something um so essentially that was in one of the company I worked with ripling and uh what we had to do was uh to make we I was in the payroll filing team tax filing and US taxation is really really complicated tax 10,000 different taxes so if you are in one area you will get tax for that state for that City for that area separately even in your income tax and was taking care of their payroll so essentially what it could do was calculate all the taxes for you so when you're paying the employee you don't need to worry about it all the taxes will get calculated deducted paid on time everything and that was the team I was leading so uh so yeah so we took a ambitious project that you know we were using a software to do this we were talking to that software and that software used to do the filing so imagine you go to uh some government website and do the filing great awesome yeah and we thought okay uh we can't rely on that software because that we were reaching at a scale that where that software and the and it was a service not a software just so it was a software plus service so they were reaching a scale that they were not able to handle the kind of scale we wanted because our customers were growing all of them all the clients were growing and we were using them for the filing thing and they were not able to handle that skill so we okay ambitious project let me let's move it in-house okay very very challenging I think uh one of the most one because there was not just challenges from our side yeah yeah like a good system but you also have to make sure that you are able to automatically talk to so many websites now when you say these websites they are government websites and believe it or not us India government websites are very unreliable right and then they have they policies you cannot access it on this time yeah they are uh from this IP sometime it doesn't work just straight up doesn't work sometime they go for maintenance for a day right now then you are talking about hundreds at least hundreds of websit like this if not thousands so very very challenging it was and uh at some point we had to even revisit our architecture a little where we said okay um we cannot assume that the you know when you when you do something you say okay I'll try this and I'll retry three times after that they say it's a failure right but we have to get away from the design we have to think that okay we will keep retrying we will think we will figure out we will have to add strategies that okay if this website you're talking about if this filing you're talking about mode then you have to go go this way this time this IP all that introduction had to be had to be added um but yeah very interesting challenges uh again the idea was at some point you will be you will feel slightly frustrated right okay come on this this website should have been working is it not working yeah my thing is working right but uh we did not I mean we had to make sure that we are still keeping it we still tackling it one challenge at a time and getting better and better and I I think first quarter we tried it we could not do it we still had to rely on that service we had of course not given service so was to was to do the major websites major of the filing we were able to do which was in itself was a big achievement for us next quarter we did 7 80% I think fourth quar third or fourth quarter onward able to do 100% nysis so very challenging uh for very uh rare reason and most of the time your reasons will be around uh Tech challenges and everything this is a tech challenging in some way but very external challenges thank you very much ver now guys if you have any more questions you can ask yeah yes you can ask a question hey Mr kanana how are you yeah I'm good so I'm paru from second year and my question is what do you think about the scope of software development or app development in the gaming industry for India as compared to the foreign uh countries okay let me uh repeat the question what do I think of software development and app development in gaming industry in India versus us right is that the question yeah okay yeah um so when you talk about software development and app development that is what is there to it right in gaming every game which you play um and we are talking about of course online gaming is is a software right so that's very broad when you talk about it app development soft development of course uh if there is a online gaming industry you will need software development you will need app development right why I do I say software development of course is a very very broad thing right so even when we are uh playing a simple game online or even on your phone it's a software now if you talk about app development and why is it I think whenever there's an industry like gaming industry app development will be critical is because uh because of the reach right when I talk about app here I'm specifically talking about mobile apps uh very rarely you will see some mobile apps or let's say even desktop apps or something very rarely you will see games where you don't have a counter app counterpart because of the reach of course uh mobile has much much much bigger reach than laptops and desktop now when you talk about us versus India I think uh us Industries gaming Industries are very very uh saturated right now but uh yes again very very big industry uh casual gaming U real rmg's um so essentially the the there are the industry is very huge right now even us and across lot of other nations also in India I think it's still very early stage I know it has come a long way uh and I I'm inan games SC being part of that Forefront uh but I think we we still have a lot of scope in terms of gaming in India uh be it casual RGS or uh any other new thing which come up or even um you know um virtual realities and gaming yes so short answer big scope in India yes uh app development and software development will play a like mandatory role I won't even say important that's of course thank you thank you thank you so much that was very one of question which our student ask in the Q&A form says how can the design accommodate new features and updates without major overl walls great question okay that's that's a absolutely great question now how do you how do you make changes to a design and overall without making major changes very short answer is by foreseeing it beforehand so or keeping your design extendable I'll give example let's say you are building any any system give me one example of any system you want to build from you someone any system like you can say let's go for gaming like simple gaming system where you can like P games and then I eight on like one scale okay let's talk about one game now let's say I talk about a game very simple game casual um yeah yeah two player game right yeah you're building a two- player game now you're building it and uh you did a great job now it's working perfectly fine but now uh the requirement comes uh no no now we want to be able to allow teams to play yeah instead of two players it becomes a two vers to let's say or multiple people can play now if you hadn't thought of it earlier when you were designing it and you put constraints in your architecture in your DB design in your Bo bases in your low level design high level design that will always be one player versus one player now it's going to be overall right yeah now it will be as good as like half half of the effort probably half of the effort of Orin building it will go into just for extending it from two player to multiple players but if you had thought of it earlier right and now you're saying that okay add when you were designing it you were like okay there can be multiple players for now I am allowing two but in future there can be but I have not coded for it but I have not put constraint that it will be one only right so then your change is become much easier you have thought of it already yeah yeah uh when when you talk about I think one thing which I missed in ND was extendability and great that this question touched upon that when you talk about a design it will always get evaluated from this point of view also in the future requirements one which are obvious or which you can think of uh it should be easy to make those changes right as a general good practice as a software developer as a designer architect you should put as less constraint as you can as you can right uh however of course you can't foresee all the possible changes which can happen in future but at least from your side you have to put lesser constraints in the design you have to think from generic point of view you you have to design your uh hld to low level design also in a very gen way I'll give you one example on low level hld right uh there's there's a meme about it also so bad a programmer good programmer will write destroy Earth as a function will WR destroy send the planet name yeah I know it's it's a very fun funny take on this but if you think about it the the thing which we are trying the joke is the layer uh point is trying to make is you make it generic you make the functions generic let's say you are saying U send mail right you you and you have to do you have to send an email through Gmail but you your function also in a way that it works with Gmail only probably named it also send email through Gmail now in future if you have to do something else you'll say okay I'll copy paste my code and everything right this is a very lowlevel flaw right uh you should have of course used very generic thing where you can say I have multiple implementation I can use a or b or c in future right not use it in the beginning let's say you just have to use Gmail in the beginning all great you don't have to use it use multi multiple things but you should be able to extend your design to use multiple things without requiring a redesign right now this is a low level right but if you think about high level also when you're doing high level designing also yeah yeah you have to think from the begin okay yeah thank you so much any more question guys want to ask there are a lot of questions in the document also we can pick that also yeah it's one of the question is in the Q&A uh that is if someone gets started with a small scale application and would choose one product for example DB to be used for building the application how to choose it like DB is there any parameters to consider here because sometimes there is a chance of migration due to internal flexibility and ease of use the question asked by DB you're talking about [Music] DB um I I'll take a little step back here so uh this session right yeah level design probably will not help you directly in your uh interviews because you'll be asked problem solving coding or something right very rarely you will be expected as a fresher to know high level Design Concepts right right it is still important why is that because of the projects you do because of uh knowing that how things work and I'll give you example when we when and I think it was one in one of the question also that uh when when you take interviews of people right of course problem solving is one coding is of course another thing you also go for their interest level what they have done what kind of project they have done and of course they will say some project I did a ticketing system I did this I Hospital booking Library I understand there are more academic projects but the choices you make in that right and how much research you did for that is important so this question when you choose a DB and you say that how should I choose it yeah more important question is what should be your re like how important is this to know that uh I have made the right choice right you should be able to justify so okay I can give you a short answer that you know if if your data is more structured if you need Atomic operations you can use more of RDM rdbms if you need more of a performance better sharting you can nol all of this you can get answered in one quick quick Google search also but most important is when you're choosing some technologies to be used it should not based on I know this or sounds cool or anything you should be able to do your research because you will get asked this question if you say I did a project I used DB for MyQ nobody's going to worry that uh whether it's a good choice or not most important thing is how did you come up with that what did you put in research so when you when anybody talks about a project in in an interview and especially being early career or freshers when I personally ask is okay what were the choices you make they should have done in and out of that project even if they use one technology I should have used which one I had the option why did I choose A over B yeah the questioning should be there if I agree with that uh argument or not most important is that they have done the research they know what they what they were doing right so for uh short answer is it's it's not my opinion which DB to go for on anything lot of data is available it should be that whenever you are doing some projects whatever Technologies you're using whatever architecture you're using you should be aware why you did that and even if you are taking you know that what something was a better design you are not able to do because of some time constraint or something you should know how you could have done better in that right let's say in your college per of course you would not put a load balancer right MH of course you will not but you should know that I could have I could have put multiple servers I could have put load Baler if I have to scale it later yeah yeah whatever application you'll come up with me and say you know this was the did I did and uh I couldn't scale it I couldn't the comp let's you did an internship project I did this this this I would ask okay how they were going to use it because your system is not going to be scale it's not going to scale then you should be aware that you know but we what we can do is we can use these we can scale it horizontally we can add more machines Auto scale and everything right we can have artical partitioning in DB right you should be aware again maybe not use it in most in some cases in some cases you might be using it actually but you should be aware that what were the options do your research for sure thank you yes yeah um hi Von I'm kamakshi and I'm a second year student it's great to have you over here so my question is uh when we are building a system that needs um to grow and handle more users how do you decide it's time to stop using a single allinone structure and shift to uh you know a structure which uh which you how you know it's time for you to shift from a monolitic architecture to a micros service and has there been a time where you um had to choose between both of these structures so I let me divide this question when you're talking me when do you know that you need to move from a single like a a non-scalable architecture to a scalable one then they talking about monolithic versus microservices first of all there are two different questions it's not that monolithic applications cannot handle skill just it it might feel like that in some constraint way but it's not the case lot of monolitic applications handle a huge scale also when you think of monting application you think that uh probably it it feels like that there are some certain service and it's reserved no monol Services can also scale okay so first thing first when you talk about monolith versus microservices it just means that here there is a big single deployment happening in most and actually can be multiple different deployments also and here there are different Services which are designed in a way that each service hand a very specific thing right so in this service uh the probably will single technology and everything same language and everything this this service each service can have its own language like you know one service you can make in Java other one in node just then go right the difference in terms of scale yes it is easier to scale microservices because then if you know that the bottleneck is this service you can just need to scale that service right if you know that you know now because of some XYZ reason uh this service is you know not uh it's it's overutilize it it should be scaled right you can scale it you don't need to just touch the other application unless they are also having the same uh performance issue in monolith it is tougher but you can still handle scale right so these are this choice between monolith versus uh micros servic is not based on scale it is based on how easy you want your application to be able to change of course um microservices have overhead also right very popular now most of the designs which we do right now last company also this one all the designs which we think about have microservice uh microservices architecture because then you can have different teams owning that microservice ownership gets Define Val it's fall the uh the blast radius gets reduced let's say this service goes down now the other services know that what to do when this service is down they can probably stop talking to the service Giver little lower experience they can say everything else is working but the email is not going on you know it will say you know you might get the email little later fine if one service has a bad design bad architecture or as a bug it can get fixed at the service level instead of affecting other other microservices so it's a better design that way because you can think of it as a the parts of a Machinery right you can replace one of the part and the other parts work fine right as long as the way they are interacting with each other doesn't change in monolith is slightly tougher right but in terms of performance lot of time monolith do better because the overhead of talking to each each other and uh have maintaining this uh communication can be saved but again it comes at a cost of that it's harder to make a change in one of the uh one of the system without affecting the others right so um it's a it's a design philosophy both of them can handle skill now coming back to the first question which was uh when do you decide you need to scale the short answer is when you're designing for the first time you need to think from scale point of view from the first time you need to ask the people whoever is giving you requirement that what are the scale we are looking for what other the scale we looking for in future right so they say no no we just need 10,000 you still ask if it goes beyond 10,000 that time you will need lot of Dev work are you okay with that yeah yeah they will say no no no let's make it in a way that if it goes till 100,000 also we should be able to then you say okay then let me design it in a way where it can scale up to 100,000 in future right now I will not because it's the cost thing but I will design it a way that it can go 100 ,000 and probably 500,000 also but after that I know that there are some limitation which I have still not taken care of right which needs to be taken care of later right so yeah so short answer you need to take care of in the beginning you need to ask questions before designing anything that what are we building it for we have you always design and I think one of the thing which also the previous question was what have you learned in your 16 years you always design things for long term don't design things for shortterm because whatever you design for shortterm will be happy uh great for some time whenever the requirement changes scale increases or anything you're going to suffer it's better to design things for long thank you so much for the question guys yeah g hi Von I'm Gori from Scot 59 so can you please share some examples of common mistake that developers make in high level design oh great question um yeah so common mistakes which developers make in high level design apart from the one which I mentioned right you know um that uh I also covered it here that you can have uh maintainability issue cost issue Etc but apart from the ones which I not did not mention uh was that they design things uh in a way where services are doing more than they are supposed to and I'll give you a simple example let's say let's talk about lowlevel design and then we can try to cover high level design let's say you're designing a class right now that class is supposed to um do a very simple task which is supposed to um it's a user user class which is supposed to get the data from user save it in DB get show it all good right now at now there's a requirement that you need to do something on that data that's some aggregation yeah now of course you can of course create some user group class or something right or you can take a shortcut and say okay I'll add few functions which will do the job right this is a very common mistake which happens in lowlevel design that you find something to do something very specific and then you start adding things to the same one right similarly what similar things happens in high level design also you design a service to do very specific things right you want it to have very well defined API contracts that okay this is how you give me input this is how I give you output right I'm not supposed to know what you're going to do with outside of this right but over the period of time or even in the beginning itself people try to make Services which are which are doing multiple things right which are unrelated but they're doing it because it was easier I already had that service up I added this thing also right so what how does that hurt the designers because over the period of time that code becomes less maintainable now there are two different things which should could have been separated are on the same one now if there is a problem with one of them the other one suffers right uh similarly let's say you are having two different services and they are sharing a database like that's a very anti microservices design that you why would you share a database right if you want to share a data then you have to have some other way you either maintain your own database or whenever you need the data you call the API of the service so common mistakes are when you try to keep the concerns not clear when you don't Define what my each service is going to do um yeah I think that's the most common mistakes people make and of course other things which I covered right uh not thinking of scale in the beginning not uh thinking of possible use cases in the beginning and designing the key we need this only no okay this is there there is a solution then whenever so this is very common discussion in any company even uh in gamecraft somebody will come with design the other people people will have the discussion and ask questions very very pointed questions that okay what happens if we have to make this change in future right this design will not allow then we say okay no no this is how we can do or we or we change the design so future cases you have to think of scalability you have to think of future scalability right and also make sure that your um the services classes high level low level everything designed in a way that you know what is the clear scope of a service class and everything is and you try to maintain that sanity right defitely thank you yeah hey guys any more questions you have would like to ask yes hi vun nice to have you here so I have a just like uh question like with rapid advancements in technology how do you evaluate with emerging trends of our worth integrating into gamecraft system uh how do you say that again the last line uh how do you evaluate which emerging Trends are worth integrating into gamecraft system great question I think that was in the document also so uh say first thing first you need to be you need to stay updated with the trends for sure right also you need to be able to uh see that how it is actually impacting not just that it can do this but what is the result it has bring in I'll give you one simple example let's say let's say you talk about art AI right now of course great you know when AI again there are a lot of sub branches like machine learning is one of them so you say okay machine learning is great it can be used to do this this this this but you need to also see where it is doing how is it able to help other organizations or uh in theory even even some papers you need to be you need to be aware that how it can help right and then you need to figure out the use cases for yourself or the other way around happens you come you come across a use case and that time you think okay what should I use here and then you say okay why not solve it through ml or why not solve it through this new technology or why not solve it through this architecture so you think from your point of view what you want to achieve and then try to see which is the best option instead of thinking what is the trend I need to go through that I need to implement just for the sake of it don't do that always think uh that what are my use cases which I can solve and it doesn't have to be a new use case also it can be a old use case where you know that something is there it's a legacy system n which is working but you now you know that there are much better Technologies or better architecture go for it I'll give you one simple example right and if there was a paper published uh by our DS team which is data science team uh regarding the fair play for game stra so essentially it was uh using machine learning to learn from people their moves and identifying patterns which which will help which helps us in giving signals that okay it's a good game it's a fair play or not right now doing that without ml will

2025-01-05 03:53

Other news