Test Mocking for Bulletproof UIs
Thank you so much for being here, everybody. And a particular thank you for Artem. To Artem for being here. I'm going to introduce ourselves and then So we're going to structure this kind of into thirds. The first 20 minutes, I'm going to give just a brief presentation just to kind of give us a context For what we're talking about here. So we're going to talk about MSW. We're going to talk about storybook We're going to talk about component testing and then kind of three different techniques you can use to mock the various dependencies in your tests. And then the next 20 minutes are going to be kind of just a casual discussion between Artem and I all about mocking and testing and kind of anything he thinks is worthwhile talking about. And then the final 20 minutes, we're going to leave plenty of time for your questions so that we can We can hopefully answer them. And then we'll go from there. Yeah, thanks again for being here. I'm calling him from Colorado for anybody curious. I can see the mountains just outside my window. It's a gorgeous day.
Yeah, thanks for saying hi, everybody. Let's get going. So we're talking about test mocking for bulletproof UIs. I'm Kyle Gatch. I'm a DX engineer on the Storybook team at Chromatic. Artem, you want to introduce yourself? Yes, my name is Artem Zakarjinko. I'm an JavaScript engineer, I'm doing a bunch of open source and I'm also doing a bunch of education at Epic Web, especially dedicated to testing. Thanks. And in case you're not familiar, Mock Service Worker is this amazing library for mocking out. Apis of all kinds. I'm sure Artem can go into a lot more detail, but the primary way I've used it is for mocking like network requests like a REST API or GraphQL API, but You can do a lot more than that. And Storybook depends on it very heavily for its mocking capabilities, which you're going to see. So storybook, what is that? I imagine a few of you probably know this already but It's a front end workbench for building your UI components and pages in isolation. So that means like separate from your app that you're actually working on.
So you develop those components in isolation. You can test those components and complex UI and interactions. And then you can document your component library or pages or whatever you're building. So it kind of has those three pillars. You develop, you test, and you document. And over here on the right-hand side, you're just seeing like a basic screenshot of Storybook. Where you've got your stories and components organized in the sidebar on the side there. And then you can see the component previewed on the canvas.
And then some add-ons in the panels down below the canvas there. So… Storybook is composed of what we call stories, which is just a way to take a component, in this case a badge component. And apply some configuration like props or what we call args so that they're like agnostic to whatever you're building in. So in this case, we're applying a label arg And the value hello world. And since it's in the default export there, that'll apply to every story in this file.
And then the export const small is our first story. And so there we're applying the small value to the size arg And then large, similarly, we're playing the large value. And so you use those to capture states and configurations of your components, and then you view them in the sidebar in your In your storybook. Now, we're going to talk about testing and then mocking specifically. But before we do that, I have to kind of give an overview of what we mean by component testing here. Opponent testing, we believe, is the best way to test your components, specifically UI components. And here's why. So it's kind of this ideal mix of a bunch of types of testing.
So let's start with end to end. So it allows you to simulate behaviors just like a user would use your code. Similar to an Indian test would. But you don't have to spin up your entire app. And that's where it's like a unit test where it's only rendering the component that you're actually testing. And we're an end-to-end test is kind of black box where you um You can't really manipulate what's going on. You can just visit a URL and then do your thing. With a component test, it's more similar to unit tests where you can actually like take in a module that your component depends on and actually manipulate that module, which we'll see in detail in a second. But in that way, it's kind of a white box test.
Which we find is really powerful for being able to access every possible state of your components. And then it's similar to manual testing, at least in the storybook experience, in that you develop and you test at the same time in the same tool. So what does that look like? So remember that stories file I just showed you? So here's a stories file with a component test. And it's specifically things like the function call here, which allows you to pass like a mock function that can be spied on.
To your components. This play function is where you actually write your test. And that contains things like simulated events, which we use user event from testing library to do. And then assertions, which we use expect from vtest to do. So you can see here we're typing into a search box with the word query. We're clicking the search button. And then we're expecting that the on submit arg, which remember was our mock function.
Was called with that query argument. And so… In component tests, this story becomes a test. And on top of that, we built this experience that we call storybook test. So that looks something like this. And what it allows you to do is run your component tests from the Storybook UI instead of the CLI. Now it's running the COI behind the scenes, but from an experience standpoint, you don't have to leave Storybook.
And the way that works is it actually uses a VTest plugin to transform your stories into real V tests, tests. So that when the tests run, it actually doesn't have anything to do with storybook anymore. It's just running v-test. And that makes it very fast. I'll show you a demo later where You can run about 90 tests in like three seconds, at least on my machine. And then it reports all those results back to the sidebar and puts like status indicators next to your Stories, which makes it really easy to know what's failing. And then you can click on those. Looks something like this to get a menu. And then that'll point you to the right add-on so that you can debug it easily. And you can see here that it's not just component tests. We built this to be extensible. And so while these don't exist yet.
Well, accessibility exists, but like performance and grammar Those don't exist yet, but you could easily see how you could do multiple types of tests all at once. And again, all in Storybook. There's a watch mode. You can run tests for your entire storybook or you can just do it for a single story or a component stories. And then if you do have failures, like in that nine plus that's in the red and orange down at the bottom there. You could filter the sidebar by those failures so that you can focus on exactly what you need to fix. And I just realized I forgot to announce, if you have any questions about what I'm sharing, feel free to put it in the chat. I'll do my best to review those by the time it comes to Q&A.
So that we make sure that we hit them all. And Artem is kind we volunteered to answer any that he knows the answer to. Yes. All right. So that's Storybook Test. It's a way to test your components directly inside a storybook.
And it uses the concept of component testing to do that. Where you simulate the behavior and then you pull it up. Oh, I forgot this. The key to component testing is that it happens in a real browser. It is not a simulated environment like JSDOM. It is a true headless, which means you just can't see it visually. Headless Chrome browser that's run through Playwright in V test browser mode. Which gives really high fidelity for what you're actually testing. So anything that uses like complex browser APIs. Those aren't the simulated version that you get in something like Happydom or JSDOM. They are the real deal. Which means you're testing the real deal. That's a big deal.
Here's what we're here to talk about. We're going to talk about how you mock your various component dependencies in Storybook tests. So one of those dependencies are things like modules. So when you import something in a component and use it That is a module dependency that your component has. And we're going to show how you mock those modules and manipulate them so that you can test what you need to test and especially so that it's consistent as you're testing it so that you're not accidentally testing a implementation of a dependency But you're instead… testing the component under test directly. The second kind of dependency we're going to talk about is a network dependency. This is where MSW or Mock Service Worker excels where You don't want to accidentally test whether your network is working or whether your API always returns the same result. You want to abstract that and mock that network request so that it's something you don't even need to think about. It is always consistent.
And you have total control. And finally, we have context. So this is common in React, although not exclusive to React, where you have like a context provider for something like a theme or a router and your components can depend on that and you need to provide that. To your component under test so that frankly, so that it renders correctly most of the time, but also you can test things like navigation. All right. So first we're going to talk about module locking. So in Storybook, we built our module mocking experience on top of a tool called Subpath Imports, which is actually a standard in Node.js. And so you define it in your package JSON like this through an imports map.
And Storybook is built to recognize the storybook condition And so for Storybook, we're going to pass… When you import the hashtag that so you might have seen like an at symbol or something in like Next.js or similar libraries for subpath imports, you have to use the pound sign. That's the standard. And so when you import pound API in Storybook, it's going to import the API or it's going to resolve to the API.mock file. But otherwise, everywhere else, it's going to resolve to just your normal API file. And so this allows you to mock your modules without changing any of your component code.
Um and our module mocking is also built on top of vtest mocking utilities like function And then we're looking into something like the vi.mock API so that you can do inline mocking. It's a little trickier than you might expect. And then finally, you can manipulate that mock in your test, which I believe I have an example for here. So, yeah. Okay. So here, this is another stories file. The kind of second code line there, import save note from pound sign app actions.muck And that .mock is only necessary if you're using TypeScript and it just gives you the right types as you're using that mock function.
But it'll resolve without the dot mock. File extension there. And so that's importing the mocked save note module. Similarly for create notes, it's pound mocks slash notes And you can see that where we have const notes equals create notes. So we're going to go ahead and initialize that module. Which then allows us to Pass that to our… our note UI component as an arg where so the note data is the first note in that notes mock data. And then we go ahead and we click the save button
And then we expect that the save note Let's see. The save note like action, which came from our mocked actions. Is called when you click the save button. And so you can see that this is using the real component is clicking the real button. But because we're using mocked modules, we can go ahead and confirm that everything's wired up correctly. Without having any sort of… side effects by using the real thing. Okay, so that was module mocking. Now we're going to talk about network requests.
So in Storebook, this is built on top of Mock Service Worker. There's an add-on for it, which allows you to configure your um your mock service worker handlers directly on the story. It'll intercept REST, GraphQL, and more. And you can configure it per test. Per component or even for the entire project can be helpful where you kind of have default handlers.
And so this is, again, a stories file. I've kind of abstracted it so that I could fit it all on the screen but um So we've got our test data that we're defining there, and that's just going to be the mock data that we use throughout the story. So for this mock success story We're using parameters, which is just the configuration that you pass to each story. And then in the MSW namespace, we're passing the handlers property. And here we're going to intercept an HTTP request so that your RESTful endpoint URL.
And when we do that, we're going to return a JSON response of that test data. And so that's showing the success state where we're just going to take that, what would normally be returned from the API, and we're just going to force it to always return the test data. And it's not even going to hit the network. And then the second story here is simulating an error. So here, same thing, same request is made, but this time it's going to wait 800 milliseconds and then it's going to return a status 403 error.
And this allows you to render your component in that error state again without hitting the network at all. And the third type of mocking that we do in Storybook is context, specifically context providers, which I said is, again, common. For React components. This uses a storybook concept called decorators, which is just a way to wrap your story in arbitrary code or components. And you can make them configurable, which is what I've done in my example over here. So here I've got not a stories file, but the preview configuration file for Storybook, which applies to every story in the project.
And so I've got the theme provider from styled Components. I have my themes, which is really just an object, white and dark. And then I've defined a decorator here that reads from the story context to pull the parameters on a story. And so that's going to take… It's going to read the theme parameter and it's going to default to light. And then it's going to pull that off of my themes that I've imported and just apply the correct theme based on the parameters around the story that's being rendered. I think. Nope, I guess. So then what that looks like is um In the story, you then define parameters, theme.dark, for example, would then force that story to render in the dark theme.
Okay, that was a whirlwind overview. Thank you for that. But hopefully that provides some context. For the discussion that we're going to go into now, and then especially for your questions. I see that you've already put some in. Thank you so much.
Let's go. Artem, let's talk about testing and specifically mocking. So in your expert opinion, and make no mistake, you are an absolute expert in testing and mocking, no matter what you say. So in your opinion, what is mocking and what does it actually do to your code? So mocking is a technique to distribute what I call test boundaries basically to draw the lines in your test In this case, for example, your component tests. And to say that nothing beyond those lines matters. And what I mean by that is that nothing beyond those lines can influence the behavior and the result of of this test. And this is where this is a lot of people maybe misunderstand mocking. There's a lot of questions that stem from this misunderstanding, like when should I mock? When should I not use marks, over mocking, under mocking? But in the end, it's just the technique to kind of draw this line. And to define this isolation, right? Like you mentioned, we have these component tests. And the point of these tests in particular is to run in isolation of this controlled environment. In this case we have storybook stories which is an isolated environment to showcase different states of your ui
And you achieve that isolation partially because of mocking. So you draw these lines, you can exclude network, for example, you can mock out different dependencies work. And it comes down to two things when you use mocking. You usually use it for either excluding certain behaviors from your test, for example, or gaining control over certain dependencies or side effects. And sometimes it's both like when we're talking about network for example you are excluding the network so requests are not happening in actuality. But you're also gaining control. Like you've just seen, you can define different response scenarios based on the story that you're trying to show. And sometimes it's just exclusion. Maybe you have some navigation, maybe you have some dependency that has no reason to be running the test. So you can use marking for that as well. So that's sorry, a lengthy introduction, what I see is mocking. No, very helpful. Um.
And so what's it's it What's it doing to your code? Yeah, I think… like when you when you imagine, I have a very nice website. Maybe I can show that just a second. Absolutely. Yeah, feel free. Yeah, I have a little visualization for mox Just a second. It should be this URL. Yay. So let me share my screen and I will show everybody.
Someone else is sharing. I will send a request. Please approve. Of course. Okay. So I will share my desktop. Cool. So hopefully you can see my screen in my browser. And I built this little visualization app to build this app help people understand what mocks do to their code For example, in tests So imagine if you have a test for this like one GS module And this module has a bunch of dependencies, so it depends on this 2GS module through some method maybe calls The method from that module and then the module itself has its own dependencies, maybe taps into node modules, some third party dependencies And if you don't do anything about this in tests, you will achieve your end-to-end test in terms of that everything Executes during the test run. Sometimes that's what you want. So this is an important distinction that You are in charge of marketing. This is not something you must do. It's something you choose to do. And I will talk about why in a second. So if you just do nothing about this, you will run all of your dependencies.
But very often you want to trim down this dependency tree to particular points, because this will allow you to focus just on particular behaviors. For example, what if I just want to test this a method? So I can just draw this line here, this boundary, this mock. And suddenly I'm in charge of everything from up here. And this means I can just ignore it. Maybe it's irrelevant for this use case. Or I can also control it. Maybe this a method depends on how this B method behaves. I can tap into module walking in this case, and I can gain control over this B method and module different scenarios. Maybe it throws an error and I can see how my A method reacts to that. So this is exactly what happens to your code when you mock. You draw this line.
And then you decide what you want to do with this. So do you want to completely exclude this? Do you want to control this? And it's important that you still be faithful like in terms of what these dependency do. So you don't want to test against something that doesn't exist in production right And this is where it's very important to have mocks If you decide to do mock implementation Have them reasonable and not like deviate too much And what I talked about, the purpose, I think. Well, it allows you to scope down your expectations. So now nothing here in this dependency tree can affect your test. Because your test is in charge. I'll follow these dependencies. So that's the answer what mocks do to your code.
I can also share this. This URL as well. So if you want to click around, this is a Thanks, Arna. I had no idea that. Website existed. That was really helpful. All right. I think when you're deciding whether or not to use a tool, it's just as important to understand when you might not want to do it. I'm sure there could be an over-reliance on mocking. So can you speak to that a little bit Yeah, I think very often the struggle when it comes to mocking involves this lack of understanding when you need it. And there are cases when you don't really need to mark anything. And it's really important to understand that If you're testing certain behaviors, well, you obviously don't want to mock those behaviors. Like in this example that I just showed. You don't want to mark the A modules because then you're testing your mark and that's a useless test.
But they're also at the same time use cases that are pretty much must have for marking like the network. In the context of component testing. It comes down to your decisions. This is kind of the circling theme. That I started a little bit early in the conversation so you decide If you need the mock and what kind of mock you need based on what you want to test right now. And I think there's a very good also resource on this. I will share it early on. Go this one. So you can definitely explore this deployed workshop. This is a workshop called mocking techniques that I prepared for Epic Web.
You can explore it for free. It's deployed. You can see a lot of explanations. You just can see the videos because those are paid. But the materials are there and I go into depth Why you would use marks, when you would use marks, maybe some rules when you must use marks in comparison. I think I have one Here, the good explanation on over mocking in the first exercise. So this is definitely a must-have. This whole resource, it walks you basically through everything. What are mocks, why you need them, when you need them, and then all the mocking techniques in JavaScript. Functions, date and time, modules, network, anything. Everything is there. It uses vtest, which is very nice.
I think… Here we go. Sorry. Zoom is… Yeah, now you should see it in chat. And also my previous example for my previous example For the app. Here we go. Yay. Yeah, thanks. Yeah, so definitely check it out. That's a lot of information there, much more than we can cover today. But for a high level answer. Yeah, there are use cases when you don't need mocks and you definitely need to decide for yourself what kind of mocks you need. Okay. I'm sure you've helped a lot of people with this as you've put on workshops and developed the library itself and solved GitHub issues.
What are some common difficulties that you've seen when you're helping those people? I think sometimes i think sometimes there's a lot of struggle when it comes to setting up mocks. Because you have your UI components so you develop them Because they're real components and real apps, they have a lot of dependencies. They depend on modules and third party things and network. And sometimes people struggle to kind of filter those dependencies to create proper testing setup.
And I understand this. Testing setup is quite a difficult phase, but it's also the most important phase. And people often overlook it. But it's really important to understand your dependencies of the things you're testing and then introduce appropriate mocks like module logs that you showed, network mocks these will cover probably 95% of all dependencies you need. And then there's also more specialized box maybe you're Especially when talking about end-to-end tests, not so much component tests, but in end-to-end test, it's very often the question, how do I mock my database? Do I really run my end-to-end test against a mock instance of a database? Do I mock the database client? And there's a lot of different solutions there. I wouldn't say any of them is particularly wrong. Once again, it's the shifting of this boundary. How much of your app you want to test, how much of your database, for example, you want to test.
And then you can lift this boundary up or down and this gives you different kind of test. So definitely setup can be tricky and it's okay. I encourage people to embrace that this is probably going to be difficult, but once you figure it out That will pay off in test because you will have all the tools you need to test any component or any User flow in end-to-end tests. Other things, I think sometimes people think sometimes misunderstand what power mocks give them especially in network marketing, I often hear questions like how do i mock this particular endpoint when the actual server has a bunch of business logic behind it. So maybe I submit some form and the server tries to Validate the inputs, then produce different responses. And people are asking, should I really put all of this business logic into my mock?
And the answer is no. You need to understand that mocking gives you control over behaviors So they become given. So you can create a test case, maybe a story that says. This is how my component reacts to an error. And in the network mock, you just always respond with an error. So it doesn't matter when the actual server responds with it. It matters that in the context of this test. It always receives this error. So you're mocking, your network mocking becomes this boundary And it allows you to model different scenarios. So you don't need to know what business logic results in 403 or maybe some exception, all you need to know is that there's this contract And in this test case, you're just tapping into the end result of this contract and it allows you to model the test context and then just run your assertions. See how your component handles it and so forth.
I guess along that same vein, but maybe the opposite. What have been some of the best use cases that you've seen for mocking in a component test? Hmm. Naturally network is something you have to do in component tests. And there's a very good also reference in chat like very often people consider mocking coat smell and i disagree. It becomes code smell when it's poorly configured, when people don't know why they why they pull all of these mocks. Then it becomes a cold smell. But there are some testing levels, especially component testing, where you must have mocks. Things like network, things like third-party dependencies. You cannot really test your component in isolation if you allow those things to run. So you use mocks because mocking is just a technique, just the same technique as testing itself. As separation of responsibilities is just a tool. It's not evil. It's not cut smell maybe We can turn it into cold smell because we're engineers. We tend to make mistakes. We tend to over-engineer things.
That's fine. But in itself, it's a very good technique to use and it gives a lot of value because it provides you a lot of control. So network mocking is definitely a very good use case for mocking in general. I'm not a big fan of like mock functions, but sometimes they're really handy when you certain APIs or components that are kind of transparent. So maybe a component that has like unsubmit callback.
And then it is the usage, it is the parent surface that provides this callback. So in this case, it's very handy to have the Smock functions, which I also mentioned in the workshop. So basically just create v.fn or maybe fn from Storybook. And you have this special magic function that records calls to itself and keeps track of the arguments. And this is a very good use case for this too. But you should definitely be careful not to overuse this. I really love use cases when people use mocks to showcase their work. Hmm.
Surprisingly, a bunch of companies use MSW to deploy to some demo or staging environments and then just showcase, hey, this is what we've built And because it's a seamless interception Once, for example, the backend is ready, you can just turn MSW off and your components communicate, your UI communicates with the actual network. Which is really nice. Hmm. Yeah. I'm actually going to try to share a different part of my screen and show Let's see. There we go. I want to show um what I think is a pretty cool example of mocking here. So here we've got… I believe this is just Next.js's kind of to-do app. Showing how to use RSCs, which we kind of sort of made it work in storybook and then the React team changed that API significantly and now it doesn't work at all but The mocking concept won't get across. So here we've got this story. Login should get OAuth token and set the cookie.
And so what we do is we… we find the we find the Log in to add button. Here, let me just go back. So one thing I didn't cover is when you write a Component tests and a play function in Storybook, you get this interactive debugger And so I went back to the beginning of it and now I can go forward. So it looks like we're making an assertion of something not being there and then We click this login to add button. Well, first we find it. Then we cook it. And then we make sure that um this use router.push method was called with the correct route.
And finally, we expect, I'm not sure why this is undefined. But we expect the user Which it did not work. Let me see if it works without stepping through. That's interesting. Hmm. It's… It passed every test when I did it. So it must be a little flaky for some reason.
Yeah, it's just live demo. In any case, what I really want to show you was what that story looks like. So let me go to the code. So we've got this. In our stories file for that page component here, we're importing db.m. And so I just want to show you what that mock file actually looks like here. So it's a prisma database and Prisma has this great library, Prisma mock So you can just create a Prisma client mock And then… We export that database and the initialization.
So that we can just Let's see when does we initialize that? So we initialize it for the empty state. And then for that. Login. So here we're using MSW and so we're mocking network requests and we're mocking modules. So we've… When we hit GitHub's OAuth. We go ahead and intercept that and just return the correct access token to sign in.
And then similarly with the user API for GitHub. We're just um Yeah, we're mocking that authorization request. And then we're mocking the implementation. Of our login function. To return the correct token. And then making some assertions on that. So I just think it's really cool that you can mock the entire Prisma database and then make sure that your components are wired up, but not actually worry about the implementation of that database. It's a really nice separation of concerns. Okay. For my final question here, what are you excited about lately in the world of mocking or Testing. I'm really excited about the kind of the whole movement of component testing i'm really happy to witness this renaissance like we are migrating from browser-like environments like GSDOM and HappyDOM to actual in-browser testing with tools like Storybook.
Of course, we just browser mode, there's component testing in Playwright and Cypress. This is just so good Because we've been testing in this contrived environments for decades. Almost two decades and it has It had its applications definitely when it first came out and sometime after that. But as tools developed, I think there's really no reason to use browser-like environments today. And I really strongly encourage people to explore these different solutions to look into how you can do this.
I'm also working on component level testing workshop for Vita's browser mode, which is really cool. People absolutely love the material like your tests from React Testing Library in GSDOM are almost identical in Vita's browser mode. You just swap the inputs. Suddenly the render function comes from one package and the screen function, which is page. Comes from a different and in your practices, your actual structure of the test is still the same. That's really cool. I always come back to the fact how influential React testing library was. Yeah. Yeah.
It's very nice to see its you know echo in different solutions right now. Mm-hmm. I'm also pretty excited about GraphQL mocking, I think we finally can complete the GraphQL support by adding subscriptions.
Hmm. And I already have AR working and ready. I just need to polish a few things, but since we have WebSocket support, now we can just use it as the underlying layer for GraphQL subscriptions. And it's so nice This is really cheating because MSW follows the standards so For us to support things like GraphQL subscription, I just have to look at the standard and how you actually use subscriptions and model the mocks out of this. So it's the same some PubSub object. You publish data to it. Hmm. We almost don't have to do anything, just create this layer this api for people to use. I just love it.
That doesn't sound like cheating at all. That sounds like smart engineering. I mean, I know… When MSW had its big 2.0 version, a huge reason for that was to bring not just the APIs, but even the fundamental like underlying engineering more in line with web standards that are out there. Precisely to enable things like this, I imagine, so that you weren't constantly maintaining a bunch of custom code just to make something work.
So. Yeah. I was very happy to remove all of that code. Some of the best VRs I've made because it was just deleting everything and relying on fetch API. I'm glad to hear that's still paying off. Okay, at this point, we're going to transition to trying to answer your questions. And I see there are quite a few of them. Give us a second while we review them and um I've been trying to answer some. Typing them out, but I'll probably start just verbally answering from this point. Let's see.
I have no idea how to use Zoom, honestly. I'm trying to select a username and it just refuses Oh, yeah. It looks like the chat is more limited than a typical Zoom call. Like I noticed you can't like mention anybody, you can't react to anything. There's no reply threads. So, yeah, it's a little… A little basic. Meanwhile, I can maybe answer some of the questions I see folks like following up. There's been a really good question regarding the synchronization of mocks. So you maybe have a reaper and you have front-end repo. And of course, there's probably some sort of contract api contract Mm-hmm.
And that was a good point that you need to synchronize those two. So there's a follow-up that, yeah, I shared that I'm not a big fan of committing the generated marks because I think you have an extra step that can potentially be out of date. So instead, I advocate for runtime generation. So you have your, let's say, open API SPAC. And no matter where it is you can you can pull it from there and you can use NSW gs slash source that just parses that spec and basically generates handlers for you So if your backend updated the SPAC, then your next front end run will pull the updated spec. At least one of the ways you can orchestrate that. Yeah, we had some discussion about this in terms of like runtime versus build time generation. I am a fan of runtime generation. I think it saves you trouble, especially because you rarely need the whole spec for particular tests. So you can also trim them down. There's transformer functions too.
To just pick the operations you need. Yep, that's basically the follow-up. Yeah, I was just going to say, I've seen multiple questions around that. So thanks for tackling. There's also Good. Yeah. Performance is always a looming question, especially when we come to like in-browser testing And I'm still exploring this. I have a repo with benchmarking. I started benchmarking comparing GSDOM and RTL to vita's browser mode And so far I have good results, but I haven't done any good More kind of real world benchmarking. I just started with a very simple scenario i have a thousand tests. They're very simple access the DOM, create a button, click a button, assert what happened. Not real world stuff you do, but still something Yeah.
And their results are very good. And I will try to elaborate more introducing React and some side effects and some actual UI updates and then we'll see. Definitely something I want to publish but when it comes to performance i always say That's not the most important thing when it comes to tasks But of course, it is important that tests are fast and the iteration loop is short. Component level test when you're talking about in-browser tests, yeah, they will be probably short, probably longer Or in comparison to GS dome tests. But it's not the only factor and it's not as long as you would think. Because there's a lot of also performance overhead that comes from this browser-like environments, especially when it comes to determining accessible roles. One of the pending issue with RTL has been that querying by role is very slow. But I believe that's not the case in the browser. It's just as fast as writing any query selector.
Hmm. So there's definitely a scale of things. And I think in the end, it's not going to be much slower than your traditional GSDOME test. So I'm really excited to dive deeper and actually have some good article to publish, not just Not just thoughts. Hmm. Yeah, I did not know that about the querying by role. Being just as fast. That's great. That's honestly one of my very favorite parts of testing library. Is it encourages you to write your tests in a way that make your components accessible? To more users, which I think is super important in our work. And I'm glad to see a tool that just kind of like gently nudges you that way.
No. Hmm. Hmm. Yeah, yeah. And you also have to consider like things like ESM when it comes to performance. Since we're running in the browser, we can now just run ESM. You can write tests in ESM. We don't have to compile them. Maybe if you're using TypeScript, sure, there will be a step, but no build step. Yeah. And then there will be a very good performance win when it comes to module logging, especially you can mock this through HTTP. And, you know, just coming back to the discussion that it's not just slow or fast, that's actually a trade-off from multiple sites.
Yeah. Sure. Sure. From both sides. One of the questions that was submitted um very rightly calls me out that in the context mocking section, that wasn't technically mocking. It was just rendering your component inside of another component. So the question is like Could you, yeah, it's more setup.
That's set up. But could you mock a context provider using module mocking? And yeah, I think you absolutely could. It's an imported dependency in your component, just like anything else is. If that's the route you want to go to give your give yourself more control or more… internal consistency and how you're approaching the problem. Yeah, you could totally do that. Yeah, I mean, it doesn't mean you should because like if you ask yourself you have this theme provider. What do you want to do with it? Do you want to exclude it from the test? Probably no, because it gives your components visual appearance. And that's important CSS styles can directly affect accessibility you can have certain elements overlapping or simply using styles that render them inaccessible. So yeah, you kind of want those styles. And then the question is.
Yeah. How much do I want to control this style component or emotion or something else? And usually with this library, the answer is like I don't really want to deal with that. So using the the wrapper set up like that Kyle showed is the way to go. And the same with routing Yeah. Yep. When you need to render components that depend on the route context, like React Router, you just wrap it in the setup phase and your components are ready. They're actually using all these hooks like use location and so forth.
Yeah. So probably not the best idea for module mocking Unless actually like a little spoiler, if you go through the mocking techniques workshop There is a section in module logging and I talk there that module marking is your last resort. I'm convinced you don't need module marking. In like 99% of cases, you don't need that. You just need to know which behaviors you don't want in your tests and just mock those. But there are cases where module mocking is good. And one use case is performance. Like when you're pulling some big dependency which you either don't need at Or maybe you need just a subset of it. Sometimes those dependencies can also be written a little incorrectly. For example, they can expose a lot of things in global scope. There's nothing you can do about So module mocking in that case is a really good thing. So maybe if you're using a styling library that pulls I don't know, a megabyte of code in your test. Yeah, probably then mock that provider. But that's probably not the case.
Okay. Actually, I'm going to continue on that just a little bit. So the example I showed was a theme provider. Because that's nice and contained. And I could demonstrate it in a pretty small example. But probably, as he pointed out, routing is probably a more common use and a more realistic use. For that sort of wrapping. With routing, your real app is going to use your real router, of course. But then like React Router provides like a in-memory router. And that's actually what you wrap around And that allows all the wrapping to happen in your isolated story But in a way that doesn't have any sort of side effects, doesn't try to navigate for real, that kind of thing.
It's pretty powerful. Pattern. Yeah. There's one more question I see people asking regarding playwright and MSW. So definitely something I want to answer. The question was about the performance, like about the usually about the at least the question is that playwright can be slow with MSW or MSW with playwright I haven't really experienced this myself. Like we have 240 something tests in MSW repo That use exclusively playwright and MSW. But I don't know what would be like the common reasons for this being slow. I don't know. Maybe fetching the worker, but that shouldn't be the problem.
There's like different setups. I usually recommend just enabling MSW in your app So then the first time it mounts, it fetches the worker puts it in the browser context. You didn't have to fetch it anymore. So you're not going to be paying this bandwidth price. I don't know what else. Perhaps there are some other areas to consider or maybe some other areas that affect this setup. If you have some reapers, I can take a look when I have a moment, like what makes them slow. I haven't really experienced this.
So yeah, that's what I can say. Thanks so much for all these questions. Having a hard time getting through them. Oh, a really cool question about GraphQL. When you mock queries like sometimes you have to define a lot of like fields and where the data is coming from It does sound like you need schema-first mocking. Because we do that and we support this and it's actually pretty cool because it uses just GraphQL packages as your production server does.
So definitely something you need to look into. This is usually a sign that you have probably a complex mocking setup. And this is where specifying fields individually may not be as flexible And I understand this. And this is very good Use let me yeah mock scheme. To use mock schema instead. I will share the link in chat. So the whole idea is that you can take your schema, either production schema or write a mock schema put it to GraphQL function, grab the query that MSW intercepts alongside the variables, feed it in the function. And define your source of data, your resolvers. So this is something I recommend. It's very flexible. It will actually give you field filtering like you used to. So if your client only needs ID, you don't have to return the whole JSON. So explore this. And this is probably what you Need.
Oh, I see a question about store book using both Jest and VTest at the same time. Okay. I don't see any other questions. I think what that might be about is a previous version of the testing experience in Storybook was built on Jest instead of V-test. But I don't think… I don't think you'll be using both at any point. Like the older versions would use Jest and newer versions would use vtest. We encourage you to upgrade. Vtest is faster. It seems to be receiving a lot more updates. We're actually kind of Yeah, I should touch on this. Partnering in a lot of ways with the V test team and so um While we've been building out this storybook test experience, we've um We've worked directly with the VTest team report bugs that we've uncovered and we've submitted PRs to their team. They've submitted PRs to us. Yeah, it's been a great collaboration to build that out.
Because as I said, it's built on top of their browser mode, which runs your tests in the real browser, Chrome by default. Yeah. Great sterrobic tests and playwright will be slower. Slower than what? Like it might technically be slower than just running in node through JSDOM, but… In our benchmarking it is barely slower.
If you're talking about if you're talking about running tests in Storybook versus running them in vtest directly in their browser mode. Again, because it's running ultimately in the background through the CLI, it's actually virtually the same. There is no statistical speed difference between the two. And even though it might technically be slower than like a happy DOM or JS DOM kind of situation.
The fidelity increase you get and being able to test the real browser events are so much worth the like 10% slowness maybe. At least in our experience. I already have a large library of MSW functions supporting my vtest test. Will I be able to use those same functions in Storybook? Yeah, I'd say so. You can just import them in your stories and use them that way. The way that you provide the handlers through the MSW parameter in your stories That API is exactly the same as MSW itself. And so, yeah, you should be able to reuse those. If anybody's familiar with the component encyclopedia part of the Storybook website.
That entire thing was built on top of MSW before the real thing. Every page, every component is fully mocked through MSW that way while I was building it. So, and then those same mocks are used in the real app so that I could spin up the real thing fully mocked.
And we also, I mentioned this, we spoke with Kyle before the meeting that we were working on the new version of the add-on that's going to bring it's simultaneously closer to your storybook experience in terms of how you initiate it and use it. But also closer to MSW experience. So there would be like different way to layer the handlers as you just used to in plain MSW. So it's going to bridge maybe a little bit of experience gap that we have right now. So yeah, once again to the point, it's the same MSW experience. You just provide the handlers of storybook routes them to the story correctly. I see a question about a question. Skipping tests in storybook So you do that by applying tags Let's see. Yeah, I'm still sharing my screen. So if I wanted to take this test here, I'm just going to collapse down.
The noisy things for a second. If I wanted to skip this. I would apply a tag. And then you can negate attack by doing this bang And then test. And that will prevent it from being tested. We're looking at better APIs for writing tests in Storybook. Let me go ahead and… If I can pull that up.
Yeah, here we go. So we like to operate through RFCs or requests for comments. And so this one, I'm just going to go ahead and put in the overall chat. That is focused on a new API for defining tests for your stories. In a way that has the syntax of the play function so like um your simulated behavior, your assertions, all that stays the same. But it provides a nicer way to write like labeled tests. And then at this point, we could provide a nicer API for skipping too.
If you have thoughts on that, we definitely welcome uh Any feedback you want to provide there for Whether it's going to meet your use cases or how excited you are. Okay. Tips on mocks and test organization. So not just yet. I'm actually writing a guide for that as part of… The Storobic Test Boot Camp. So I should talk about this a little bit. Yeah, here we go. So we're in the middle of what we're calling the sterobic test boot camp program.
Where we're walking through what storybook test is and how to use it. And we are available to help you use it in your project. And so we have like office hours two times a week.
And here's the schedule. So we've covered kind of a general overview, some testing concepts. How to get it running in your CI. We talked all about component tests. This week, we're talking about mocking best practices. And then next week, we're talking about coverage. And on and on. If you want to join this program, I'm going to put that URL in the chat.
That's just a Google form to sign up for the program and then you'll start receiving weekly email updates. And so the way we're doing this is we'll do a written guide and then where it's relevant, we'll do a video overview as well. So let me click into one of those. I can't click that screen. This one. So here's what a guide looks like. So this is kind of the overview of just all about how to start testing. So it's got the installation. Of how you install the storybook test experience. It's got kind of like an overview of what it actually is here. Some key concepts. As you go through.
And then finally, all the different types of tests that you could do within Storybook. So render tests is just that. It's testing that your component renders. It's technically a component test just without any simulated behavior. Then you've got the component tests. And then you can do visual tests. Full disclosure, I guess, Storybook is built by a company called Chromatic that does visual tests as a service. And so we built in the chromatic service into the storybook experience if that's something you're looking for.
And it basically does a before and after comparison of what your actual rendered components look like in the browser. You can also do automated accessibility tests. This is one that I'm super, super excited about. So Storybook has had an accessibility add-on for years and years and years built on top of X. And… now you've… You could write a recipe to do it, but it's always required quite a bit of boilerplate and like manual setup to do it.
Now built within Storybook Test, you can just run those accessibility checks right alongside your component tests. And it is fantastic. Let's see here. Then you've got snapshot tests if you want to test like an error state that is hard to represent visually, you can always fall back to snapshotting the DOM.
And you can actually reuse the stories in like end-to-end test or unit tests if you're more comfortable in that environment, but then you don't get the storybook experience. I see a question about where you get that guide. Go ahead and sign up for the program and then you'll get an email. Or I'll just go ahead and I'll share that URL. Just make sure that in Zoom you click on this like little blue stuff that says two and select everyone.
Because right now only I can see these urls i mean I appreciate the form. Oh my gosh, I've been doing that the whole time. Oh, thank you so much. All right. It's okay. I reposted the previous links, don't worry. It's okay. There's the guide. Thanks, Artem. I did not realize that.
Yes, please do. There's I know, and then we close to the limit There is one more question I want to answer because I see a lot of people asking very similar questions so What do you do with your like mock setups? Do you create repositories for them? How do you store them? Do you generate them? How do you keep them in sync and so forth Really, you're the best judge? For us, how complex your test setup should be. Because sometimes your apps are complex and that complexity will reflect on your test setup. Sometimes they're not. Just maybe good advice here is try to keep your test setup simple as much as you can. Because every extra layer of abstraction you have will be extra layer that can introduce bugs. Extra layer you have to go through when tests fail. So try to keep them really simple. That's why MSW promotes just inlining handlers and tests as the default. That's really simple.
You go to the test, you see this is the network. If you abstract it into maybe third-party package, now there's one extra step you need to go to see this is the network. If you generate it from the spec. This is one extra step to go. So nothing bad about that. Specifically, just bear in mind, don't bring more complexity to this than it should be and don't be clever in test setup. Just be explicit.
Make it absolutely clear what are the dependencies that affect your test. So I hope this gives good guidance because it's very hard to advise because I don't know the context of your app. So I trust you know that best, but keeping things less complex is always a good idea. Because you can always make them more complex. Thank you. Can't imagine better advice to end on than that. Well, thank you so much, Artem, for this conversation and for winding your expertise. Thank you, everyone, for joining us. This was a lot of fun and you all had excellent questions.
I'm going to, I'm assuming I can access these questions and chat. And I'm going to try to… answer some in the follow-up that I send as part of the boot camp so If you're part of that, expect some further answers to what we couldn't get to and Yeah, thank you so much for being here. I hope everybody has a fantastic day. Yeah, thank you so much for coming. Thank you, folks. And just remember, you can still shoot questions at me at kyle at the storybook team just you know reach out to discords and twitter This is not the only place to ask about testing or storybook. So hope to see more questions.
That is a great shout. Yeah. Thanks. Take care.
2025-03-17