DevLin2022 - Simon Brown - Diagrams as code 2.0
So this is a very short talk and then a demo. And the topic here is "Diagrams as code 2.0" which have been retitled recently and I've been calling it "C4 models as code" for reasons that will become clear in a second. So I talked about my C4 model for visualizing software architectures this morning. It's a set of abstractions and a set of four hierarchical diagram types and it's zoom in, zoom in, zoom in, zoom in. Diagrams are maps... I think I said that
quite a lot this morning. The System Context diagram shows you the thing you are describing, who's using it and how it fits into what's around it in terms of system integration points.And that was the example diagram I showed you this morning. So the red box is the system we're documenting, a couple of different users roles, actors, personas and a bunch of systems around the outside. Pinch to zoom in, we get to the Container diagram.
The Container diagram shows you applications and data stores, answers those questions. And this is the example diagram I showed you this morning. So a bunch of applications, a bunch of State Stores. We're all good. Notation independent. That's something I'm going to talk about a little bit later. This is actually quite important.
Tooling. That's what we're here to talk about. I had a bunch of questions after my talk during the break along the lines of what tooling do I recommend for doing diagrams. And this is exactly that thing. So most teams are still using general purpose diagramming tools Vizio, lucidchart, diagrams.net, draw.io, Gliffy etc. etc. These tools are very easy to get started with but they have a bunch of issues. Number one: you can do anything you want. So you open a diagramming tool it's a blank canvas. You add boxes and arrows to a diagram canvas and you can do anything you want to.
With the C4 context diagram, the System Context diagram, you should really only have people and systems on your context diagram. But if you're using a diagramming tool you can add other things like components. Now the C4 model doesn't really have many rules but that's one of them: don't have components on your context diagram. But that's easy to do with a general purpose diagramming tool, which is bad. Yeah I don't need to say much here.
If you try and export many of these... file formats to text you get a complete and utter mess. All the content and the presentation is completely mixed up and they can't be separated. If you want to draw two diagrams in a diagramming tool, say a context diagram and a container diagram...
You draw your context diagram, open up a blank diagram canvas, you copy paste some elements across and then you start renaming things and if you're not careful you end up in this situation. So, you'll notice the user here is called "Business User" and on the other diagram it's called "Report Viewer". Are they the same person? Did we rename one and not the other? Who knows? But they're inconsistent... Again same problem. These text formats behind Vizio etc. are not made to diff so they're not easy to stick in Version Control and try and figure out what the changes are. And you can't automate this stuff.
There is a way to automate Vizio to some extent, and you can automate draw.io, diagrams.net to some extent. You can feed it a CSV file but it is quite hard to do. It's also very time consuming. So when I was creating some example diagrams for
this presentation... what you won't notice here is that arrow is not straight. It's like one pixel out. And if you nudge the box one way or the other it doesn't it just it's always a diagonal line. You can never get it straight, And it's really annoying. And that's the sort of thing you spent an hour trying to fix and it adds absolutely no value. But once you've seen it you can't unsee it... So... ...back in October 2020... this was... the Thoughtworks Technology radar blipped "diagrams as code". Diagrams as code is a technique so that what you're doing is you're writing some degree of a text file and having some tooling generate some diagrams for you.
There's a bunch of example tools out there there's PlantUML, which is probably the most popular one that I've come across. There's Mermaid, there's GraphViz, kind of the original one. This is great. Diagrams as code is great because it's text and as developers we like text. We've got lots of text based editors, our source code control systems like text. We can do diffing and there's lots of opportunities to integrate these things into CI/CD build pipelines which is awesome. The one... Somebody has created, specifically to cover the C4 model, is called C4-PlantUML and what this does is it gives you a set of macros on top of the raw PlantUML syntax that allow you to talk about people and software systems and relationships.
So this little bit of text here gets you that one diagram. And that's it. It gets you that one diagram, nothing else. If you want to create multiple diagrams you have to have now two text files, one per diagram.
So therefore, do you remember that inconsistency where we renamed a user on one diagram and not the other? That's very easy to do once you start having multiple text files representing multiple diagrams. Now sure, we can use global search and replace in our tools and we can have an include mechanism in PlantUML to kind of share elements across diagrams. But this doesn't work as well you think it should.
So during lockdown I had nothing to do apart from go surfing, and you can't go surfing every day in Jersey... So I built something called the Structurizr DSL and this is a text based domain specific language that I built specifically to allow people to model C4 style architectures or rather architectures using the C4 model. And the key thing here is that it's model based, so you craft up one model of your architecture and you generate to multiple diagrams based upon that one model. That gives you consistency. So now all of your diagrams are completely in sync.
So enough of slides let's drop into a demo... So I have some additional tooling I built called Structurizr Light. This is a set of free tooling I've built. It's available as a Docker container and a Java application you can download and run. And it's completely free to use. So I've got that all installed here and I'm going to start Structurizr Light against a folder on my laptop that does not exist currently...
So here's my examples folder... I'm going to point Structurizr Light to a DevLin folder... ... and fingers crossed this will work... So this is now booting up Structurizr Light... ... because that folder doesn't exist it's going to create that folder for me... And inside that folder... there we go... is this thing called a workspace.dsl file. if we open up this workspace.dsl file... this is essentially the Structurizer DSL.
So what we're doing here is we're defining a workspace. A workspace is nothing more than a wrapper for two things. Number one, a model. And the model is really all of your people, your software systems, your containers, your components and the relationships between these things. And thing number two, views. And this is a set of views that ultimately will get rendered into diagrams in some rendering engine. So in this example we are defining a person with the name "User"... nice huh... and we're assigning it to a variable called user. We're defining a software system with the name Software System...
...when you do this yourself please use better names than I have... We're creating a link between the user and the software system with the description of "uses". Those of you who came to my workshop yesterday, please don't use "uses" either... I'm showing you all of the very bad examples here... So that's our model, it's a person, a system and a link between them.
We are now defining a System Context view, we're defining a System Context view for that Software System we defined at the top. Ignore this it's just a diagram identifier. We're saying "include *". What "includes *" here does is it includes the Software System and it includes everything that has a link to that Software System automagically. And we're saying apply automatic layout. And that's it. That's a nice simple example of a structurizer DSL file defining a workspace.
But we need some diagrams. So where do we get diagrams from? So Structurizr Light is a... a server-side rendered... Java spring boot application. And it will render you a bunch of diagrams in your browser. So if you go to localhost:8080,
...which is where I've done a port map inside Docker and outside Docker... ... always read the T's and C's... accept, of course... they basically says don't do bad stuff. I promise! ... and there's our diagram. So that little bit of DSL you just saw creates this wonderful "two gray boxes and one arrow" diagram. Now, let's start messing around with this... so I don't like automatic layout. I built this tooling to to basically be "manual layout first", so if I comment this out... and refresh... now I have the ability to move all of these boxes around and
change the page size and all sorts of stuff, like that, I can add a link here... I can move the text a bit further down, I can change the routing, I can click "save workspace"... I can go back here and I can change this back to let's say Simon... ... go back here... refresh... and there's a there's a merging algorithm that is trying to figure out what you've changed and where that box should be sitting on your diagram. So the Structurizr engine will try and retain the diagram layout as far as possible. You'll notice there's no X Y coordinates in this text file...
which raises the question, well, where are they? And the answer is they're in this file here, so there's a Json document that's created for you automatically by the tooling. This is actually the data structure that all of my structurizing tooling uses under the hood. And it's really just a Json document just describing the model. ...there I am...
There's the software system, there's my system context view here and you'll see some XY coordinates so all of this is kind of done for you behind the scenes. So the the DSL format is kind of layout agnostic, rendering tool agnostic and it's the Structurizr Light engine that does all the rest for you. So let's close that... let's revert this and go back to automatic layout left/right... ...we'll go back in here... refresh... so that's why I'm going to diagram we had before. Who likes gray boxes? Yay, who doesn't like gray boxes? Everybody else! So how do we fix gray boxes? You have a bunch of options...
I created the ability to add themes to these things and there's a default theme... Can anybody guess what the default theme does? Blue! Correct I know I should not have done this... This is where we figure out how to have Wi-Fi... there we go! So that automatically applied some styling to those boxes. But we don't want to do that... How do we add styles ourselves? So we need a little Styles section... I'm going to save this... go back... we'll get our gray boxes.
Every element and every relationship has a set of tags. Anyone who's done any web development, HTML and CSS, you'll be familiar with CSS classes, this is essentially the same thing. So if I enable tooltips on this diagram... If I hover over the mouse you'll see there's a little boxing element and a box saying "person" so the person element has an element tag and a person tag. The software system has an element tag and a software system tag, the relationship has a relationship tag and what we do to style this thing is that we basically create a bunch of styles for those tags. So we can create an element style for element and we can say backgrounds... what color do you like? I'm going to go green... so we're going to go green background
hopefully a white foreground... So what I've done now is I've changed all of the colors for all elements. If we want to have people in a different colour we can add a specific tag for people... There's some cascading here like you see with CSS... What's this going to look like? Now we have blue people and green software systems... Who likes plain boring boxes? Well not really... How do we change that?
There's a person shape... and you just basically do the same thing over and go over and again you can change border styles and you can individually style elements and that sort of thing. for those who came to my workshop yesterday I said you should create your diagrams and include a diagram key... This tooling will generate your diagram key automatically based on all the stuff that sits in the model. So that's kind of an introduction to creating your first diagram with the Structurizr DSL and there's a bunch of other things I'm going to show you in a second.
So let's imagine we have our context diagram, and it's awesome, and now we want to create a container diagram. How do we do that? With C4-PlantUML you would craft up a new file with this one you just add more things to this same file So we're going to add some things inside our Software System and we start typing away... Let's have a whatever that means a container called Web application and we'll have another container... oh my goodness I can't type... called database and we're going to say that the user has a link to the web app "uses"...
and the web app has a link to the database "reads from and writes to"... Right, so if I save this and refresh... What do you think is going to happen? Nothing? Why nothing? Because there's no view! Correct.
So all I've done is I've added things to the model but I've not visualized those things with a view so how do we fix that? Well, we just basically copy paste this block here... We change the diagram identifier, we change this to "container". So what we want to do now is create a Container view for that software system. "include *" in this example now says include all of the
containers inside that software system and find things that are linked to those containers and again apply automatic layout. So now... if we go back here... We still have this diagram and I can double click this and we get the Container diagram. And again the diagram key is automatically generated. And we can do the same thing... so let's imagine we want to make our database look like a cylinder... the shape that
everybody thinks is part of UML but it's actually not... how do we do that? I had a set of curly braces tags database and we're going to go down here we're going to say create an element style for the database tag shape what do you think shape is? Cylinder, correct! It's not database... it's cylinder. And there we go! our diagram key now has the cylinder shape! Awesome huh? Rinse and repeat! Create more diagrams. So that's the kind of quick basic overview of the Structurizr DSL. One of the things that's really nice about this is its rendering tool agnostic. So let me copy this into my clipboard... I'm going to go here... I'm going to go to...
So I have a demo page for all of this stuff at structurizr.com/DSL... Let me get rid of that... I'm going to copy all this and paste it into this text box and click render and hopefully we'll get the same diagrams you've seen already. So this is now just on the web, a cloud version of what we saw before... Let me get rid of that because we've seen it... I said that PlantUML was one of the diagrams as code tools and in order to create multiple diagrams you need multiple PlantUML source files. This tooling generates all those things for you. So there's an export, there's a bunch of export tools that I've built, it's all in Java, it's all on GitHub and you can create a bunch of exports to, in this case, PlantUML.
And you get a nice little PlantUML version of the diagram. There's the source here and a diagram key and you get all of the diagrams defined in your workspace, which is quite nice. There's a C4-PlantUML version as well, which by default gives you the stylings Water by C4-PlantUML and there's a way to customize all of that stuff. There is a Mermaid export. So the Mermaid is another diagrams as code tool so this generates you a bunch of Mermaids source here which you can then render with the mermaid rendering engine. By the way GitHub recently added support for rendering Mermaid in GitHub Markdown Readme's so this is really nice.
if you build a tool chain together you can generate these diagrams, insert the sources in your readme's and they have these things displayed in GitHub. And just for completeness it shows you that there as well. So that's the basics... I'm going to close that down... ...because I want to show you something else... One of the things we often want to do is put our diagrams inside documentation. So I have another example workspace here... and if I open up this one again in another tab...
This is not going to take us to the diagram editor. it's actually going to take us to a bunch of browser rendered markdown documentation... So this workspace is essentially what you've seen before, it's a bunch of people in software systems and I've attached a folder full of Markdown documentation to this workspace and there's a feature here where we can embed the live version of the diagram inside AsciiDoc and Markdown documentation. So this is also
supported by Structurizr Light. It is the interactive version so you've got the diagram key and so on and so forth. Just to show you what that looks like... ... this is the folder here... So this is our workspace DSL it's the same thing, those who came to the workshop yesterday will recognize some of these things, we've got our couple of people, we've got some software systems... I'm basically saying include all of the docs in the docs folder.
The docs in the docs folder are here... and they're just pretty standard Markdown with the exception of this little tag here... which I'm using to embed a diagram from the workspace. So this is a really nice way you kind of put all this stuff in one folder and then you get a a set of rendered Markdown or AsciiDoc documentation with those views and diagrams in your workspace kind of automatically included.
You can do the same thing with Architecture Decision Records... So here we have a folder full of Architecture Decision ¤ecords which again is just bog standard Markdown as created with the ADR tools, command line tool by Nat Price, highly recommend that if you've not seen it. In the workspace here we're just saying go and attach Architecture Decision Records from that adrs folder and you also get a nice browser-based thing and this is what I showed you briefly in my talk this morning. This is the web-based, browser-based, viewer for Architecture Decision Records.
And you can scroll through them, it shows you things that are superseded, all the hyperlinks work and you get that nice little graph feature here. So this is another nice way to kind of visualize our visualize Architecture Decision Records. There's one more thing I want to show you in the remaining six minutes we have... ...so I'm going to start this up against another example... One of the questions that people ask me with C4 is well, how does this work with like big systems because all of my example diagrams have got like six boxes or ten boxes and they all look kind of nice and they all fit on a slide.
So how does this work in the real world with big systems? So let's go here, open up a new tab, go back to Structurizr Light... So imagine we have a Micro Services style architecture where we've got maybe a bunch of services and every service is comprised of an API talking to a database. So here we've got eight different services and I've color coded all the different services. This kind of works this diagram with the eight services, but if you had 80 services this diagram is going to get very complicated very, very quickly. So how do we solve that problem?
One approach is rather than showing one diagram with all services you draw a bunch of diagrams each focusing on a single service. So here is a kind of subset of that previous picture I've just shown you focused in and around Service1 only. So this is showing you things coming into Service1 and things coming out of Service1. We create the same thing for Service2. So here Service2, here's things coming into Service2 and things coming out of Service2 Can you imagine how much of a complicated mess this would be with a tool like Vizio? You'd be copy and pasting elements across tabs and trying to keep all these things up to date... Because we have a model here what we can actually do is we can basically leverage that model and essentially it's a force directed graph, sorry, it's a directed graph...
...the force directed bit comes in a minute... So here's our workspace DSL file, same thing Software System, web app, bunch of services etc etc. If I scroll down you'll see the definitions of those views. So this is the... this one here shows us all containers inside that Software System.
So that gives you all of the services. This one here is the Service1 version, so we're showing the User, we're showing Service1, we're showing things coming into Service1, with this notation, and things coming out of Service1. And then it's rinse and repeat for Service2, Service3 etc. And actually, what sits underneath the Structurizr DSL is a bunch of Java code so you could automate this process as well by writing some Java code. And there are other um other libraries and other languages that also let you do similar sorts of things.
So that's option one for dealing with scale and complexity. Option two... let me go back to here... you click this magic little button... and... Look at that! So this is a d3js force directed graph now... it's showing exactly the same data as that traditional diagram but it's interactive and it's a much more concise way to view lots and lots of data. And you can start clicking on things and it's got tool tips and there's a search feature here so if you want to find Service3 database to check no one's using it...
You know this is a really nice way to kind of explore that model. The last thing I want to show you in the remaining two minutes is... let me copy all of this into my clipboard... and I'm going to go to my Structurizr DSL page online I'm going to remove that... paste this in... hope I get the same diagram I just showed you... Perfect! This is tab at the end called ilograph. And it's a Yaml based format, a Yaml based syntax that it's generated for you with my exports tool... this is all open source...
I'm going to take this... put it into my clipboard, open up a new tab, go to app.ilograph.com. I'm going to click new diagram, so this tooling is nothing to do with me, but I think it's super cool so I supported it. I'm going to remove their example put mine in... click this and look at that! So this is another alternative version of essentially that same information. And now this is a much more kind of visual collaborative, it's a visual and interactive way to explore that same model. You're interested in Service1...
Service2... It's nice isn't it! It's a really nice way to kind of show large amounts of hierarchical data and because the C4 model by nature is hierarchical this is a really good fit for that tooling. So that is a Whistle Stop tour of the Structurizr DSL tooling. If you think this is useful and you want to try it out you can go to the GitHub repo, so "structurizer/DSL".
There's a little cookbook I've created which gives you a bunch of little recipes and tells you how to do some of the basic stuff. And if you want to try this out without installing any tooling you can go to structurizer.com/DSL and you get to type text in one side and press render and get diagrams on the other side Don't type anything super top secret into this by the way, but I mean it just ends up on the server and it gets deleted immediately, but just be careful of course. So yeah, thank you very much!
2022-12-11 07:28
