Paweł Głowacki - Integrating with Amazon Dynamo DB w/ Delphi CDATA Connector - CodeRage XII

Welcome. To integrating. With Amazon, DynamoDB, using. Delfy see data enterprise, connector, webinar, my. Name is pavel kovarsky, I'm working for Embarcadero, a sales consultant and. Today, we are going to focus. On. Connecting. Your. Delphi and C++. Builder applications, using the, newly added. Enterprise. Connectors. To. One, of the available, services, and specifically, Amazon, DynamoDB. So. Amazon DynamoDB, is. One of the leading no. Sequel, databases. And. It's available. To. Integrate, through, enterprise. Connectors, so, why. Dynamo. DB. And why I have chosen this. Special. Service as, an example so. Dynamo. DB is, a no. Sequel, cloud. Service, Amazon from, available. From Amazon, Web Services, you can, use it either as a standalone or. Also, use. It as a, local, database. Service. For. So you can download. A local version and for testing, and, development, what, makes it really, easy. Interesting. Really, interesting is, that it is very fast so this is a specifically. Something. That is. Characteristics. Of no sequel databases. Because. We trade some, of the, flexibility. And some of the asset. Functionality. Atomicity. And. The. Typical, sequel. Characteristics. But, to get. A super, speed so you, can have a, single. Millisecond. Latency and. With operations, at any scale so you can specify when. You provision. And the. Tables, in dynamo dB, what. Is the throughput. That you, want to use, so you can get. To. The. Single, and milliseconds, latency. And so, that's very interesting, let's. Have a look on that website. Of. Enterprise. Connectors, to, learn more about, these. Connectors, and in, this presentation, I'm going to focus. On just, DynamoDB, a connector. If, you navigate to wwm. Arcade or comm, slash product slash enterprise. Connectors, and, you can learn more about these. New Enterprise. Connectors, that have, been recently made. Available by, Embarcadero. With the partnership, with. C, data which. Is a company, that specialized. In the. Building. And architecting, very, performant. Drivers. To different. Systems. One of the offerings. One of the types of the drivers that C. Data offers now, are. Fired, up drivers, so, what, is special about these, new enterprise connectors, is. The fact that different. Services. And that, you might. Want to integrate, typically. Provide. Different, API, so you don't need to. Learn. Different, api's, of different, services. What. This. Drivers. Provides is a sequel. 92. Interface. So, you can use a familiar, sequel. Statements, like select, statements, insert insert statements, and. So on and to. Integrate. With, all kinds, of services so. This. Is also, available as past, implemented. Technically, as fire. Duct drivers, so you can use a standard folder, components, like connection, components, like, query, components. Like stored. Procedures, like metadata. Information. Components. To. Work. With, all kinds of services so one, of the examples of the services, that are, available through. This. New. Connectors. Is. Dynamodb. You, can see the whole list of different. Applications. Different systems, that you can integrate with, using. Those. Enterprise connectors, the. Offering is in. Two different variants, and there is an enterprise, and. Enterprise. Enterprise. Plus that. Contains, more.

Systems. The, dynamodb, is available. In both of both of these offerings, so. You will it's, enough, to have just enterprise. Subscription. So how it works is that you. Can download. The. Trial. Version of, this, dynamic, DB drivers. From inside of Delfy or C++ builder using a gated, package manager, but, if you want to use. A retail, version then you can contact, Embarcadero. Or one of its partners and, purchase, separately, a serial. Number or license, to. Use those. Connectors, so you get just one serial, number and, you have access to all the connectors, for. The duration. Of the year so you can build. Applications. You. Can deploy them and you have access to all the new drivers. That are being added but, after this, one year your application, will still work. So even. Though you will not longer have, this. Maintenance. Of our subscription, which we, yeah. It's a good idea to have it perpetually. But, your applications. That you have built will continue, to work even if, the subscription, is over so. This. Is a very. Interesting and, very powerful. And, productive. Way of integrating, to sometimes, maybe very. Exotic. Services, so, one of the services that we want. To work. With today is. Amazon. DynamoDB. And so. It is a no simple database, so. It's very different from a traditional databases. Sequel, databases. Because it's kemal schema-less, so. When you define, tables, you don't need to define. Upfront the, names. And, the types of all the columns before you can work. With it the only thing that you need to specify is the primary key, and. Then you can different. Items. That corresponds. To records, in a database, can, have a different, attribute that corresponds. To column. So you, can have a different, items. But with completely different attributes. The, only thing that they need to have in common is the same type. Of a primary. Key also. This. Really is one of the fastest. Fastest. Databases, on. The market, so. You don't, specify. A size of a database but our, database. Table but you rather specify. A throughput capacity and, the, more. The. More capacity, you. Have the, more you need to pay so it's on a pay. What. You use, basis. Also, something, that was relatively. Recently, added, is them Dynamis, the Amazon, DynamoDB accelerator. Which. Makes. It possible to get to, go from milliseconds, to microseconds. Response. Time so this could be very. Interesting, in, situations, like when you have a high, end real-time, and, multiplayer games, that you really need to have. This very few milliseconds to, generate, a next, frame and but still you need to contact, with. With. The system, and to see the, state of the game and. See. What you need to display. So, there are many different. Examples and this is very interesting to, to. See that, this this. Is used. So there. Are different ways. Of integrating with, Amazon. DynamoDB. So. You can use language. SDKs. So. Under, the hood those language, as the case and also, the, connector, is. Also, fit in this category is. Using the REST API so. There is an arrest, interface. That is that. You can that. That you can learn. About and. There. Is also an, interactive. A console. So. You can. Do. Everything the, management, of the dynamic, DB using Co web console but, there is also a command-line, tools that, also are, very handy to, integrate, to work to, manage the. Different Amazon. Web. Services. So. What. Are the first steps to. Get. Our Delphi applications, to work with, DynamoDB, and, so first of all you need to have had your. Own. Dynamodb. Account. So maybe, I just for. A moment will, get, out of this and just show you, how. You can set. Up the. Dynamo. DB, account so. There is something called F Amazon. Free tier so. You can create a free. Account you. Still need to have a valid credit card but. For, a certain services, that are part of the free tier you. Will not be charged so you can play and. See. If this is something, for, you so, one of them services. That are also available. As. A part. Of the free tier is. The dynamo. DB, so you get a 25. And gigabytes, of storage and, also. Some. Of the. Write, and treat. Capacity. So. You can start, playing, with your, applications. Just. Without. Paying. Anything upfront so. That's, that's, a first, prerequisites. You prerequisite. You need to have an Amazon. Account. Once, you have you your, account you, can use an Amazon. Management, console, to, login into them to. The to. The console, and then you can just start playing with with, dynamodb, so, this. Is a list of different web. Services available from, Amazon. One of it is dynamic.

DB, There are also other interesting services, like for example ec2. For virtual, servers, in the cloud or lumber, for. Creating, serverless, applications. Here, we are going to focus on a dynamo, dB so, click, on dynamo DB and, here. We have a, certain. Dashboard. And you. Can see I have been already playing with it so, I have a number of different tables. Created. For, it so. Right. Now what I'm going to do is. To first. Create a table and then. I'm going to populate, this, table with some. Sample. Data and then later on we, are going to access this data from. Inside of the Delphi applications. And the rad studio, ID, so. The first thing is to create a table so I'm going to use, a sample. Data and this. Is part. Of the documentation. Here. Available. Online is. A some. Sample, data files. And some steps how to create, a table and how, to populate it with data, so. Here we have a sample. Two. Samples, two use cases product. Catalog and forum application, so, we are going to use just this first one a product. Catalog so, when we create a table, product. Catalog we only need to specify the primary, key and, because it's schema-less. So. Here. We, have a whole, song the. Location. Of some. Test, data so, we can download sample, data and it will just, download certain. JSON, file that we can use for. Loading data so first let's. Start with creating a table so, I'm going to create a product catalog. Table, I have already created. Two. Of them so I'm going to specify, the third one so, a, product. Catalog. And that. Would be product catalog three and the. Partition key is ID, and. The. Type. Is number. Also. Here, we have a possibility, to add a short key so. That's another way of creating. Databases. There. Are certain, resources. Available, in the internet, that explains, best. Practices, of architecting. Performant. No C no sequel DynamoDB, solutions, basically. The. Data in, dynamo, DB is stored, in s SSD, drives, and. This, partition, key is. Used, by. The. Internals. Of the dynamic, DB engine to, calculate, a hash, value. That. Then it's basically. The. Sides of that physical. Location, of. Individual. Data. So, it's, a good idea to have to select, as a partition. Key and some. Value that is properly. Distributed. So you, will get a better performance. So. But let's probably, beyond the scope of this, this. Presentation but, that's something, that is is. Good to learn if you want to really do. It for real so I can create a product catalog right, away and do. I create, okay. So I, can. See that my. Table. Is being created, so in, order to load data. A, test, data here, I can just click on a load data. Into the table and here.

We Have. One. Of the easiest way, is. To actually use a command, line tools. So. This is a set. Of them you. Had the, command. Line tools for Amazon, that. You can use. For Windows Mac, or Linux here, we have a I can, just download, them. 64-bit. Version. Of this, of these tools so that. Would download. For me a Installer. You, can see I have a number of those installers, of already. Download. That it's a very small download, you. Can just. Open. It in the folder and. And. Run it so let me show. You this just. Click on open. This. Should run. The standard installer, and this, is a useful tool for, managing. Different. Amazon. Services so something that we are also going to use to. Load sample. Data so I can click on next accept. The agreement and it, will install, it in the Program, Files amazon. Amazon CLE, i'm, not going to go, further because i have already installed. This. Command-line. Interface. Okay, so the next step really. Is to, generate. Special. Security. Credentials. So. In. Order to use the command line interface, or. Interact. With Amazon. Web Services through. REST API s we, need to have a certain keys. The two strings. One is a key one is a secret key that needs. To be embedded, in. Every, HTTP, request we send to, DynamoDB. So, there. Is a special service, here, if. We go to the, services. Already. Locked. In here I have my account, and. I can go to the my. Account, and. Management. Console, and here. In, the, category. Of security. Identity, and compliance, there, is an identity, access, management. Service. That I can use to. Create. A user, and, as. A side effect I'm going to create a credentials. That I will need to communicate. With Amazon. So I can hear at a user, and. This could be any like, for example, Delfy. Tanna dynamo. Dynamo. DB, user. And it's, important, to make sure that I check the access type, to programmatic, access because, I want to generate, access. Key and secret access key. And, yeah. It's it's an, that. The, security, is a very important, part of any cloud services, so. The model here is that we. Communicate. With. With. The service on behalf of a certain user, so. The. Certain user will. Have a, yes. Access, to a certain services, so here, we have a Delphi, DynamoDB, user so we only need this user to. Be able to. Access. Dynamic. DB and not any other. Service. So here we, can specify the to attach existing, policies, directly, to the user the, other option, is to put, the user to the group so we have more. Functionality. And, here, I need to specify which. Policy. I wants to add to it so here, we have Amazon, DynamoDB full. Access so I need to check this policy. So. I can, now. Use this policy, to yeah. To grant this, user that I'm creating full.

Access To dynamic, DB on the next page of the, wizard I have, a review so, that would be my user, and we'll. Have, all access. To dynamo DB and, I can create a user so. In a moment i should see a. Screen. That will show me the access, key and the, secret secret. Access key so. This is the only moment that I'm able to actually obtain. The. Password, or secret access key for this user and this, is intentionally, you can create as many. Users. As you want and. It's a good practice to rotate, them, and if. I don't I. Can. Easily. Show you this disk. This access, key because at any moment I can disable, this this, user so that's that's how it is. And what is important, I need to get this access, key and the secret access key I can. Download them as a. CSV, file or I can just copy, these. Two strings, to. Notepad because, I will later. Use it so. Yeah. Once we got, this and we can go to them. To. The next step and let's. See what is our next step we have just created and. The. User and. We can now jump. Into the process of loading sample, data so. Here. We. Have a topic. Also. Here load data, and, we. Can use. AWS. Command. Line to. Request. Items so after, I have extracted, the sample data there. Is an product, catalog JSON file as part of it and now I need to execute. This statement from command line so, my command. Line, is. Already. Running. And you can see this, is the, operation. That I need to execute. To to, load data in, fact I have already added this, data to them to. These tables are not going to run it really. After installing, the first thing you need to do is, to do the. AWS. Configure. And then. You can specify the access, key you, can specify the secret, key you, can specify the default Reagan. Is also, very important, where, we are going to work so. In the Amazon, console, in. My account, so here in the, account. If I go back to. The dynamo, DB I can. See that in. The top right corner I have see I can, see that all the different regulo, it's important, to, make sure that I'm creating. The, these. Tables, in a specific. Regulan. See the different tables are. In. Different, Regan's maybe it's not viewed. Here but I know, it where or in Ireland, so it's important, and, when I'm working with this, to. Specify the proper. Proper. Regulan. Also, default output format, JSON so I don't change anything so I just show you this but, here I can just and, go and execute. This, this. Statement, to. Load the. Into the table. So, once I'm. Done with it I can. Go again to. My console, and I can preview and, then different. Catalog. Items, and this, is how it looks like this. Is very interesting because you can see that different, different. Records are in no sequel, different items. They have a different. Different. Attributes. Also. There. Is some bicycles. With different, attributes. Like bicycle, type color. And there, are different books that have, a different attributes, like outer dimensions. Or ISBN, number, so, this is typical, for non. Sequel databases, the. Only required. Attribute, is the ID and that unique and that. It. Identifies. And a given record. Okay, so we have already. Gone. Through the first step. So now we have. We. Have now data. We, have created, an account. We. Have created, the, table we have populated, with data we, have credentials. Now we can go. To Delfi. Okay. We, are now inside, of rad studio and. In. Order to install. Them the. Driver, that. The. Fire duct driver or enterprise, connector, you need to go, to the C data, website. And. Then download and, to, register, appropriate. Setup. Program. Ok. So here, we have. We. See the C data website. With, dynamic. Dynamo, DB fire. Duct driver, there are other, drivers. Available. Here so we need to navigate, to the driver and, that we want to use so you can see that I see, data also offers other. Types, of, drivers. Not necessarily, a fire, duck but very specialized, with communicating. With also. Using other, technologies. So, here we have an Amazon, DynamoDB fire. Duck driver so, you need to separately. Poor choice and, that, serial. Key and. Then you need to enter it here product, key and your email address so during the process you will download the setup dot excel file, when. You run it you will be also asked to register online for. The first first. Time when you install the first. Connector. All, subsequent, connectors. You don't need to register because you are already registered and. After. You have installed successfully, this. Driver you should see.

This, Driver installed, inside of that rad studio if. You just want to give. Those drivers, that try you can use a gated, package manager, so. We have here. The. Connectors, category, and inside. Of the connectors, category, you can just quickly. Go to that dynamo. DB, so this is this driver we, can I could. Have installed, at trial but instead. Of using the trial and you, can just click on it and. Accept. Them EULA, and. Then. I'm not going to to. Continue because I have already installed. The. Commercial. Driver so, here if everything. Is okay after the installation you should find in, this in, the file in the data explorer the new, C data dynamic, DB datasource and the. First thing that you want to do is to create, a new, connection. So you right-click on this one select. Add new connection. Here you can specify the, name of the connection, I have. Already done it so, I have one connection specified. Dynamo. DB 1 and, we. Need to configure this connection, so right, click on the connection. Select modify. And here, we can enter, our access. Key and a, secret key and also, the. Regulan when, we want to when, we keep our data. Also. There, are many other, parameters. Here that, controls. How the drivers. Work so. There is a documentation. That comes with the, installation. Of the driver also, this documentation. Is available on. Where. You can actually see, how. Different, parameters. Here, controls. The operations, of the viaduct driver the. First thing is to click on test to, verify the connection, is established. Successfully. And we. Are good to go so. If we open now them this dynamic, DB, node. Inside. Of the data Explorer we can see all, the different tables. That are here, represented, as a standard sequel table so for, example my product, catalog will, have a primary. Key and different, attributes. So. We can preview. This data directly, from here so. This is exactly the same data that, we have seen, on. Using, the web. Console, but, it's also interesting here too to notice there are two standard. Stored. Procedures one, for creating, table and table, and one for creating, the schema and that you can use. Like a standard, fire duct stored, procedures, to. Either a create, a table so, we don't need to do it manually we can do it in. Code or create a schema, the, schema is also very important, because. How. No. Sequel, databases. Work is that there is no schema but, for, for. Fire, duct and. The sequel. Concepts. You need to have a schema, so, there are two ways of operation, so maybe. It's a good idea now, to show you how. The. Online. Documentation. Looks like. So. Here if I go back to my. Fire. Duct components. Here, we have had different, interesting. Things to. Learn, so, this is a locally. Installed. You. Can see local, help. File and, so you can see. Certain things like think, that we have already done creating.

Account. Creating. Credentials. Writing. Some basic, access, code, we have not done it yet but. Also, very interesting information about, how this. Works so here, we have a two ways. How, the driver work works there is an automatic, schema discovery, and custom. Custom, schema definition so, basically. The. Driver itself. And. This is controlled, by row, scan, depth so, by default it will look, into the first 50. Records. Or items, in the table and they will try to and. It, will create a, automatic. Schema, that. Represents. That the items. That it has found. There there, are also two, additional, parameters. On the connection, flatten arise and flatten objects, so. This is how. You can basically. Specify. What. Should happen with, attributes. That contains. Contain. JSON, themselves, so here for example we have a court. Attribute. That, is an array, of two double, value, so to make our life, easier, to work with code with this data we can set. The flatten our I stood through and then, this, field, will be translated. Into the two float. Values. There. Is also. Interesting. Concept. Of the JSON, functions, so, these are the, extensions. To the sequel language that the driver, knows. About and you can use them to. Easily. Extract. Different. Elements, of JSON from, inside of the. Attributes. That are part of the table so, let's maybe start from. And just creating, a simple, table. Okay. I'm going to go back here so this is my table in a data Explorer, I'm, going to create. A new. New. Visual, forms application, in this case and. I'm. Going, to just take this product, catalog drop. It onto them form, a DAT, T, datasource. Component. Connected. To the query, component, so here we have a query and this. We can preview. The just. A standard select, star from product. Catalog we can. Executed, from here to see that it, works. As expected and. Like we can very quickly build, a simple, simple. Simple. Application. To. Work, with data so, it's not. A rocket science. Something. That we've been doing since, Delphi one I'm, going to add a checkbox here, to. Make the. This. Product. Catalog. Table. Or query active, so here, I'm can change this. To active. Ok. Like. This and. If. Then. My product. Catalog. Table. Is active, only, if the my checkbox. Is, checked. So. In this way I will be able to, open. And close down, the table I need. To add two, more. Two. More elements, so, I'm going to add a TDB, navigator. So, I can put it on the on, the, panel, and. I, can add a TDB grid, well-known. Component. I can. Use, quick. Edits, to align. It to the client, and. I, can use shift, key. To select both, components, and to. Connect them in, one operation to the data source so. In this way I can. Now save it you need to project. Two great, names. So. I can run my application and, I. Can verify that I can, see, data. From, from. The underlying. Table, so, here, we see that we. Can ask for example change, the outer free, to outer, for. We, can post, this change and if. We just come, communicate. Again we can see that we have readwrite. Access, to. The underlying data. So. This, is very. Powerful, but. It's obviously, it's just just, a beginning and, we can use a different neck connection, properties. Parameters. To. Work with different aspects, of a. Database, so for example we. Can play, a little bit with this scanning. Option so we see that there, are three. There. Are five, items. That. Are about with information. About bicycles. And three items with, information. About books. So, by, default if you open, these the, parameters, you can also see it from. Here from, the object inspector there are many different. Operations. So there is this. These. Options, to flatten, our eyes and flatten objects, but. Also we have this option. To to, specify, the this. Depth. Ropes. Can depth so instead of 50 we let's make it free. And let's. See what will happen if. I now run. The application, again and open. The table. I should. See that I only.

The. Fact is that I have eight records, but. I have only the. The. The. Certain. Certain. Columns. That are only applicable to bicycles, so you don't see the. Book size, or, book, ISBN. Number, so, it has used just the first three records. To to, infer, the. The schema, so, that's a many. Other options. Here also. What is interesting, option. Is the support, for caching, and, so this component. Can work also. In, them in the offline mode so. There is a special, offline. Property. That I can set to true and there are some cash. And, meta data and cache data, I can, specify the database so that would be a local file with, a cached, data, so. This is all very nicely. Described. In. The documentation, of this, of. This driver, let. Me jump back here. Again to my to. My slides so. Yeah. I have already, done. This and. Wanted. To show you also this JSON. Functions, how they work so I have another. Sample. Sample. Sample. Project, here and. That will be not. Project, but it is I. Need. To open it from the. My. Existing, projects, it's called students. And. So basically what I have done I created. Manually. This. This. Table, that was, described. Here so it's a he. Said it's a table with. Different. Student information so, I have just. Used a web, console, to. Add. Create. A students table with a primary. Key called name, and the. Great. Attribute. That contains, this this. RI of. Of. Scores, and I can use different. Extensions. To them to. Sequel language to to, use this, extract. A certain, element. From, the array I can, calculate how many elements are in the array I can, find the total. Of the, elements. A minimum, or maximum, value. So this, is very straightforward. This. Demo basically. Again is just the. Same connection, but this time we have a query. Component, that connects. To the students, table and. We can execute, it so we have just a Smith and, thus. The attribute, is just a JSON. Array. And. Here if. I call, execute, I'm, just clearing. The sequel, and taking the the, currently, selected sequel. From the combo in. The combo box I, have. A different. Different. Queries. That. I have copied from the. From. The documentation, so now I can just run. This this. This. Demo and see how. That would work so, I can execute it and it. Should work okay so I have if. I want to for. Example have a number of grades, execute, and now, I have five. Grades and so on so that's that's, interesting. Extension, to. The language one. More demo that I wanted to show you is them working. With. With. Some sample data, so. As part of the online. Documentation. We. Have. Different. Different. Test. Data and, so, here we have also the. Movie data so if you want. To work. With different, SDKs. So. There are different. Examples. In. How. To work with, movies. Data and, how, to work, with dotnet. Java, JavaScript, and. Node.js and, other, SDKs. That are available all, those as, the case are, using. The same, movie. Data and. So what I have done I have extracted. A small. Chunk of this movie data because, it's quite big and the, objective, is to, insert.

This Data. Inside. Of the table and, so let. Me quickly run this movie. Sample, and, so I have written, some code to to. Read data that. JSON data movie, sample. Data from, a file and this, is how it looks. Just. Converting. To the string and here, is some processing. So let me very quickly show. Show, you what what's, here and so, there is a special. Special. Unit. Here that models. An individual. Entry. Individual, item, in the movies database, there, is a movie. Rex. So, this is a, general. Generic, list and also, this has a class. Function, that, is a kind of a constructor. Like that. Takes a plain. JSON, and. Returns. Generic. Lists of Rex and this records. Can be done programmatically. Inserted. Into the table. So. You. Know let me show you, how. This works so first, of all you. Can. Use the. Query. Components, but this time, instead. Of. Using. Dynamic. Connection. Component, query component, on. The form is a better practice, to put it on a data module, so, my data module, has Sam now insert. Movie, and insert, movies, so this is two. Possibilities, how you can enter. Data into. The database, so you can use just a standard syntax, when you, yeah. I have a parameterised. Query you, can see how it looks like it. Has a just insert, movies, and a standard, sequel, syntax. For an insert statement and, in, parameters, I need to specify the, types. Of this parameter, so now I can just programmatically. Assign. To these parameters. And execute, query to. Insert a new item but, it's also possible. And more efficient, to use. So. Called our idml. Syntax. And, so you can use an array, size property so instead. Of executing. Insert insert, statements, one by one you. Can put. Them in a batch so. You've specified how. Many parameters should have and this is a two-dimensional array, of parameters, where you specify not. As integer, but as integers. And then you specify which. Element, and after. You have assigned, the parameters, to, all the records then you can execute them in one operation so, under. The hood these and statements. Translate, into them this REST API. But-but. Insert. Items that. We've been using to populate, the. Product, catalog table so, these, are some of the samples, it's a quite an beak and as I just in. This presentation I just wanted to to. Give you a feeling. What. It is to. To. Work with. Dynamic, DB, in Delfy so. It's a it's a very interesting technology, and with. With. Them new. Enterprise. Connectors. You. Can use a familiar sequel. 92, syntax. You. Can use fire. Duck framework. And familiar, components, like connections, queries, and integrate. With all kinds of the systems, that are, not necessarily.

Databases. Or maybe completely, different systems, like for example Google Calendar, or. Salesforce. Or Twitter, and then, you can look. At these systems, this api's, as if, they were a traditional sequel. Relational. And database. Ok, so that's. It for today thank, you very much.

2017-12-09 19:50

Other news