I recently launched a Slack app to help with channel bloat! Simply installing it would help me out a bunch. I have 2/10 installations required to submit my app to the Slack Marketplace. Thanks for helping me reach that goal.
In my Guide To Building With Serverless AWS chapter about DynamoDB, I mention using the Document Client to communicate with DynamoDB instead of some other alternatives like the lower-level client. In this post I would like to discuss and give code examples for some of the specific DynamoDB APIs. When I started using the Document Client I had a little bit of trouble translating the lower level client and APIs calls into Document Client calls but after seeing some examples and reading how I translate the payloads, I hope that all of that will be much easier for you. As a side note, what I will be showing in this post uses the AWS Javascript SDK v3. While the lower versions (and potentially future versions) will most likely look similar, this code is specifically for version 3.
Transforming a payload to include data types for DynamoDB calls can be tedious. For the sake of my own understanding, I have used the lower level SDK client to manually code the DynamoDB types, and having done that, I much prefer the Document Client for ease of use. The Document Client uses native Javascript types to conclude the data type for DynamoDB and then transforms the payload for us.
The Document Client in the SDK is a thin layer on top of the lower-level/normal client that does the transformation. It is available in the @aws-sdk/lib-dynamodb package but the @aws-sdk/client-dynamodb is still required.
Before I start showing the code examples I want to make note of a couple of assumptions that the code will be making.
First is that the DynamoDB table is created and the name of the table is available through an environment variable called TABLE_NAME. In the code examples, you will see TableName: process.env.TABLE_NAME.
The code also assumes that the table is configured to use a partition key with the name id and a sort key with the name secondaryId. This will be seen in examples looking like the following.
Item: {
id: 'myId',
secondaryId: 'mySecondaryId',
},
Both the partition and sort key names can be configured to whatever you would like. The id and secondaryId keys in the Item object would then need to be changed accordingly. I suggest not hardcoding either of these values like I am in the examples. For the partition key (id), I suggest using something like a generated UUID or a value unique to that item’s data. The sort key (secondaryId) value will be more dependent on your data model, but some suggestions could be using a configured value pulled from a module, an environment variable, or a value specific to that item’s data.
If you do not know much about data modeling, I highly suggest learning more and determining how to model your data before using DynamoDB. I lightly discuss modeling in this post but there are other great resources out there from Rick Houlihan (find his re:Invent talks on YouTube) and Alex DeBrie.
Remember that these are simply examples and you can change your code however you would like. I simply want to show what the calls could look like.
Table of Contents:
Creating the Document Client first involves creating a normal client and then initializing the Document Client with the normal client. I like to use a single module and then export the client for use in subsequent modules.
const { DynamoDBClient } = require('@aws-sdk/client-dynamodb');
const { DynamoDBDocument } = require('@aws-sdk/lib-dynamodb');
const client = new DynamoDBClient({
region: process.env.AWS_REGION,
});
const documentClient = DynamoDBDocument.from(client);
module.exports = {
documentClient,
};
PutItemCreating a new item in DynamoDB involves either a PutItem or BatchWriteItem API call. An Item is passed in and the partition and sort key need to be present. Any additional values in the Item will be written as attributes.
const myItem = {
hello: 'world',
value: 3,
};
await documentClient.put({
TableName: process.env.TABLE_NAME,
Item: {
id: 'myUniqueId',
secondaryId: 'secondaryId',
...myItem,
},
});
BatchWriteItemThis API was slightly more confusing to me. The top level of the payload contains a RequestItems key. The values for RequestItems are key-value pairs where the keys are table names and the values are arrays of requests being sent to each table. The requests in the arrays are also key-value pairs where the keys are either PutRequest or DeleteRequest and the values are either Item (similar to PutItem) or Key (similar to DeleteItem), respectively.
const myItems = [
{
hello: 'world',
value: 3,
},
{
hello: 'mars',
value: 5,
},
{
hello: 'saturn',
value: 0,
rings: true,
},
];
const requests = [];
myItems.forEach((myItem) => {
requests.push({
PutRequest: {
Item: {
// Keep in mind that each partition and sort key will need
// to be unique for each item
id: 'myUniqueId',
secondaryId: 'secondaryId',
...myItem,
},
},
// Similar payload for DeleteRequest
// DeleteRequest: {
// Key: {
// id: 'myUniqueId',
// secondaryId: 'secondaryId',
// },
// },
});
});
const batchWritePayload = {
RequestItems: {
[process.env.TABLE_NAME]: requests,
},
};
await documentClient.batchWrite(batchWritePayload);
GetItemRetrieving an item is fairly simple. We pass in the partition and sort key and get back a response containing some metadata and the Item.
const res = await documentClient.get({
TableName: process.env.TABLE_NAME,
Key: {
id: 'myUniqueId',
secondaryId: 'secondaryId',
},
});
const item = res.Item;
Edit: In one of my recent livestreams I came across a problem where GetItem was returning an empty object instead of a string set. It took me a while to dig down the rabbit hole but I finally figured out what was going on. The Document Client correctly unmarshalls the string set into a Javascript Set type, but whenever a Set is logged (or returned the API Gateway for that matter) it is represented as an empty object. The problem was not that the data returned was an empty object, just that it was represented as one. The solution is to convert the Set to an Array (using something like Array.from(item.stringSetAttribute)) before logging or returning the value. Shout out to this GitHub issue and comment that brought me to my understanding.
QueryThere are some additional fields in query that might seem confusing at first. KeyConditionExpression for a query needs to define the value of the partition key and can optionally specify a comparison with a value for the sort key. In my example, the sort key is simply compared with equals (=) to the value (secondaryId) but I could have just as easily used any of the supported key condition expressions.
The next confusing parts are the ExpressionAttributeNames and ExpressionAttributeValues. Each is a way to dynamically reference a value in the KeyConditionExpression without needing to do string substitution. With the Document Client, this is straightforward, but for some context, without the Document Client, we would have needed to add data types to the values in ExpressionAttributeValues. The strings to be referenced by ExpressionAttributeNames should be prefixed with a hash sign (#) and ExpressionAttributeValues should be prefixed with a colon (:). The prefix characters are not a convention, they are explicitly stated in the documentation.
After results have been gathered from the table, we can further refine the results by making DynamoDB filter what has been queried using the FilterExpression. This is a good place to go over results with a fine-toothed comb, but just know that a query consumes read requests units based on the results returned by the KeyConditionExpression, not only the results returned after further refinement using the FilterExpression. Again, this is a data modeling problem. The FilterExpression uses the same syntax as the KeyConditionExpression but can be written for any attribute in the Item.
Lastly, the ProjectionExpression is a comma-separated list of attributes that should be retrieved by DynamoDB and returned as a result of the query. If we were querying the items from the BatchWriteItem(#batchwriteitem) example, we would on retrieve hello and value but not rings.
We get back a response containing some metadata and the Items we match the KeyConditionExpression and FilterExpression.
const res = await documentClient.query({
TableName: process.env.TABLE_NAME,
KeyConditionExpression: 'id = :partitionKey AND secondaryId = :sortKey',
ExpressionAttributeNames: {
'#valueName': 'value',
},
ExpressionAttributeValues: {
':partitionKey': 'myUniqueId',
':sortKey': 'secondaryId',
':minValue': 3,
},
FilterExpression: '#valueName >= :minValue',
ProjectionExpression: 'hello, value',
});
const items = res.Items;
UpdateItemUpdating an item is fairly simple but involves a couple extra parameters. We pass in the partition and sort key as the Key’s value but any updates need to be conveyed using an UpdateExpression. UpdateExpressions can update any data type but they use different syntax depending on the type, but I will not cover that here because that is one area that AWS documents well. In addition to an UpdateExpression, we can also choose to dynamically pass in attribute keys and values using the ExpressionAttributeNames and ExpressionAttributeValues parameters, respectively. This gives a much cleaner feel as opposed to building a string with the keys and values directly inserted into the UpdateExpression.
await documentClient.update({
TableName: process.env.TABLE_NAME,
Key: {
id: 'myUniqueId',
secondaryId: 'secondaryId',
},
UpdateExpression: 'SET #helloAttr :planet',
ExpressionAttributeNames: {
'#helloAttr': 'hello',
},
ExpressionAttributeValues: {
':planet': 'jupiter',
},
});
DeleteItemDeleting an item in DynamoDB involves either a DeleteItem or BatchWriteItem API call. Deleting is as simple as retrieving, we pass in the item’s partition and sort key and DynamoDB handles the rest.
await documentClient.delete({
TableName: process.env.TABLE_NAME,
Key: {
id: 'myUniqueId',
secondaryId: 'secondaryId',
},
});