Design of AWS DynamoDB actual combat table

case

Use dynamodb to realize a simple collection function. Refer to the collection function of wechat. The function points include the following:

  1. Collect various types of data, including pictures, videos, links, files, music, chat records, voice, notes, locations, etc
  2. Cancel collection of collected data
  3. View all types of collection data and sort them in descending order of collection time
  4. View specific types of collection data and sort them in descending order of collection time
  5. Create label, delete label, edit label name
  6. Show all labels
  7. Tag management of collected data
  8. View all types of collection data under a tag and sort them in descending order of collection time
  9. View a specific type of collection data under a tag and sort it in descending order of collection time
  10. Search all favorite data

Table design

According to the above cases, three tables are required:

  1. TagDto, used to record all tags created by the user
  2. FavoriteDataDto is used to record the basic information of the data collected by the user
  3. FavoriteDataTagDto, used to record tag information of favorite data

According to aws dynamodb's best practices: As a general rule, you should maintain as few tables as possible in a DynamoDB application.

So here we only need to create a dynamodb table (development. Favorite), and the three dto structures share the same table

TagDto

First, let's look at TagDto. The basic operations of tag include creating, deleting and editing names. From the analysis of these operations, TagDto only needs to include three fields: useid, TagID and tagName

In addition, from the perspective of displaying all tag s, a sorting operation is also required. There are many ways to sort, such as:

  • Sort by tag creation time
  • Sort by tag name
  • Sort by the last access time of the tag

In order to support multiple sorting methods, we add two fields: critetime and lastaccesstime

To sum up, TagDto contains the following five fields:

  • userId
  • tagId
  • tagName
  • crateTime
  • lastAccessTime

After setting the fields, let's look at the key design. According to the requirements, TagDto has the following features:

  • A user can create multiple tag s
  • tagName variable
  • tagId is immutable
  • You need to get all tag s created by a user

To sum up, we need to create a composite key with userId as the partition key and tagId as the sorting key

In addition, three sorts need to be supported, so three local secondary indexes need to be created. The corresponding sorting keys are tagName,   crateTime, lastAccessTime.

package com.jessica.dynamodb.favorite.dto;

import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBAttribute;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBHashKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBIgnore;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBIndexRangeKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBRangeKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTable;
import com.jessica.dynamodb.constant.DynamoDBConstant;
import com.jessica.dynamodb.utils.KeyGenerator;

import lombok.Data;

@Data
@DynamoDBTable(tableName = "develop.Favorite")
public class TagDto {
	private static final String DTO_NAME = "Tag";

	@DynamoDBIgnore
	private String userId;
	@DynamoDBIgnore
	private String tagId;
	@DynamoDBAttribute
	private String tagName;
	@DynamoDBAttribute
	private long createTime;
	@DynamoDBAttribute
	private long lastAccessTime;

	@DynamoDBHashKey
	public String getPk() {
		return KeyGenerator.createHashKey(DTO_NAME, userId);
	}

	public void setPk(String hashKey) {
		String[] keys = KeyGenerator.parseHashKey(DTO_NAME, hashKey);
		this.userId = keys[0];
	}

	@DynamoDBRangeKey
	public String getSk() {
		return KeyGenerator.createRangeKey(tagId);
	}

	public void setSk(String rangeKey) {
		String[] keys = KeyGenerator.parseHashKey(DTO_NAME, rangeKey);
		this.tagId = keys[0];
	}

	@DynamoDBIndexRangeKey(localSecondaryIndexName = DynamoDBConstant.LSI_ONE_NAME)
	public String getLsiOneSk() {
		return KeyGenerator.createRangeKey(tagName);
	}

	@DynamoDBIndexRangeKey(localSecondaryIndexName = DynamoDBConstant.LSI_TWO_NAME)
	public String getLsiTwoSk() {
		return KeyGenerator.createRangeKey(String.valueOf(createTime));
	}

	@DynamoDBIndexRangeKey(localSecondaryIndexName = DynamoDBConstant.LSI_THREE_NAME)
	public String getLsiThreeSk() {
		return KeyGenerator.createRangeKey(String.valueOf(lastAccessTime));
	}
}

FavoriteDataDto table

Refer to the collection data design of wechat, including creator information, createrid, thumbnail thumbnailUrl, link contentUrl, title, data type, dataType

According to requirements, FavoriteDataDto has the following features:

  • Get all the data collected by the user and sort them in descending order according to the collection time
  • Get all collection data of a specific type and sort them in descending order according to collection time

In order to support sorting by favorite time, we add clipTime field

To sum up, FavoriteDataDto contains the following 8 fields:

  • userId
  • dataId
  • creatorId
  • title
  • thumbnailUrl
  • contentUrl
  • dataType
  • clipTime

We need to create a combination key with userId as the partition key and dataId as the sorting key, so that we can support obtaining all the data collected by the user. We only need to query with userId as the hashKey

In addition, you also need to create a Global Secondary Index with the partition key of userid & type and the sorting key of cliptime & dataid, so that you can obtain all collection data of a specific type. You only need to query the Global Secondary Index with userid & type as hashKey

package com.jessica.dynamodb.favorite.dto;

import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBAttribute;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBHashKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBIgnore;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBIndexHashKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBIndexRangeKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBRangeKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTable;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTypeConvertedEnum;
import com.jessica.dynamodb.constant.DynamoDBConstant;
import com.jessica.dynamodb.utils.KeyGenerator;

import lombok.Data;

@Data
@DynamoDBTable(tableName = "develop.Favorite")
public class FavoriteDataDto {
	private static final String DTO_NAME = "FavoriteData";

	@DynamoDBIgnore
	private String userId;
	@DynamoDBIgnore
	private String dataId;
	@DynamoDBAttribute
	private String creatorId;
	@DynamoDBAttribute
	private String title;
	@DynamoDBAttribute
	private String thumbnailUrl;
	@DynamoDBAttribute
	private String contentUrl;
	@DynamoDBTypeConvertedEnum
	@DynamoDBAttribute
	private FavoriteDataType dataType;
	@DynamoDBAttribute
	private String clipTime;

	@DynamoDBHashKey
	public String getPk() {
		return KeyGenerator.createHashKey(DTO_NAME, userId);
	}

	public void setPk(String hashKey) {
		String[] keys = KeyGenerator.parseHashKey(DTO_NAME, hashKey);
		this.userId = keys[0];
	}

	@DynamoDBRangeKey
	public String getSk() {
		return KeyGenerator.createRangeKey(dataId);
	}

	public void setSk(String rangeKey) {
		String[] keys = KeyGenerator.parseRangeKey(rangeKey);
		this.dataId = keys[0];
	}

	@DynamoDBIndexHashKey(globalSecondaryIndexName = DynamoDBConstant.GSI_ONE_NAME)
	public String getGsiOnePk() {
		return KeyGenerator.createHashKey(DTO_NAME, userId, dataType.getValue());
	}

	public void setGsiOnePk(String hashKey) {
		String[] keys = KeyGenerator.parseHashKey(DTO_NAME, hashKey);
		this.userId = keys[0];
		this.dataType = FavoriteDataType.valueOf(keys[1]);
	}

	@DynamoDBIndexRangeKey(globalSecondaryIndexName = DynamoDBConstant.GSI_ONE_NAME)
	public String getGsiOneSk() {
		return KeyGenerator.createRangeKey(clipTime, dataId);
	}

	public void setGsiOneSk(String rangeKey) {
		String[] keys = KeyGenerator.parseRangeKey(rangeKey);
		this.clipTime = keys[0];
		this.dataId = keys[1];
	}
}

package com.jessica.dynamodb.favorite.dto;

public enum FavoriteDataType {
	IMAGE("image"),
	LINK("link"),
	FILE("file"),
	POSITION("position");
	
	private String value;
	
	private FavoriteDataType(String strValue) {
		this.value = strValue;
	}
	
	public String getValue() {
		return value;
	}
}

FavoriteDataTagDto

FavoriteDataTagDto is used to record tags added by users to data, so it needs to include userid, dataid and TagID fields. According to requirements, FavoriteDataTagDto has the following features:

  • Get all tag s added on the favorite data
  • Get all the collection data with a tag added, and sort them in descending order according to the collection time
  • Get all the collection data of a specific type with a tag added, and sort them in descending order according to the collection time

According to the latter two features, the table also needs to add clipTime and dataType fields

To sum up, FavoriteDataTagDto contains the following five fields:

  • userId
  • dataId
  • tagId
  • dataType
  • clipTime

We need to create a combination key with userid & dataid as the partition key and tagId as the sorting key, so that we can support the acquisition of all tag s added to the collected data. We only need to query with userid & dataid as the hashKey

In addition, you need to create two global secondary indexes

The partition key of the first Global Secondary Index is userid & TagID, and the sorting key is cliptime & dataid, so you can get all the favorite data with a tag added. You only need to query the first Global Secondary Index with userid & TagID as hashKey

The partition key of the second Global Secondary Index is userid & TagID & type, and the sorting key is cliptime & dataid, so you can get all the collection data of a specific type with a tag added. You only need to query the second Global Secondary Index with userid & TagID & type as the hashKey

package com.jessica.dynamodb.favorite.dto;

import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBAttribute;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBHashKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBIgnore;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBIndexHashKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBIndexRangeKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBRangeKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTable;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTypeConvertedEnum;
import com.jessica.dynamodb.constant.DynamoDBConstant;
import com.jessica.dynamodb.utils.KeyGenerator;

import lombok.Data;

@Data
@DynamoDBTable(tableName = "develop.Favorite")
public class FavoriteDataTagDto {
	private static final String DTO_NAME = "FavoriteDataTag";

	@DynamoDBIgnore
	private String userId;
	@DynamoDBIgnore
	private String dataId;
	@DynamoDBIgnore
	private String tagId;
	@DynamoDBAttribute
	private String clipTime;
	@DynamoDBTypeConvertedEnum
	@DynamoDBAttribute
	private FavoriteDataType dataType;

	@DynamoDBHashKey
	public String getPk() {
		return KeyGenerator.createHashKey(DTO_NAME, userId, dataId);
	}

	public void setPk(String hashKey) {
		String[] keys = KeyGenerator.parseHashKey(DTO_NAME, hashKey);
		this.userId = keys[0];
		this.dataId = keys[1];
	}

	@DynamoDBRangeKey
	public String getSk() {
		return KeyGenerator.createRangeKey(tagId);
	}

	public void setSk(String rangeKey) {
		String[] keys = KeyGenerator.parseRangeKey(rangeKey);
		this.tagId = keys[0];
	}

	@DynamoDBIndexHashKey(globalSecondaryIndexName = DynamoDBConstant.GSI_ONE_NAME)
	public String getGsiOnePk() {
		return KeyGenerator.createHashKey(DTO_NAME, userId, tagId);
	}

	public void setGsiOnePk(String hashKey) {
		String[] keys = KeyGenerator.parseHashKey(DTO_NAME, hashKey);
		this.userId = keys[0];
		this.tagId = keys[1];
	}

	@DynamoDBIndexRangeKey(globalSecondaryIndexName = DynamoDBConstant.GSI_ONE_NAME)
	public String getGsiOneSk() {
		return KeyGenerator.createRangeKey(clipTime, dataId);
	}

	public void setGsiOneSk(String rangeKey) {
		String[] keys = KeyGenerator.parseRangeKey(rangeKey);
		this.clipTime = keys[0];
		this.dataId = keys[1];
	}

	@DynamoDBIndexHashKey(globalSecondaryIndexName = DynamoDBConstant.GSI_TWO_NAME)
	public String getGsiTwoPk() {
		return KeyGenerator.createHashKey(DTO_NAME, userId, tagId, dataType.getValue());
	}

	public void setGsiTwoPk(String hashKey) {
		String[] keys = KeyGenerator.parseHashKey(DTO_NAME, hashKey);
		this.userId = keys[0];
		this.tagId = keys[1];
		this.dataType = FavoriteDataType.valueOf(keys[2]);
	}

	@DynamoDBIndexRangeKey(globalSecondaryIndexName = DynamoDBConstant.GSI_TWO_NAME)
	public String getGsiTwoSk() {
		return KeyGenerator.createRangeKey(clipTime, dataId);
	}

	public void setGsiTwoSk(String rangeKey) {
		String[] keys = KeyGenerator.parseRangeKey(rangeKey);
		this.clipTime = keys[0];
		this.dataId = keys[1];
	}
}

Tool class

package com.jessica.dynamodb.constant;

public class DynamoDBConstant {
	private DynamoDBConstant() {

	}

	public static final String HASH_KEY = "pk";
	public static final String RANGE_KEY = "sk";
	public static final String GSI_ONE_NAME = "gsiOne";
	public static final String GSI_ONE_HASH_KEY = "gsiOnePk";
	public static final String GSI_ONE_RANGE_KEY = "gsiOneSk";
	public static final String GSI_TWO_NAME = "gsiTwo";
	public static final String GSI_TWO_HASH_KEY = "gsiTwoPk";
	public static final String GSI_TWO_RANGE_KEY = "gsiTwoSk";
	public static final String LSI_ONE_NAME = "lsiOne";
	public static final String LSI_ONE_RANGE_KEY = "lsiOneSk";
	public static final String LSI_TWO_NAME = "lsiTwo";
	public static final String LSI_TWO_RANGE_KEY = "lsiTwoSk";
	public static final String LSI_THREE_NAME = "lsiThree";
	public static final String LSI_THREE_RANGE_KEY = "lsiThreeSk";
}
package com.jessica.dynamodb.utils;

public class KeyGenerator {
	public static final String SEPARATOR = "#";

	private KeyGenerator() {
	}

	public static String createHashKey(String itemName, String... hashKeys) {
		StringBuilder hashKeyBuilder = new StringBuilder(itemName);
		for (String hashKey : hashKeys) {
			hashKeyBuilder.append(SEPARATOR).append(hashKey);
		}
		return hashKeyBuilder.toString();
	}

	public static String createRangeKey(String... rangeKeys) {
		StringBuilder rangeKeyBuilder = new StringBuilder();
		for (int i = 0; i < rangeKeys.length; i++) {
			rangeKeyBuilder.append(rangeKeys[i]);
			if (i < rangeKeys.length - 1) {
				rangeKeyBuilder.append(SEPARATOR);
			}
		}
		return rangeKeyBuilder.toString();
	}

	public static String[] parseHashKey(String itemName, String hashKey) {
		String keySubStr = hashKey.substring(itemName.length());
		return keySubStr.split(SEPARATOR);
	}

	public static String[] parseRangeKey(String rangeKey) {
		return rangeKey.split(SEPARATOR);
	}
}

Create table

According to aws dynamodb's best practices: As a general rule, you should maintain as few tables as possible in a DynamoDB application.

So here we only need to create a dynamodb table (development. Favorite), and the three dto structures share the same table

To sum up, the table needs two global secondary indexes and three   Local Secondary Index. Because three DTOs share the same table, and the partition key and sorting key are in the form of multiple field combinations, the partition key of the table is represented by pK (section key), and the sorting key is represented by sk(sort key). The Secondary Index and name, partition key and sorting key are all named in a common way, such as gsiOne,   gsiOnePk,gsiOneSk.

Create the serverless.yml file and use sls deploy -v for deployment

service: jessica-favorite-dto
                 
provider:
  name: aws
  stage: ${opt:stage, 'develop'}
  region: ${opt:region, 'ap-southeast-1'}
  stackName: ${self:provider.stage}-${self:service}

resources:
  Resources:
    FavoriteTable:
      Type: AWS::DynamoDB::Table
      Properties:
        TableName: ${self:provider.stage}.Favorite
        AttributeDefinitions:
          - AttributeName: pk
            AttributeType: S
          - AttributeName: sk
            AttributeType: S
          - AttributeName: gsiOnePk
            AttributeType: S
          - AttributeName: gsiOneSk
            AttributeType: S
          - AttributeName: gsiTwoPk
            AttributeType: S
          - AttributeName: gsiTwoSk
            AttributeType: S
          - AttributeName: lsiOneSk
            AttributeType: S
          - AttributeName: lsiTwoSk
            AttributeType: S
          - AttributeName: lsiThreeSk
            AttributeType: S
        KeySchema:
          - AttributeName: pk
            KeyType: HASH
          - AttributeName: sk
            KeyType: RANGE
        ProvisionedThroughput:
          ReadCapacityUnits: 1
          WriteCapacityUnits: 1
        BillingMode: PROVISIONED
        LocalSecondaryIndexes:
          - IndexName: lsiOne
            KeySchema:
              - AttributeName: pk
                KeyType: HASH
              - AttributeName: lsiOneSk
                KeyType: RANGE
            Projection:
              ProjectionType: ALL
          - IndexName: lsiTwo
            KeySchema:
              - AttributeName: pk
                KeyType: HASH
              - AttributeName: lsiTwoSk
                KeyType: RANGE
            Projection:
              ProjectionType: ALL
          - IndexName: lsiThree
            KeySchema:
              - AttributeName: pk
                KeyType: HASH
              - AttributeName: lsiThreeSk
                KeyType: RANGE
            Projection:
              ProjectionType: ALL
        GlobalSecondaryIndexes:
          - IndexName: gsiOne
            KeySchema:
              - AttributeName: gsiOnePk
                KeyType: HASH
              - AttributeName: gsiOneSk
                KeyType: RANGE
            Projection:
              ProjectionType: ALL
            ProvisionedThroughput:
              ReadCapacityUnits: 1
              WriteCapacityUnits: 1
          - IndexName: gsiTwo
            KeySchema:
              - AttributeName: gsiTwoPk
                KeyType: HASH
              - AttributeName: gsiTwoSk
                KeyType: RANGE
            Projection:
              ProjectionType: ALL
            ProvisionedThroughput:
              ReadCapacityUnits: 1
              WriteCapacityUnits: 1
        TimeToLiveSpecification: 
          AttributeName: ttl
          Enabled: true
        PointInTimeRecoverySpecification:
          PointInTimeRecoveryEnabled: false
        SSESpecification:
          SSEEnabled: false
        ContributorInsightsSpecification:
          Enabled: false
        Tags:
          - Key: product
            Value: ${self:service}

Complete code

GitHub - JessicaWin/dynamodb-in-action: introduce usage of dynamodb

Added by Tokunbo on Sun, 10 Oct 2021 05:45:28 +0300

Popular Keywords

©2025 Programming VIP