Getting started with Tablestore data management

In order to let users of table store better understand the product data management ability and SDK use, this article will describe the function and use mode of data management in combination with the code. This article only talks about the management and use of the main table data, not all the data management capabilities of the Tablestore. The content related to the query ability of secondary index and multiple index will be introduced separately in the future articles. For example code link address, see: Tablestore examples project.

Data management overview

The data of the Tablestore is identified by the PrimaryKey as the unique row. The management of the table data is also based on the PrimaryKey to add, delete, modify and query. For the main table data management of the Tablestore, it can be divided into two types according to the operation quantity: single line operation and batch operation. According to the type of operation, it can be divided into two categories: write and read. RowChange interface is implemented for adding, deleting and modifying single row data. Therefore, in batch operation, you can request to add, delete and modify data at the same time.

Data management use

Single row operation

Data insertion

For data row level insertion, you need to provide a complete PrimaryKey to identify the unique ID of the row, and then configure the property value of the row.
matters needing attention:

  • The primary key must be complete and consistent with the primary key name and type of the table attribute. If not, there will be parameter errors;
  • If the row data exists, the original data will be overwritten by default. If you don't want to overwrite it, you can set the condition to verify it. If it exists, it will throw an error.
PrimaryKey primaryKey = PrimaryKeyBuilder.createPrimaryKeyBuilder()
    .addPrimaryKeyColumn(PK1, PrimaryKeyValue.fromLong(1L))
    .addPrimaryKeyColumn(PK2, PrimaryKeyValue.fromString("string"))
    .build();

RowPutChange rowChange = new RowPutChange(TABLE_NAME, primaryKey);
rowChange.addColumn("string", ColumnValue.fromString("string value"))
    .addColumn("long", ColumnValue.fromLong(1L))
    .addColumn("boolean", ColumnValue.fromBoolean(true))
    .addColumn("double", ColumnValue.fromDouble(1.1))
    .addColumn("binary", ColumnValue.fromBinary(new byte[]{1, 2, 3}));

PutRowRequest putRowRequest = new PutRowRequest(rowChange);

syncClient.putRow(putRowRequest);

Data insertion - conditional insertion

Conditional write provides three line level conditional checks, all of which support write operations. The API provided by RowChange interface mainly includes the following methods:

  • IGNORE / / no check, default
  • Expect ﹣ exist ﹣ expect the line to exist before writing
  • Expect not exist / / the expected line does not exist before writing
// If the condition verification fails, it will throw an error: OTSConditionCheckFail
rowChange.setCondition(new Condition(RowExistenceExpectation.EXPECT_NOT_EXIST));

Data insertion - Auto increment column

Auto add column data insertion is supported, provided that the corresponding primary key column is set to auto add column when the table is created. When inserting data, the primary key of the column is set to a special column value.

PrimaryKey primaryKey = PrimaryKeyBuilder.createPrimaryKeyBuilder()
    .addPrimaryKeyColumn(PK1, PrimaryKeyValue.fromLong(1L))
    .addPrimaryKeyColumn(PK2, PrimaryKeyValue.fromString("string"))
    .addPrimaryKeyColumn(PK3, PrimaryKeyValue.AUTO_INCREMENT)    // Special column value of auto increasing column
    .build();

RowPutChange rowChange = new RowPutChange(TABLE_NAME_INC, primaryKey);
rowChange.addColumn("string", ColumnValue.fromString("string value"));

rowChange.setReturnType(ReturnType.RT_PK);                         // Get the actual inserted value of the primary key auto increment column

PutRowRequest putRowRequest = new PutRowRequest(rowChange);

PutRowResponse putRowResponse = syncClient.putRow(putRowRequest);

// The actual stored value of the auto increment column can only be obtained in the returned Response
PrimaryKey returnPrimaryKey = putRowResponse.getRow().getPrimaryKey();

Data deletion

Delete the data of the corresponding row based on the given complete PrimaryKey. If the row does not exist, the deletion operation will not be thrown wrong.
matters needing attention:

  • The primary key must be complete and consistent with the primary key name and type of the table attribute. If not, there will be parameter errors;
PrimaryKey primaryKey = PrimaryKeyBuilder.createPrimaryKeyBuilder()
    .addPrimaryKeyColumn(PK1, PrimaryKeyValue.fromLong(1L))
    .addPrimaryKeyColumn(PK2, PrimaryKeyValue.fromString("string"))
    .build();

RowDeleteChange rowChange = new RowDeleteChange(TABLE_NAME, primaryKey);
DeleteRowRequest getRowRequest = new DeleteRowRequest(rowChange);

syncClient.deleteRow(getRowRequest);

Data updating

Data update updates the specified fields of the specified rows, including the addition, deletion and modification of columns.
matters needing attention:

  • The primary key must be complete and consistent with the primary key name and type of the table attribute. If not, there will be parameter errors;

  • Support condition update:

    • Support single column value condition
    • Composite column value condition and, or, non combination of single column values

  • The operations can be:

    • Delete the specified version of the specified column
    • Delete all versions of the specified column
    • Add new columns
    • Update existing column
    • Specify column (integer column) to add atom: if you want to get the added value, you need to set ReturnType
PrimaryKey primaryKey = PrimaryKeyBuilder.createPrimaryKeyBuilder()
    .addPrimaryKeyColumn(PK1, PrimaryKeyValue.fromLong(1L))
    .addPrimaryKeyColumn(PK2, PrimaryKeyValue.fromString("string"))
    .build();

RowUpdateChange rowChange = new RowUpdateChange(TABLE_NAME, primaryKey);
rowChange
    .deleteColumns("binary")                                        // Delete all versions of the specified column
    .deleteColumn("double", timestamp)                                // Delete specified column specified version
    .put("string", ColumnValue.fromString("new string value"))        // Update existing column
    .put("newCol", ColumnValue.fromString("new col"))                 // Add new columns
    .increment(new Column("long", ColumnValue.fromLong(100L)));        // Atom plus
        
rowChange.setReturnType(ReturnType.RT_AFTER_MODIFY);                // Get the modified value of the attribute column atom
UpdateRowRequest updateRowRequest = new UpdateRowRequest(rowChange);

UpdateRowResponse updateRowResponse = syncClient.updateRow(updateRowRequest);

Data update - condition update

The condition of the row level supported by the Tablestore is more new. The user can provide the condition verification of the attribute column value to update the condition. Condition update is also an API provided by RowChange interface.
Condition update supports single column condition verification and combined attribute column condition verification. Only when the conditions are met can data update be completed. If the conditions are not met, errors will be thrown.

Single attribute column condition verification

Condition condition = new Condition(RowExistenceExpectation.EXPECT_EXIST); // Condition updating
condition.setColumnCondition(
        new SingleColumnValueCondition(
                "boolean",
                SingleColumnValueCondition.CompareOperator.EQUAL,
                ColumnValue.fromBoolean(true)));

rowChange.setCondition(condition);

Combination attribute column condition verification (and, or, or, non combination of multi column attributes)

CompositeColumnValueCondition colCondition = new CompositeColumnValueCondition(CompositeColumnValueCondition.LogicOperator.AND);

ColumnCondition subColCondition1 = new SingleColumnValueCondition(
        "boolean",
        SingleColumnValueCondition.CompareOperator.EQUAL,
        ColumnValue.fromBoolean(true));

ColumnCondition subColCondition2 = new SingleColumnValueCondition(
        "double",
        SingleColumnValueCondition.CompareOperator.GREATER_THAN,
        ColumnValue.fromLong(0L));

colCondition.addCondition(subColCondition1).addCondition(subColCondition2);
Condition condition = new Condition();
condition.setColumnCondition(colCondition);

rowChange.setCondition(condition);

Data query

Description: read the row data based on the given full PrimaryKey. If the data does not exist, Response.getRow() is empty.
matters needing attention:

  • The primary key must be complete and consistent with the primary key name and type of the table attribute. If not, there will be parameter errors;
  • All attribute columns are returned by default. If you specify attribute columns, only attribute column values are returned;
  • If the property column is specified, but the row data does not have the specified column property (there are other property columns), null will be returned, which does not mean that the row must not exist;
  • Multi version tables can return data of multiple versions of columns by setting MaxVersion;
PrimaryKey primaryKey = PrimaryKeyBuilder.createPrimaryKeyBuilder()
    .addPrimaryKeyColumn(PK1, PrimaryKeyValue.fromLong(1L))
    .addPrimaryKeyColumn(PK2, PrimaryKeyValue.fromString("string"))
    .build();

SingleRowQueryCriteria criteria = new SingleRowQueryCriteria(TABLE_NAME, primaryKey);
criteria.setMaxVersions(1);

GetRowRequest getRowRequest = new GetRowRequest(criteria);

GetRowResponse getRowResponse = syncClient.getRow(getRowRequest);

Batch operation

Data write - add, delete and modify

Both bulk data operation and single line operation are based on the request construction implemented by RowChange. Therefore, a BatchWriteRowRequest can contain three different operations: add, delete, and modify. At the same time, the table name parameters are contained in their own row information, so the mixed write operation of multi table data can be realized. Users can try it on their own.
matters needing attention:

  • Request failed, all line operations failed
  • If the request is successful, it does not mean that every line is successful. You need to determine whether the length of the failure list is zero. For failed rows, rebuild the request to try again.
  • There is a limit to the requests for batch write (add, delete, and modify) operation construction: the number of lines is less than or equal to 200, the total size is less than or equal to 4m, and errors will be thrown if the limit is exceeded
BatchWriteRowRequest batchWriteRowRequest = new BatchWriteRowRequest();

/**
 * Multi Put Row
 */
for (long i = 1L; i <= 10L; i++) {
    PrimaryKey primaryKey = PrimaryKeyBuilder.createPrimaryKeyBuilder()
        .addPrimaryKeyColumn(PK1, PrimaryKeyValue.fromLong(i))
        .addPrimaryKeyColumn(PK2, PrimaryKeyValue.fromString("string"))
        .build();

    RowPutChange rowChange = new RowPutChange(TABLE_NAME, primaryKey);
    rowChange.addColumn("long", ColumnValue.fromLong(i));

    batchWriteRowRequest.addRowChange(rowChange);
}

/**
 * Multi Update Row
 */
for (long i = 11L; i <= 20L; i++) {
    PrimaryKey primaryKey = PrimaryKeyBuilder.createPrimaryKeyBuilder()
        .addPrimaryKeyColumn(PK1, PrimaryKeyValue.fromLong(i))
        .addPrimaryKeyColumn(PK2, PrimaryKeyValue.fromString("string"))
        .build();

    RowUpdateChange rowChange = new RowUpdateChange(TABLE_NAME, primaryKey);
    rowChange.put("long", ColumnValue.fromLong(i));

    batchWriteRowRequest.addRowChange(rowChange);
}

/**
 * Multi Delete Row
 */
for (long i = 21L; i <= 30L; i++) {
    PrimaryKey primaryKey = PrimaryKeyBuilder.createPrimaryKeyBuilder()
        .addPrimaryKeyColumn(PK1, PrimaryKeyValue.fromLong(i))
        .addPrimaryKeyColumn(PK2, PrimaryKeyValue.fromString("string"))
        .build();

    RowDeleteChange rowChange = new RowDeleteChange(TABLE_NAME, primaryKey);

    batchWriteRowRequest.addRowChange(rowChange);
}

BatchWriteRowResponse batchWriteRowResponse = syncClient.batchWriteRow(batchWriteRowRequest);
int totalRowCount = batchWriteRowResponse.getRowStatus(TABLE_NAME).size();
int failedRowCount = batchWriteRowResponse.getFailedRows().size();

Batch write - failed row retry

After the batch write operation is successful, it does not mean that all the row operations requested by the batch are successful. The status of the row level needs to be judged separately. Return the failed row, rebuild the BatchWriteRow request for the failed row, and reissue the request.

if (batchWriteRowResponse.getFailedRows().size() > 0 ) {
    BatchWriteRowRequest retryRequest = batchWriteRowRequest.createRequestForRetry(
        batchWriteRowResponse.getFailedRows());

    syncClient.batchWriteRow(retryRequest);
}

Multi row random query

Batch read query supports the ability to read batch data at the same time under a single table. Users only need to add the complete PrimaryKey of the row when building a request.
matters needing attention:

  • Batch read request failed, each line read failed
  • If the batch read request is successful, it does not mean that every request is successful. You need to determine whether the length of the failure list is zero.
BatchGetRowRequest batchGetRowRequest = new BatchGetRowRequest();
MultiRowQueryCriteria criteria = new MultiRowQueryCriteria(TABLE_NAME);

for (long i = 1L; i <= 30L; i += 10) {
    PrimaryKey primaryKey = PrimaryKeyBuilder.createPrimaryKeyBuilder()
        .addPrimaryKeyColumn(PK1, PrimaryKeyValue.fromLong(i))
        .addPrimaryKeyColumn(PK2, PrimaryKeyValue.fromString("string"))
        .build();

    criteria.addRow(primaryKey);
}

criteria.setMaxVersions(1);
batchGetRowRequest.addMultiRowQueryCriteria(criteria);

BatchGetRowResponse batchGetRowResponse = syncClient.batchGetRow(batchGetRowRequest);
int totalRowCount = batchGetRowResponse.getBatchGetRowResult(TABLE_NAME).size();
int failedRowCount = batchGetRowResponse.getFailedRows().size();

Range query

The data in the table is arranged orderly based on the PrimaryKey, and the table is a special union index based on the primary key. Therefore, data range query follows the leftmost matching principle of union index. That is, if a column provides a specific range value (not MIN or MAX) during range query, the range limit of the next column is ignored because the range constraint of the next column is invalid. Refer to the following figure for details:

matters needing attention:

  • When the result is inconsistent with the condition, please refer to the leftmost matching principle
  • Range left open right closed, and includes start, but does not include end
  • In positive sequence, start must be less than end; in reverse sequence, start must be greater than end
GetRangeRequest getRangeRequest = new GetRangeRequest();

PrimaryKey start = PrimaryKeyBuilder.createPrimaryKeyBuilder()
    .addPrimaryKeyColumn(PK1, PrimaryKeyValue.fromLong(2))
    .addPrimaryKeyColumn(PK2, PrimaryKeyValue.fromString("5"))
    .build();

PrimaryKey end = PrimaryKeyBuilder.createPrimaryKeyBuilder()
    .addPrimaryKeyColumn(PK1, PrimaryKeyValue.fromLong(4))
    .addPrimaryKeyColumn(PK2, PrimaryKeyValue.fromString("4"))
    .build();

RangeRowQueryCriteria criteria = new RangeRowQueryCriteria(TABLE_NAME);
criteria.setInclusiveStartPrimaryKey(start);
criteria.setExclusiveEndPrimaryKey(end);
criteria.setDirection(Direction.FORWARD);            // Setup order
criteria.setMaxVersions(1);                            // Number of returned versions
criteria.setLimit(20);                                // Limit of rows returned per page

getRangeRequest.setRangeRowQueryCriteria(criteria);

GetRangeResponse getRangeResponse = syncClient.getRange(getRangeRequest);

Range query - continuous page turning

When a single request for range condition data fails to get the full amount of data, the user needs to page continuously based on NextStartPrimaryKey to traverse all hit data. If GetRangeResponse.getNextStartPrimaryKey() is not empty, there must be data. By setting the acquired PrimaryKey to the original request, rebuild it, and then initiate the request again.

PrimaryKey nextStartPrimaryKey = null;
do {
    GetRangeResponse getRangeResponse = syncClient.getRange(getRangeRequest);
    List<Row> pageRowList = getRangeResponse.getRows();

    nextStartPrimaryKey = getRangeResponse.getNextStartPrimaryKey();
    if (nextStartPrimaryKey != null) {                 // Determine the existence of the next page, and refactor the request
        criteria.setInclusiveStartPrimaryKey(nextStartPrimaryKey);
        getRangeRequest.setRangeRowQueryCriteria(criteria);
    }
} while (nextStartPrimaryKey != null);              // Keep turning until there is no next page

Range query - continuous pager iterator

In order to facilitate users to traverse full data, we provide an iterator interface. Users don't need to care about the logic of request construction and result judgment. They just need to build GetRangeIterator with asynchronous Client and request body as parameters.
matters needing attention:

  • The asynchronous Client used by iterator can be reconstructed or transformed by synchronous Client;
  • The iterator automatically initiates requests internally, and automatically initiates the next batch of requests after consuming one batch;
AsyncClient asyncClient = (AsyncClient) syncClient.asAsyncClient();
GetRangeIterator getRangeIterator = new GetRangeIterator(asyncClient, getRangeRequest);

while (getRangeIterator.hasNext()) {
    Row row = getRangeIterator.next();
}

Getting started with Tablestore

This paper introduces the basic functions and usage of Tablestore in data management with the interface calling code of Java SDK. The code has been open-source in the Tablestore examples project, and users can run it directly. Based on the sample code and articles, new users can start to use the Tablestore more simply and quickly, and welcome new and old users to use and make suggestions.

Through the continuous output of basic use functions, we will organize a complete set of start-up manuals (including executable samples), please look forward to.

Expert services

If you have any questions or need better online support, welcome to join pin group: "open communication group of form storage". Free online expert service is provided in the group, welcome to scan code to join, group number: 23307953


Keywords: Java Attribute less SDK

Added by Drayton on Tue, 03 Dec 2019 21:15:46 +0200

Popular Keywords

©2024 Programming VIP