Ytukyg

I have a record with two lists.
Multiple services are appending to those lists simultaneously, each service updates exactly one of those lists.

this is my code for creating the UpdateItemSpec

private UpdateItemSpec updateList1(List<String> values) {

        return new UpdateItemSpec()

                .withPrimaryKey("recordKey", "key")

                .withUpdateExpression("SET list1 = list_append(if_not_exists(list1, :initialVal), :val)")

                .withValueMap(new ValueMap().withList(":val", values)

                        .withList(":initialVal", new ArrayList<>()))

                .withReturnValues(ReturnValue.ALL_NEW);

    }



    private UpdateItemSpec updateList2(List<String> values) {

        return new UpdateItemSpec()

                .withPrimaryKey("recordKey", "key")

                .withUpdateExpression("SET list2 = list_append(if_not_exists(list2, :initialVal), :val)")

                .withValueMap(new ValueMap().withList(":val", values)

                        .withList(":initialVal", new ArrayList<>()))

                .withReturnValues(ReturnValue.ALL_NEW);

    }

Given that the updates should get me to the state in which the lists are equal (I don't know how many iterations are needed), my goal is to check if the lists are equal after each update (using the item that returned from the update). Do I need to do anything to make sure that the updates are atomic across the records and I won't miss the iteration in which the lists became equal?

In other words, does the update locks the record entirely, or is it just locks the currently updated field?

An update on a single item is atomic, so if you use ReturnValues.ALL_NEW, you will be able to see the full state of the item, and you will not miss it if the lists are equal. (See update documentation.)

Even though you will see when the lists are equal, it is possible that the lists will no longer be equal by the time you are able to react to them being equal. If you want to make sure that the list stays equal until you can react, then you will need a condition expression to prevent further updates or a distributed locking solution.

Edit (2018-11-22)

There is no documentation stating directly that an update on an item happens atomically, but we can reason that all attributes in a single item are updated atomically.

Let's start with the following assumptions:

1. A conditional write is entirely successful for the item if the condition is true.


2. A conditional write is rejected if the condition is false.


3. Conditional writes can be used for
   Optimistic Locking with Version Number.


4. A write does not update all attributes atomically.

Suppose we have an item with the attributes hashkey, version, and data. Let us say that hashkey=123, data = "Foo", and version = 1.

Suppose also that there are two requests (Request A and Request B) that come in at nearly the same time which are both using optimistic locking and both are trying to set a new value for data.

Here's a possible sequence of events

• Request A – Check condition version==1
  


• Request B – Check condition version==1
  


• Request A – data = "Bar"
  


• Request B – data = "Bag"
  


• Request A – version=version+1
  


• Request B – version=version+1
  

The final state of the item is

hashkey=123

version=3

data="Bag"

This state should not happen because of the optimistic locking.

If you use this strategy, then your database writes are protected from being overwritten by the writes of others — and vice-versa.

Now, you might object that all this shows is that the version increment occurs atomically with the condition evaluation. However, you should also remember that a condition expression can have an arbitrary number of conditions on an arbitrary number of attributes, so then all attributes mentioned in the condition expression would have to be updated atomically along with the condition expression evaluation.

However, this only demonstrates that at least some of the attributes must be updated atomically, not that all of them are. Well, there are two possible implementations here. (1) DynamoDB determines which attributes are present in the condition expression (anywhere from 0 to all of them) and updates only those ones atomically. (2) DynamoDB updates all the attributes for an item atomically. But consider this example in which the attributes not in the ConditionExpression affect the correctness of the update.

Suppose our item has 1 more attribute: canEdit, which is initially true. This time Request A is setting canEdit=false and Request B is setting values = list_append(values, "Bar").

If all attributes are not updated atomically, here's a possible sequence of events:

• Request A – Check condition version==1 AND data=="Foo"
  


• Request A – version = version+1
  


• Request A – `data = "Foo"


• Request B – Check condition canEdit==true
  


• Request B – version = version+1
  


• Request B – canEdit = true
  


• Request A – canEdit = false
  


• Request B – data = "Bar"
  

In both cases, the condition is true when initially evaluated, and in both cases, something the request wrote was overridden by the other request. DynamoDB will not allow both of these conditional updates to succeed, therefore DynamoDB must update all attributes in an item atomically because that is the only way to guarantee that either A or B will fail with a ConditionCheckException.

If my reasoning is correct, then in order for 1-3 to be true, 4 must be false, and all attributes in a single item are updated atomically.