Yet another optimistic locking question

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

Yet another optimistic locking question

Gili

        A question about how optimistic locking is currently implemented. Do we
implement it like this?

1) Lock row
2) Read row
3) Compare read row to DataObject version of row
4) If values mismatch, unlock the row and throw an exception
5) If values match, continue with update and unlock row

        or do we not lock the database at all? If we don't lock it at all, how
can we ensure that no changes occur after step 3 but before step 5?

Thank you,
Gili
--
http://www.desktopbeautifier.com/
Reply | Threaded
Open this post in threaded view
|

RE: Yet another optimistic locking question

Gentry, Michael (Contractor)
Optimistic locking never locks the row in the database (it is
optimistic).  Read:

http://www.objectstyle.org/confluence/display/CAY/Optimistic+Locking+Exp
lained

It explains how Cayenne can ensure that no changes occurred between the
SELECT and UPDATE phase.  If you still have questions I'll try to answer
them.

Thanks,

/dev/mrg


-----Original Message-----
From: Gili [mailto:[hidden email]]
Sent: Thursday, September 01, 2005 12:06 AM
To: [hidden email]
Subject: Yet another optimistic locking question



        A question about how optimistic locking is currently
implemented. Do we
implement it like this?

1) Lock row
2) Read row
3) Compare read row to DataObject version of row
4) If values mismatch, unlock the row and throw an exception
5) If values match, continue with update and unlock row

        or do we not lock the database at all? If we don't lock it at
all, how
can we ensure that no changes occur after step 3 but before step 5?

Thank you,
Gili
--
http://www.desktopbeautifier.com/
Reply | Threaded
Open this post in threaded view
|

Re: Yet another optimistic locking question

Gili

        Oh my. It all makes so much more sense now... So if I understand it
correctly,

1) We store the perceived DB value somewhere
2) We store the cached (maybe modified) value elsewhere
3) When a commit occurs, we compare the objects in 1 and 2, then issue a
UPDATE only for fields which have changed.

        Cool :) This also sounds quite efficient to me.

Thank you,
Gili

Gentry, Michael (Contractor) wrote:

> Optimistic locking never locks the row in the database (it is
> optimistic).  Read:
>
> http://www.objectstyle.org/confluence/display/CAY/Optimistic+Locking+Exp
> lained
>
> It explains how Cayenne can ensure that no changes occurred between the
> SELECT and UPDATE phase.  If you still have questions I'll try to answer
> them.
>
> Thanks,
>
> /dev/mrg
>
>
> -----Original Message-----
> From: Gili [mailto:[hidden email]]
> Sent: Thursday, September 01, 2005 12:06 AM
> To: [hidden email]
> Subject: Yet another optimistic locking question
>
>
>
> A question about how optimistic locking is currently
> implemented. Do we
> implement it like this?
>
> 1) Lock row
> 2) Read row
> 3) Compare read row to DataObject version of row
> 4) If values mismatch, unlock the row and throw an exception
> 5) If values match, continue with update and unlock row
>
> or do we not lock the database at all? If we don't lock it at
> all, how
> can we ensure that no changes occur after step 3 but before step 5?
>
> Thank you,
> Gili

--
http://www.desktopbeautifier.com/
Reply | Threaded
Open this post in threaded view
|

Re: Yet another optimistic locking question

Mike Kienenberger
Yeah, it's basically an atomic db operation that says UPDATE set
values WHERE all fields marked for optimistic locking haven't changed
values from the last time we read them.

On 9/1/05, Gili <[hidden email]> wrote:

>
>         Oh my. It all makes so much more sense now... So if I understand it
> correctly,
>
> 1) We store the perceived DB value somewhere
> 2) We store the cached (maybe modified) value elsewhere
> 3) When a commit occurs, we compare the objects in 1 and 2, then issue a
> UPDATE only for fields which have changed.
>
>         Cool :) This also sounds quite efficient to me.
>
> Thank you,
> Gili
>
> Gentry, Michael (Contractor) wrote:
> > Optimistic locking never locks the row in the database (it is
> > optimistic).  Read:
> >
> > http://www.objectstyle.org/confluence/display/CAY/Optimistic+Locking+Exp
> > lained
> >
> > It explains how Cayenne can ensure that no changes occurred between the
> > SELECT and UPDATE phase.  If you still have questions I'll try to answer
> > them.
> >
> > Thanks,
> >
> > /dev/mrg
> >
> >
> > -----Original Message-----
> > From: Gili [mailto:[hidden email]]
> > Sent: Thursday, September 01, 2005 12:06 AM
> > To: [hidden email]
> > Subject: Yet another optimistic locking question
> >
> >
> >
> >       A question about how optimistic locking is currently
> > implemented. Do we
> > implement it like this?
> >
> > 1) Lock row
> > 2) Read row
> > 3) Compare read row to DataObject version of row
> > 4) If values mismatch, unlock the row and throw an exception
> > 5) If values match, continue with update and unlock row
> >
> >       or do we not lock the database at all? If we don't lock it at
> > all, how
> > can we ensure that no changes occur after step 3 but before step 5?
> >
> > Thank you,
> > Gili
>
> --
> http://www.desktopbeautifier.com/
>
Reply | Threaded
Open this post in threaded view
|

Re: Yet another optimistic locking question

Gili

        And I forgot to mention, in step 3 I assume we look at the return value
from the DB and if we expected 1 change and got 0 this means we detect
that our DB representation was out of date and we throw an exception,
correct?

Gili

Mike Kienenberger wrote:

> Yeah, it's basically an atomic db operation that says UPDATE set
> values WHERE all fields marked for optimistic locking haven't changed
> values from the last time we read them.
>
> On 9/1/05, Gili <[hidden email]> wrote:
>
>>        Oh my. It all makes so much more sense now... So if I understand it
>>correctly,
>>
>>1) We store the perceived DB value somewhere
>>2) We store the cached (maybe modified) value elsewhere
>>3) When a commit occurs, we compare the objects in 1 and 2, then issue a
>>UPDATE only for fields which have changed.
>>
>>        Cool :) This also sounds quite efficient to me.
>>
>>Thank you,
>>Gili
>>
>>Gentry, Michael (Contractor) wrote:
>>
>>>Optimistic locking never locks the row in the database (it is
>>>optimistic).  Read:
>>>
>>>http://www.objectstyle.org/confluence/display/CAY/Optimistic+Locking+Exp
>>>lained
>>>
>>>It explains how Cayenne can ensure that no changes occurred between the
>>>SELECT and UPDATE phase.  If you still have questions I'll try to answer
>>>them.
>>>
>>>Thanks,
>>>
>>>/dev/mrg
>>>
>>>
>>>-----Original Message-----
>>>From: Gili [mailto:[hidden email]]
>>>Sent: Thursday, September 01, 2005 12:06 AM
>>>To: [hidden email]
>>>Subject: Yet another optimistic locking question
>>>
>>>
>>>
>>>      A question about how optimistic locking is currently
>>>implemented. Do we
>>>implement it like this?
>>>
>>>1) Lock row
>>>2) Read row
>>>3) Compare read row to DataObject version of row
>>>4) If values mismatch, unlock the row and throw an exception
>>>5) If values match, continue with update and unlock row
>>>
>>>      or do we not lock the database at all? If we don't lock it at
>>>all, how
>>>can we ensure that no changes occur after step 3 but before step 5?
>>>
>>>Thank you,
>>>Gili
>>
>>--
>>http://www.desktopbeautifier.com/
>>
>
>

--
http://www.desktopbeautifier.com/
Reply | Threaded
Open this post in threaded view
|

Re: Yet another optimistic locking question

Mike Kienenberger
Yep!

On 9/1/05, Gili <[hidden email]> wrote:

>
>         And I forgot to mention, in step 3 I assume we look at the return value
> from the DB and if we expected 1 change and got 0 this means we detect
> that our DB representation was out of date and we throw an exception,
> correct?
>
> Gili
>
> Mike Kienenberger wrote:
> > Yeah, it's basically an atomic db operation that says UPDATE set
> > values WHERE all fields marked for optimistic locking haven't changed
> > values from the last time we read them.
> >
> > On 9/1/05, Gili <[hidden email]> wrote:
> >
> >>        Oh my. It all makes so much more sense now... So if I understand it
> >>correctly,
> >>
> >>1) We store the perceived DB value somewhere
> >>2) We store the cached (maybe modified) value elsewhere
> >>3) When a commit occurs, we compare the objects in 1 and 2, then issue a
> >>UPDATE only for fields which have changed.
> >>
> >>        Cool :) This also sounds quite efficient to me.
> >>
> >>Thank you,
> >>Gili
> >>
> >>Gentry, Michael (Contractor) wrote:
> >>
> >>>Optimistic locking never locks the row in the database (it is
> >>>optimistic).  Read:
> >>>
> >>>http://www.objectstyle.org/confluence/display/CAY/Optimistic+Locking+Exp
> >>>lained
> >>>
> >>>It explains how Cayenne can ensure that no changes occurred between the
> >>>SELECT and UPDATE phase.  If you still have questions I'll try to answer
> >>>them.
> >>>
> >>>Thanks,
> >>>
> >>>/dev/mrg
> >>>
> >>>
> >>>-----Original Message-----
> >>>From: Gili [mailto:[hidden email]]
> >>>Sent: Thursday, September 01, 2005 12:06 AM
> >>>To: [hidden email]
> >>>Subject: Yet another optimistic locking question
> >>>
> >>>
> >>>
> >>>      A question about how optimistic locking is currently
> >>>implemented. Do we
> >>>implement it like this?
> >>>
> >>>1) Lock row
> >>>2) Read row
> >>>3) Compare read row to DataObject version of row
> >>>4) If values mismatch, unlock the row and throw an exception
> >>>5) If values match, continue with update and unlock row
> >>>
> >>>      or do we not lock the database at all? If we don't lock it at
> >>>all, how
> >>>can we ensure that no changes occur after step 3 but before step 5?
> >>>
> >>>Thank you,
> >>>Gili
> >>
> >>--
> >>http://www.desktopbeautifier.com/
> >>
> >
> >
>
> --
> http://www.desktopbeautifier.com/
>
Reply | Threaded
Open this post in threaded view
|

Re: Yet another optimistic locking question

Gili

        Here is an idea for us to further optimize the process. Can we perhaps
detect whether the user ever modified a field without comparing the two
states? For example, if one of my fields is a large BLOB (byte[]) then
when I get() that array I could concievable modify it. So then what I'm
thinking is if the user ever invoked get() or set() on that field, we
toggle the appropriate value in a BitSet to indicate we should look at
it in step 3. If the user never touched a field, we can very quickly
(regardless of its size) know that it has not been modified without
comparing the actual contents.

        Using a BitSet this would be very cheap to do as well. What do you think?

Gili

Mike Kienenberger wrote:

> Yep!
>
> On 9/1/05, Gili <[hidden email]> wrote:
>
>>        And I forgot to mention, in step 3 I assume we look at the return value
>>from the DB and if we expected 1 change and got 0 this means we detect
>>that our DB representation was out of date and we throw an exception,
>>correct?
>>
>>Gili
>>
>>Mike Kienenberger wrote:
>>
>>>Yeah, it's basically an atomic db operation that says UPDATE set
>>>values WHERE all fields marked for optimistic locking haven't changed
>>>values from the last time we read them.
>>>
>>>On 9/1/05, Gili <[hidden email]> wrote:
>>>
>>>
>>>>       Oh my. It all makes so much more sense now... So if I understand it
>>>>correctly,
>>>>
>>>>1) We store the perceived DB value somewhere
>>>>2) We store the cached (maybe modified) value elsewhere
>>>>3) When a commit occurs, we compare the objects in 1 and 2, then issue a
>>>>UPDATE only for fields which have changed.
>>>>
>>>>       Cool :) This also sounds quite efficient to me.
>>>>
>>>>Thank you,
>>>>Gili
>>>>
>>>>Gentry, Michael (Contractor) wrote:
>>>>
>>>>
>>>>>Optimistic locking never locks the row in the database (it is
>>>>>optimistic).  Read:
>>>>>
>>>>>http://www.objectstyle.org/confluence/display/CAY/Optimistic+Locking+Exp
>>>>>lained
>>>>>
>>>>>It explains how Cayenne can ensure that no changes occurred between the
>>>>>SELECT and UPDATE phase.  If you still have questions I'll try to answer
>>>>>them.
>>>>>
>>>>>Thanks,
>>>>>
>>>>>/dev/mrg
>>>>>
>>>>>
>>>>>-----Original Message-----
>>>>>From: Gili [mailto:[hidden email]]
>>>>>Sent: Thursday, September 01, 2005 12:06 AM
>>>>>To: [hidden email]
>>>>>Subject: Yet another optimistic locking question
>>>>>
>>>>>
>>>>>
>>>>>     A question about how optimistic locking is currently
>>>>>implemented. Do we
>>>>>implement it like this?
>>>>>
>>>>>1) Lock row
>>>>>2) Read row
>>>>>3) Compare read row to DataObject version of row
>>>>>4) If values mismatch, unlock the row and throw an exception
>>>>>5) If values match, continue with update and unlock row
>>>>>
>>>>>     or do we not lock the database at all? If we don't lock it at
>>>>>all, how
>>>>>can we ensure that no changes occur after step 3 but before step 5?
>>>>>
>>>>>Thank you,
>>>>>Gili
>>>>
>>>>--
>>>>http://www.desktopbeautifier.com/
>>>>
>>>
>>>
>>--
>>http://www.desktopbeautifier.com/
>>
>
>

--
http://www.desktopbeautifier.com/
Reply | Threaded
Open this post in threaded view
|

Re: Yet another optimistic locking question

Gili

        Forgot to mention, for primitive types (like int, float, etc) the
getter wouldn't flag the field as 'touched' because they're returned by
value. So for the majority of cases (i.e. Strings) we'd have a *very*
fast comparison.

Gili

Gili wrote:

>
>     Here is an idea for us to further optimize the process. Can we
> perhaps detect whether the user ever modified a field without comparing
> the two states? For example, if one of my fields is a large BLOB
> (byte[]) then when I get() that array I could concievable modify it. So
> then what I'm thinking is if the user ever invoked get() or set() on
> that field, we toggle the appropriate value in a BitSet to indicate we
> should look at it in step 3. If the user never touched a field, we can
> very quickly (regardless of its size) know that it has not been modified
> without comparing the actual contents.
>
>     Using a BitSet this would be very cheap to do as well. What do you
> think?
>
> Gili
>
> Mike Kienenberger wrote:
>
>> Yep!
>>
>> On 9/1/05, Gili <[hidden email]> wrote:
>>
>>>        And I forgot to mention, in step 3 I assume we look at the
>>> return value
>>> from the DB and if we expected 1 change and got 0 this means we detect
>>> that our DB representation was out of date and we throw an exception,
>>> correct?
>>>
>>> Gili
>>>
>>> Mike Kienenberger wrote:
>>>
>>>> Yeah, it's basically an atomic db operation that says UPDATE set
>>>> values WHERE all fields marked for optimistic locking haven't changed
>>>> values from the last time we read them.
>>>>
>>>> On 9/1/05, Gili <[hidden email]> wrote:
>>>>
>>>>
>>>>>       Oh my. It all makes so much more sense now... So if I
>>>>> understand it
>>>>> correctly,
>>>>>
>>>>> 1) We store the perceived DB value somewhere
>>>>> 2) We store the cached (maybe modified) value elsewhere
>>>>> 3) When a commit occurs, we compare the objects in 1 and 2, then
>>>>> issue a
>>>>> UPDATE only for fields which have changed.
>>>>>
>>>>>       Cool :) This also sounds quite efficient to me.
>>>>>
>>>>> Thank you,
>>>>> Gili
>>>>>
>>>>> Gentry, Michael (Contractor) wrote:
>>>>>
>>>>>
>>>>>> Optimistic locking never locks the row in the database (it is
>>>>>> optimistic).  Read:
>>>>>>
>>>>>> http://www.objectstyle.org/confluence/display/CAY/Optimistic+Locking+Exp 
>>>>>>
>>>>>> lained
>>>>>>
>>>>>> It explains how Cayenne can ensure that no changes occurred
>>>>>> between the
>>>>>> SELECT and UPDATE phase.  If you still have questions I'll try to
>>>>>> answer
>>>>>> them.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> /dev/mrg
>>>>>>
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Gili [mailto:[hidden email]]
>>>>>> Sent: Thursday, September 01, 2005 12:06 AM
>>>>>> To: [hidden email]
>>>>>> Subject: Yet another optimistic locking question
>>>>>>
>>>>>>
>>>>>>
>>>>>>     A question about how optimistic locking is currently
>>>>>> implemented. Do we
>>>>>> implement it like this?
>>>>>>
>>>>>> 1) Lock row
>>>>>> 2) Read row
>>>>>> 3) Compare read row to DataObject version of row
>>>>>> 4) If values mismatch, unlock the row and throw an exception
>>>>>> 5) If values match, continue with update and unlock row
>>>>>>
>>>>>>     or do we not lock the database at all? If we don't lock it at
>>>>>> all, how
>>>>>> can we ensure that no changes occur after step 3 but before step 5?
>>>>>>
>>>>>> Thank you,
>>>>>> Gili
>>>>>
>>>>>
>>>>> --
>>>>> http://www.desktopbeautifier.com/
>>>>>
>>>>
>>>>
>>> --
>>> http://www.desktopbeautifier.com/
>>>
>>
>>
>

--
http://www.desktopbeautifier.com/
Reply | Threaded
Open this post in threaded view
|

Re: Yet another optimistic locking question

Mike Kienenberger
In reply to this post by Gili
No, that doesn't work.  The "checking" part is executed as part of the
database operation.
The database "checks" if the value has changed as part of the update
statement, not the java code.   We supply the original values as part
of the query, and the database does the comparison.  Optimistic
locking in general isn't specific to cayenne so the process is well
understood and probably as optimized as it can be.  Optimizations to
the concept are the timestamp and versioning alternatives of
optimistic locking where you only lock on a timestamp (assumes that
any database operation must occur at different timestamps) or versions
(which forces the caller to maintain versioning).   The downsides of
these optimizations are that they take up extra database space (on
field per table) and that they consider "differences that make no
difference" as a difference.

Ie, attribute locking works even if, in the mean time, someone changed
a field value then changed it back.   But versioning/timestamping will
fail even if the current state is the same as the original perceived
state.

The downsides of attribute locking is that it requires more bandwidth
(multiple where clauses transmitted) and processing on the database
(multiple where clauses computed)

On 9/1/05, Gili <[hidden email]> wrote:

>
>         Here is an idea for us to further optimize the process. Can we perhaps
> detect whether the user ever modified a field without comparing the two
> states? For example, if one of my fields is a large BLOB (byte[]) then
> when I get() that array I could concievable modify it. So then what I'm
> thinking is if the user ever invoked get() or set() on that field, we
> toggle the appropriate value in a BitSet to indicate we should look at
> it in step 3. If the user never touched a field, we can very quickly
> (regardless of its size) know that it has not been modified without
> comparing the actual contents.
>
>         Using a BitSet this would be very cheap to do as well. What do you think?
>
> Gili
>
> Mike Kienenberger wrote:
> > Yep!
> >
> > On 9/1/05, Gili <[hidden email]> wrote:
> >
> >>        And I forgot to mention, in step 3 I assume we look at the return value
> >>from the DB and if we expected 1 change and got 0 this means we detect
> >>that our DB representation was out of date and we throw an exception,
> >>correct?
> >>
> >>Gili
> >>
> >>Mike Kienenberger wrote:
> >>
> >>>Yeah, it's basically an atomic db operation that says UPDATE set
> >>>values WHERE all fields marked for optimistic locking haven't changed
> >>>values from the last time we read them.
> >>>
> >>>On 9/1/05, Gili <[hidden email]> wrote:
> >>>
> >>>
> >>>>       Oh my. It all makes so much more sense now... So if I understand it
> >>>>correctly,
> >>>>
> >>>>1) We store the perceived DB value somewhere
> >>>>2) We store the cached (maybe modified) value elsewhere
> >>>>3) When a commit occurs, we compare the objects in 1 and 2, then issue a
> >>>>UPDATE only for fields which have changed.
> >>>>
> >>>>       Cool :) This also sounds quite efficient to me.
> >>>>
> >>>>Thank you,
> >>>>Gili
> >>>>
> >>>>Gentry, Michael (Contractor) wrote:
> >>>>
> >>>>
> >>>>>Optimistic locking never locks the row in the database (it is
> >>>>>optimistic).  Read:
> >>>>>
> >>>>>http://www.objectstyle.org/confluence/display/CAY/Optimistic+Locking+Exp
> >>>>>lained
> >>>>>
> >>>>>It explains how Cayenne can ensure that no changes occurred between the
> >>>>>SELECT and UPDATE phase.  If you still have questions I'll try to answer
> >>>>>them.
> >>>>>
> >>>>>Thanks,
> >>>>>
> >>>>>/dev/mrg
> >>>>>
> >>>>>
> >>>>>-----Original Message-----
> >>>>>From: Gili [mailto:[hidden email]]
> >>>>>Sent: Thursday, September 01, 2005 12:06 AM
> >>>>>To: [hidden email]
> >>>>>Subject: Yet another optimistic locking question
> >>>>>
> >>>>>
> >>>>>
> >>>>>     A question about how optimistic locking is currently
> >>>>>implemented. Do we
> >>>>>implement it like this?
> >>>>>
> >>>>>1) Lock row
> >>>>>2) Read row
> >>>>>3) Compare read row to DataObject version of row
> >>>>>4) If values mismatch, unlock the row and throw an exception
> >>>>>5) If values match, continue with update and unlock row
> >>>>>
> >>>>>     or do we not lock the database at all? If we don't lock it at
> >>>>>all, how
> >>>>>can we ensure that no changes occur after step 3 but before step 5?
> >>>>>
> >>>>>Thank you,
> >>>>>Gili
> >>>>
> >>>>--
> >>>>http://www.desktopbeautifier.com/
> >>>>
> >>>
> >>>
> >>--
> >>http://www.desktopbeautifier.com/
> >>
> >
> >
>
> --
> http://www.desktopbeautifier.com/
>
Reply | Threaded
Open this post in threaded view
|

RE: Yet another optimistic locking question

Gentry, Michael (Contractor)
In reply to this post by Gili
Well, step 1 is actual (not just perceived) at the time of the SELECT.
We are optimistically (hence the name) expecting it to be actual at the
time of an UPDATE, too.

Step 2: Yes, Cayenne caches the original DB values.  This is how Cayenne
can determine what has changed (if anything).  I don't believe Hibernate
has this feature, but I could be wrong.

As for 3, there should only be SET clauses for things that have changed
(since Cayenne knows this).  So, for example, your BLOB wouldn't be sent
to the DB (in the SET clause section) unless you had changed it, but
this has nothing to do with optimistic locking.  :-)  Optimistic locking
is really about the WHERE clause of the UPDATE (which is determined by
your primary keys and the fields you mark for optimistic locking).
Cayenne being efficient about what it SETs is just Cayenne being
efficient -- and if nothing was changed, no UPDATEs.

This is also why you don't want to optimistically lock on a BLOB (you'd
always have to send the entire BLOB over, even if it didn't change).
And if the BLOB did change and you are optimistically locking on it, you
get to send it over TWICE (original + changed).  Imagine that for a
multi-megabyte values.  It is far more efficient to compute a 24 byte
(or whatever size) SHA/MD5 hash of the BLOB and only lock on it.  Write
a cover method on your setBlob() and re-compute and set the hash when
the BLOB changes.  And store BLOBs in a separate table.  You'll be
happier and faster.  :-)

Also, when you do your commitChanges(), Cayenne is going to update the
snapshot of DB values with what was committed.  This is why you can't
rollback past the commitChanges().  What the DB has and what Cayenne has
are in-sync when the commit succeeds.  If there is an error, you get the
exception and can handle it as appropriate for your application.

/dev/mrg


-----Original Message-----
From: Gili [mailto:[hidden email]]
Sent: Thursday, September 01, 2005 11:02 AM
To: [hidden email]
Subject: Re: Yet another optimistic locking question



        Oh my. It all makes so much more sense now... So if I understand
it
correctly,

1) We store the perceived DB value somewhere
2) We store the cached (maybe modified) value elsewhere
3) When a commit occurs, we compare the objects in 1 and 2, then issue a

UPDATE only for fields which have changed.

        Cool :) This also sounds quite efficient to me.

Thank you,
Gili

Gentry, Michael (Contractor) wrote:
> Optimistic locking never locks the row in the database (it is
> optimistic).  Read:
>
>
http://www.objectstyle.org/confluence/display/CAY/Optimistic+Locking+Exp
> lained
>
> It explains how Cayenne can ensure that no changes occurred between
the
> SELECT and UPDATE phase.  If you still have questions I'll try to
answer

> them.
>
> Thanks,
>
> /dev/mrg
>
>
> -----Original Message-----
> From: Gili [mailto:[hidden email]]
> Sent: Thursday, September 01, 2005 12:06 AM
> To: [hidden email]
> Subject: Yet another optimistic locking question
>
>
>
> A question about how optimistic locking is currently
> implemented. Do we
> implement it like this?
>
> 1) Lock row
> 2) Read row
> 3) Compare read row to DataObject version of row
> 4) If values mismatch, unlock the row and throw an exception
> 5) If values match, continue with update and unlock row
>
> or do we not lock the database at all? If we don't lock it at
> all, how
> can we ensure that no changes occur after step 3 but before step 5?
>
> Thank you,
> Gili

--
http://www.desktopbeautifier.com/
Reply | Threaded
Open this post in threaded view
|

RE: Yet another optimistic locking question

Gentry, Michael (Contractor)
In reply to this post by Gili
I believe he is talking about the SET clause, not WHERE clause, of an
UPDATE statement (we've veered off optimistic locking).

Cayenne does indeed do comparisons to determine what to include in the
SET clause.  It's been a few months since I looked at it, but I think it
brute-force compares every single attribute, so it is possible some kind
of mask to exclude things that never had set* called on them could be
useful.  Of course, in a web application where you might have your
object bound to fields in the GUI, set* would be called all the time,
even if nothing changed.

I think this is worth discussing, but it might end up being a wash for
most things.  For most objects, doing the comparisons isn't terribly
time consuming.  Of course, for a large DataContext, this could slow
things down, too.

/dev/mrg


-----Original Message-----
From: Mike Kienenberger [mailto:[hidden email]]
Sent: Thursday, September 01, 2005 11:34 AM
To: [hidden email]
Subject: Re: Yet another optimistic locking question


No, that doesn't work.  The "checking" part is executed as part of the
database operation.
The database "checks" if the value has changed as part of the update
statement, not the java code.   We supply the original values as part
of the query, and the database does the comparison.  Optimistic
locking in general isn't specific to cayenne so the process is well
understood and probably as optimized as it can be.  Optimizations to
the concept are the timestamp and versioning alternatives of
optimistic locking where you only lock on a timestamp (assumes that
any database operation must occur at different timestamps) or versions
(which forces the caller to maintain versioning).   The downsides of
these optimizations are that they take up extra database space (on
field per table) and that they consider "differences that make no
difference" as a difference.

Ie, attribute locking works even if, in the mean time, someone changed
a field value then changed it back.   But versioning/timestamping will
fail even if the current state is the same as the original perceived
state.

The downsides of attribute locking is that it requires more bandwidth
(multiple where clauses transmitted) and processing on the database
(multiple where clauses computed)

On 9/1/05, Gili <[hidden email]> wrote:
>
>         Here is an idea for us to further optimize the process. Can we
perhaps
> detect whether the user ever modified a field without comparing the
two
> states? For example, if one of my fields is a large BLOB (byte[]) then
> when I get() that array I could concievable modify it. So then what
I'm
> thinking is if the user ever invoked get() or set() on that field, we
> toggle the appropriate value in a BitSet to indicate we should look at
> it in step 3. If the user never touched a field, we can very quickly
> (regardless of its size) know that it has not been modified without
> comparing the actual contents.
>
>         Using a BitSet this would be very cheap to do as well. What do
you think?
>
> Gili
>
> Mike Kienenberger wrote:
> > Yep!
> >
> > On 9/1/05, Gili <[hidden email]> wrote:
> >
> >>        And I forgot to mention, in step 3 I assume we look at the
return value
> >>from the DB and if we expected 1 change and got 0 this means we
detect
> >>that our DB representation was out of date and we throw an
exception,
> >>correct?
> >>
> >>Gili
> >>
> >>Mike Kienenberger wrote:
> >>
> >>>Yeah, it's basically an atomic db operation that says UPDATE set
> >>>values WHERE all fields marked for optimistic locking haven't
changed
> >>>values from the last time we read them.
> >>>
> >>>On 9/1/05, Gili <[hidden email]> wrote:
> >>>
> >>>
> >>>>       Oh my. It all makes so much more sense now... So if I
understand it
> >>>>correctly,
> >>>>
> >>>>1) We store the perceived DB value somewhere
> >>>>2) We store the cached (maybe modified) value elsewhere
> >>>>3) When a commit occurs, we compare the objects in 1 and 2, then
issue a

> >>>>UPDATE only for fields which have changed.
> >>>>
> >>>>       Cool :) This also sounds quite efficient to me.
> >>>>
> >>>>Thank you,
> >>>>Gili
> >>>>
> >>>>Gentry, Michael (Contractor) wrote:
> >>>>
> >>>>
> >>>>>Optimistic locking never locks the row in the database (it is
> >>>>>optimistic).  Read:
> >>>>>
>
>>>>>http://www.objectstyle.org/confluence/display/CAY/Optimistic+Lockin
g+Exp
> >>>>>lained
> >>>>>
> >>>>>It explains how Cayenne can ensure that no changes occurred
between the
> >>>>>SELECT and UPDATE phase.  If you still have questions I'll try to
answer

> >>>>>them.
> >>>>>
> >>>>>Thanks,
> >>>>>
> >>>>>/dev/mrg
> >>>>>
> >>>>>
> >>>>>-----Original Message-----
> >>>>>From: Gili [mailto:[hidden email]]
> >>>>>Sent: Thursday, September 01, 2005 12:06 AM
> >>>>>To: [hidden email]
> >>>>>Subject: Yet another optimistic locking question
> >>>>>
> >>>>>
> >>>>>
> >>>>>     A question about how optimistic locking is currently
> >>>>>implemented. Do we
> >>>>>implement it like this?
> >>>>>
> >>>>>1) Lock row
> >>>>>2) Read row
> >>>>>3) Compare read row to DataObject version of row
> >>>>>4) If values mismatch, unlock the row and throw an exception
> >>>>>5) If values match, continue with update and unlock row
> >>>>>
> >>>>>     or do we not lock the database at all? If we don't lock it
at
> >>>>>all, how
> >>>>>can we ensure that no changes occur after step 3 but before step
5?

> >>>>>
> >>>>>Thank you,
> >>>>>Gili
> >>>>
> >>>>--
> >>>>http://www.desktopbeautifier.com/
> >>>>
> >>>
> >>>
> >>--
> >>http://www.desktopbeautifier.com/
> >>
> >
> >
>
> --
> http://www.desktopbeautifier.com/
>
Reply | Threaded
Open this post in threaded view
|

Re: Yet another optimistic locking question

Mike Kienenberger
In reply to this post by Gentry, Michael (Contractor)
On 9/1/05, Gentry, Michael (Contractor) <[hidden email]> wrote:
> Step 2: Yes, Cayenne caches the original DB values.  This is how Cayenne
> can determine what has changed (if anything).  I don't believe Hibernate
> has this feature, but I could be wrong.

I think Hibernate has this but only so long as your DataObject is part
of a hibernate Session.
Once you disconnect the object, then you lose the information.

If I am remembering correctly, Gili pointed out to me earlier that a
Session maintains both the backing cache and the database connection
and a Transaction maintains modifications.   Under Cayenne, we
maintain the connections separately, but combine the caching and
transactions in our data context.
Reply | Threaded
Open this post in threaded view
|

Re: Yet another optimistic locking question

Gili
In reply to this post by Mike Kienenberger

        I think you misunderstand me.

        I'm saying that in part 3 Cayenne compares its internal representation
of 1 and 2 and it gathers up all fields that the user has changed during
the transaction. Then it generates a "UPDATE SET" statement only
consisting of fields the user has changed. Correct?

        All I am saying is that we can optimize comparison on Cayenne's part
(when comparing part 1 and 2) because you could use a BitSet to know
whether a field has been touched or not without having to compare the
entire field value.

        This even makes BLOB optimistic locking much cheaper because the only
time this becomes expensive. If we can wrap the BLOB data in some
read-only structure so the getter() returns an unmodifiable array, then
we can quite easily detect whether a blob has been changed or not
without comparing its entire value.

        We still have send the changed data to the DB and granted this is slow,
but I'm saying we can optimize the comparison that occurs before that.
Did I miss something? :)

Gili

Mike Kienenberger wrote:

> No, that doesn't work.  The "checking" part is executed as part of the
> database operation.
> The database "checks" if the value has changed as part of the update
> statement, not the java code.   We supply the original values as part
> of the query, and the database does the comparison.  Optimistic
> locking in general isn't specific to cayenne so the process is well
> understood and probably as optimized as it can be.  Optimizations to
> the concept are the timestamp and versioning alternatives of
> optimistic locking where you only lock on a timestamp (assumes that
> any database operation must occur at different timestamps) or versions
> (which forces the caller to maintain versioning).   The downsides of
> these optimizations are that they take up extra database space (on
> field per table) and that they consider "differences that make no
> difference" as a difference.
>
> Ie, attribute locking works even if, in the mean time, someone changed
> a field value then changed it back.   But versioning/timestamping will
> fail even if the current state is the same as the original perceived
> state.
>
> The downsides of attribute locking is that it requires more bandwidth
> (multiple where clauses transmitted) and processing on the database
> (multiple where clauses computed)
>
> On 9/1/05, Gili <[hidden email]> wrote:
>
>>        Here is an idea for us to further optimize the process. Can we perhaps
>>detect whether the user ever modified a field without comparing the two
>>states? For example, if one of my fields is a large BLOB (byte[]) then
>>when I get() that array I could concievable modify it. So then what I'm
>>thinking is if the user ever invoked get() or set() on that field, we
>>toggle the appropriate value in a BitSet to indicate we should look at
>>it in step 3. If the user never touched a field, we can very quickly
>>(regardless of its size) know that it has not been modified without
>>comparing the actual contents.
>>
>>        Using a BitSet this would be very cheap to do as well. What do you think?
>>
>>Gili
>>
>>Mike Kienenberger wrote:
>>
>>>Yep!
>>>
>>>On 9/1/05, Gili <[hidden email]> wrote:
>>>
>>>
>>>>       And I forgot to mention, in step 3 I assume we look at the return value
>>>
>>>>from the DB and if we expected 1 change and got 0 this means we detect
>>>
>>>>that our DB representation was out of date and we throw an exception,
>>>>correct?
>>>>
>>>>Gili
>>>>
>>>>Mike Kienenberger wrote:
>>>>
>>>>
>>>>>Yeah, it's basically an atomic db operation that says UPDATE set
>>>>>values WHERE all fields marked for optimistic locking haven't changed
>>>>>values from the last time we read them.
>>>>>
>>>>>On 9/1/05, Gili <[hidden email]> wrote:
>>>>>
>>>>>
>>>>>
>>>>>>      Oh my. It all makes so much more sense now... So if I understand it
>>>>>>correctly,
>>>>>>
>>>>>>1) We store the perceived DB value somewhere
>>>>>>2) We store the cached (maybe modified) value elsewhere
>>>>>>3) When a commit occurs, we compare the objects in 1 and 2, then issue a
>>>>>>UPDATE only for fields which have changed.
>>>>>>
>>>>>>      Cool :) This also sounds quite efficient to me.
>>>>>>
>>>>>>Thank you,
>>>>>>Gili
>>>>>>
>>>>>>Gentry, Michael (Contractor) wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>>Optimistic locking never locks the row in the database (it is
>>>>>>>optimistic).  Read:
>>>>>>>
>>>>>>>http://www.objectstyle.org/confluence/display/CAY/Optimistic+Locking+Exp
>>>>>>>lained
>>>>>>>
>>>>>>>It explains how Cayenne can ensure that no changes occurred between the
>>>>>>>SELECT and UPDATE phase.  If you still have questions I'll try to answer
>>>>>>>them.
>>>>>>>
>>>>>>>Thanks,
>>>>>>>
>>>>>>>/dev/mrg
>>>>>>>
>>>>>>>
>>>>>>>-----Original Message-----
>>>>>>>From: Gili [mailto:[hidden email]]
>>>>>>>Sent: Thursday, September 01, 2005 12:06 AM
>>>>>>>To: [hidden email]
>>>>>>>Subject: Yet another optimistic locking question
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>    A question about how optimistic locking is currently
>>>>>>>implemented. Do we
>>>>>>>implement it like this?
>>>>>>>
>>>>>>>1) Lock row
>>>>>>>2) Read row
>>>>>>>3) Compare read row to DataObject version of row
>>>>>>>4) If values mismatch, unlock the row and throw an exception
>>>>>>>5) If values match, continue with update and unlock row
>>>>>>>
>>>>>>>    or do we not lock the database at all? If we don't lock it at
>>>>>>>all, how
>>>>>>>can we ensure that no changes occur after step 3 but before step 5?
>>>>>>>
>>>>>>>Thank you,
>>>>>>>Gili
>>>>>>
>>>>>>--
>>>>>>http://www.desktopbeautifier.com/
>>>>>>
>>>>>
>>>>>
>>>>--
>>>>http://www.desktopbeautifier.com/
>>>>
>>>
>>>
>>--
>>http://www.desktopbeautifier.com/
>>
>
>

--
http://www.desktopbeautifier.com/
Reply | Threaded
Open this post in threaded view
|

Re: Yet another optimistic locking question

Mike Kienenberger
In reply to this post by Gentry, Michael (Contractor)
Ok.  I see.
Wouldn't you still want to also do the compare after the mask filtered
out things since it may be a case of multiple updates that leave the
data matching the original state?

On 9/1/05, Gentry, Michael (Contractor) <[hidden email]> wrote:

> I believe he is talking about the SET clause, not WHERE clause, of an
> UPDATE statement (we've veered off optimistic locking).
>
> Cayenne does indeed do comparisons to determine what to include in the
> SET clause.  It's been a few months since I looked at it, but I think it
> brute-force compares every single attribute, so it is possible some kind
> of mask to exclude things that never had set* called on them could be
> useful.  Of course, in a web application where you might have your
> object bound to fields in the GUI, set* would be called all the time,
> even if nothing changed.
>
> I think this is worth discussing, but it might end up being a wash for
> most things.  For most objects, doing the comparisons isn't terribly
> time consuming.  Of course, for a large DataContext, this could slow
> things down, too.
>
> /dev/mrg
>
>
> -----Original Message-----
> From: Mike Kienenberger [mailto:[hidden email]]
> Sent: Thursday, September 01, 2005 11:34 AM
> To: [hidden email]
> Subject: Re: Yet another optimistic locking question
>
>
> No, that doesn't work.  The "checking" part is executed as part of the
> database operation.
> The database "checks" if the value has changed as part of the update
> statement, not the java code.   We supply the original values as part
> of the query, and the database does the comparison.  Optimistic
> locking in general isn't specific to cayenne so the process is well
> understood and probably as optimized as it can be.  Optimizations to
> the concept are the timestamp and versioning alternatives of
> optimistic locking where you only lock on a timestamp (assumes that
> any database operation must occur at different timestamps) or versions
> (which forces the caller to maintain versioning).   The downsides of
> these optimizations are that they take up extra database space (on
> field per table) and that they consider "differences that make no
> difference" as a difference.
>
> Ie, attribute locking works even if, in the mean time, someone changed
> a field value then changed it back.   But versioning/timestamping will
> fail even if the current state is the same as the original perceived
> state.
>
> The downsides of attribute locking is that it requires more bandwidth
> (multiple where clauses transmitted) and processing on the database
> (multiple where clauses computed)
>
> On 9/1/05, Gili <[hidden email]> wrote:
> >
> >         Here is an idea for us to further optimize the process. Can we
> perhaps
> > detect whether the user ever modified a field without comparing the
> two
> > states? For example, if one of my fields is a large BLOB (byte[]) then
> > when I get() that array I could concievable modify it. So then what
> I'm
> > thinking is if the user ever invoked get() or set() on that field, we
> > toggle the appropriate value in a BitSet to indicate we should look at
> > it in step 3. If the user never touched a field, we can very quickly
> > (regardless of its size) know that it has not been modified without
> > comparing the actual contents.
> >
> >         Using a BitSet this would be very cheap to do as well. What do
> you think?
> >
> > Gili
> >
> > Mike Kienenberger wrote:
> > > Yep!
> > >
> > > On 9/1/05, Gili <[hidden email]> wrote:
> > >
> > >>        And I forgot to mention, in step 3 I assume we look at the
> return value
> > >>from the DB and if we expected 1 change and got 0 this means we
> detect
> > >>that our DB representation was out of date and we throw an
> exception,
> > >>correct?
> > >>
> > >>Gili
> > >>
> > >>Mike Kienenberger wrote:
> > >>
> > >>>Yeah, it's basically an atomic db operation that says UPDATE set
> > >>>values WHERE all fields marked for optimistic locking haven't
> changed
> > >>>values from the last time we read them.
> > >>>
> > >>>On 9/1/05, Gili <[hidden email]> wrote:
> > >>>
> > >>>
> > >>>>       Oh my. It all makes so much more sense now... So if I
> understand it
> > >>>>correctly,
> > >>>>
> > >>>>1) We store the perceived DB value somewhere
> > >>>>2) We store the cached (maybe modified) value elsewhere
> > >>>>3) When a commit occurs, we compare the objects in 1 and 2, then
> issue a
> > >>>>UPDATE only for fields which have changed.
> > >>>>
> > >>>>       Cool :) This also sounds quite efficient to me.
> > >>>>
> > >>>>Thank you,
> > >>>>Gili
> > >>>>
> > >>>>Gentry, Michael (Contractor) wrote:
> > >>>>
> > >>>>
> > >>>>>Optimistic locking never locks the row in the database (it is
> > >>>>>optimistic).  Read:
> > >>>>>
> >
> >>>>>http://www.objectstyle.org/confluence/display/CAY/Optimistic+Lockin
> g+Exp
> > >>>>>lained
> > >>>>>
> > >>>>>It explains how Cayenne can ensure that no changes occurred
> between the
> > >>>>>SELECT and UPDATE phase.  If you still have questions I'll try to
> answer
> > >>>>>them.
> > >>>>>
> > >>>>>Thanks,
> > >>>>>
> > >>>>>/dev/mrg
> > >>>>>
> > >>>>>
> > >>>>>-----Original Message-----
> > >>>>>From: Gili [mailto:[hidden email]]
> > >>>>>Sent: Thursday, September 01, 2005 12:06 AM
> > >>>>>To: [hidden email]
> > >>>>>Subject: Yet another optimistic locking question
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>     A question about how optimistic locking is currently
> > >>>>>implemented. Do we
> > >>>>>implement it like this?
> > >>>>>
> > >>>>>1) Lock row
> > >>>>>2) Read row
> > >>>>>3) Compare read row to DataObject version of row
> > >>>>>4) If values mismatch, unlock the row and throw an exception
> > >>>>>5) If values match, continue with update and unlock row
> > >>>>>
> > >>>>>     or do we not lock the database at all? If we don't lock it
> at
> > >>>>>all, how
> > >>>>>can we ensure that no changes occur after step 3 but before step
> 5?
> > >>>>>
> > >>>>>Thank you,
> > >>>>>Gili
> > >>>>
> > >>>>--
> > >>>>http://www.desktopbeautifier.com/
> > >>>>
> > >>>
> > >>>
> > >>--
> > >>http://www.desktopbeautifier.com/
> > >>
> > >
> > >
> >
> > --
> > http://www.desktopbeautifier.com/
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Yet another optimistic locking question

Gili
In reply to this post by Gentry, Michael (Contractor)

        Right, that's what I'm talking about.

        I think that even in a GUI domain this will help. Users aren't going to
necessarily "touch" every single field. I think/hope that, like any
other form of caching, this mechanism's benefits will always be greater
than its overhead. It's a BitSet after all... Very fast, very light. :)

Gili

Gentry, Michael (Contractor) wrote:

> I believe he is talking about the SET clause, not WHERE clause, of an
> UPDATE statement (we've veered off optimistic locking).
>
> Cayenne does indeed do comparisons to determine what to include in the
> SET clause.  It's been a few months since I looked at it, but I think it
> brute-force compares every single attribute, so it is possible some kind
> of mask to exclude things that never had set* called on them could be
> useful.  Of course, in a web application where you might have your
> object bound to fields in the GUI, set* would be called all the time,
> even if nothing changed.
>
> I think this is worth discussing, but it might end up being a wash for
> most things.  For most objects, doing the comparisons isn't terribly
> time consuming.  Of course, for a large DataContext, this could slow
> things down, too.
>
> /dev/mrg
>
>
> -----Original Message-----
> From: Mike Kienenberger [mailto:[hidden email]]
> Sent: Thursday, September 01, 2005 11:34 AM
> To: [hidden email]
> Subject: Re: Yet another optimistic locking question
>
>
> No, that doesn't work.  The "checking" part is executed as part of the
> database operation.
> The database "checks" if the value has changed as part of the update
> statement, not the java code.   We supply the original values as part
> of the query, and the database does the comparison.  Optimistic
> locking in general isn't specific to cayenne so the process is well
> understood and probably as optimized as it can be.  Optimizations to
> the concept are the timestamp and versioning alternatives of
> optimistic locking where you only lock on a timestamp (assumes that
> any database operation must occur at different timestamps) or versions
> (which forces the caller to maintain versioning).   The downsides of
> these optimizations are that they take up extra database space (on
> field per table) and that they consider "differences that make no
> difference" as a difference.
>
> Ie, attribute locking works even if, in the mean time, someone changed
> a field value then changed it back.   But versioning/timestamping will
> fail even if the current state is the same as the original perceived
> state.
>
> The downsides of attribute locking is that it requires more bandwidth
> (multiple where clauses transmitted) and processing on the database
> (multiple where clauses computed)
>
> On 9/1/05, Gili <[hidden email]> wrote:
>
>>        Here is an idea for us to further optimize the process. Can we
>
> perhaps
>
>>detect whether the user ever modified a field without comparing the
>
> two
>
>>states? For example, if one of my fields is a large BLOB (byte[]) then
>>when I get() that array I could concievable modify it. So then what
>
> I'm
>
>>thinking is if the user ever invoked get() or set() on that field, we
>>toggle the appropriate value in a BitSet to indicate we should look at
>>it in step 3. If the user never touched a field, we can very quickly
>>(regardless of its size) know that it has not been modified without
>>comparing the actual contents.
>>
>>        Using a BitSet this would be very cheap to do as well. What do
>
> you think?
>
>>Gili
>>
>>Mike Kienenberger wrote:
>>
>>>Yep!
>>>
>>>On 9/1/05, Gili <[hidden email]> wrote:
>>>
>>>
>>>>       And I forgot to mention, in step 3 I assume we look at the
>
> return value
>
>>>>from the DB and if we expected 1 change and got 0 this means we
>
> detect
>
>>>>that our DB representation was out of date and we throw an
>
> exception,
>
>>>>correct?
>>>>
>>>>Gili
>>>>
>>>>Mike Kienenberger wrote:
>>>>
>>>>
>>>>>Yeah, it's basically an atomic db operation that says UPDATE set
>>>>>values WHERE all fields marked for optimistic locking haven't
>
> changed
>
>>>>>values from the last time we read them.
>>>>>
>>>>>On 9/1/05, Gili <[hidden email]> wrote:
>>>>>
>>>>>
>>>>>
>>>>>>      Oh my. It all makes so much more sense now... So if I
>
> understand it
>
>>>>>>correctly,
>>>>>>
>>>>>>1) We store the perceived DB value somewhere
>>>>>>2) We store the cached (maybe modified) value elsewhere
>>>>>>3) When a commit occurs, we compare the objects in 1 and 2, then
>
> issue a
>
>>>>>>UPDATE only for fields which have changed.
>>>>>>
>>>>>>      Cool :) This also sounds quite efficient to me.
>>>>>>
>>>>>>Thank you,
>>>>>>Gili
>>>>>>
>>>>>>Gentry, Michael (Contractor) wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>>Optimistic locking never locks the row in the database (it is
>>>>>>>optimistic).  Read:
>>>>>>>
>>
>>>>>>http://www.objectstyle.org/confluence/display/CAY/Optimistic+Lockin
>
> g+Exp
>
>>>>>>>lained
>>>>>>>
>>>>>>>It explains how Cayenne can ensure that no changes occurred
>
> between the
>
>>>>>>>SELECT and UPDATE phase.  If you still have questions I'll try to
>
> answer
>
>>>>>>>them.
>>>>>>>
>>>>>>>Thanks,
>>>>>>>
>>>>>>>/dev/mrg
>>>>>>>
>>>>>>>
>>>>>>>-----Original Message-----
>>>>>>>From: Gili [mailto:[hidden email]]
>>>>>>>Sent: Thursday, September 01, 2005 12:06 AM
>>>>>>>To: [hidden email]
>>>>>>>Subject: Yet another optimistic locking question
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>    A question about how optimistic locking is currently
>>>>>>>implemented. Do we
>>>>>>>implement it like this?
>>>>>>>
>>>>>>>1) Lock row
>>>>>>>2) Read row
>>>>>>>3) Compare read row to DataObject version of row
>>>>>>>4) If values mismatch, unlock the row and throw an exception
>>>>>>>5) If values match, continue with update and unlock row
>>>>>>>
>>>>>>>    or do we not lock the database at all? If we don't lock it
>
> at
>
>>>>>>>all, how
>>>>>>>can we ensure that no changes occur after step 3 but before step
>
> 5?
>
>>>>>>>Thank you,
>>>>>>>Gili
>>>>>>
>>>>>>--
>>>>>>http://www.desktopbeautifier.com/
>>>>>>
>>>>>
>>>>>
>>>>--
>>>>http://www.desktopbeautifier.com/
>>>>
>>>
>>>
>>--
>>http://www.desktopbeautifier.com/
>>
>
>

--
http://www.desktopbeautifier.com/
Reply | Threaded
Open this post in threaded view
|

Re: Yet another optimistic locking question

Gili
In reply to this post by Mike Kienenberger

        Yes. You would. The mask is meant to act like a hash of sorts. It'll
tell you if something was not touched but just because something was
touched doesn't mean it was modified :)

        Ok, I'm off for lunch. I'll rejoin this conversation when I get back :)

Gili

Mike Kienenberger wrote:

> Ok.  I see.
> Wouldn't you still want to also do the compare after the mask filtered
> out things since it may be a case of multiple updates that leave the
> data matching the original state?
>
> On 9/1/05, Gentry, Michael (Contractor) <[hidden email]> wrote:
>
>>I believe he is talking about the SET clause, not WHERE clause, of an
>>UPDATE statement (we've veered off optimistic locking).
>>
>>Cayenne does indeed do comparisons to determine what to include in the
>>SET clause.  It's been a few months since I looked at it, but I think it
>>brute-force compares every single attribute, so it is possible some kind
>>of mask to exclude things that never had set* called on them could be
>>useful.  Of course, in a web application where you might have your
>>object bound to fields in the GUI, set* would be called all the time,
>>even if nothing changed.
>>
>>I think this is worth discussing, but it might end up being a wash for
>>most things.  For most objects, doing the comparisons isn't terribly
>>time consuming.  Of course, for a large DataContext, this could slow
>>things down, too.
>>
>>/dev/mrg
>>
>>
>>-----Original Message-----
>>From: Mike Kienenberger [mailto:[hidden email]]
>>Sent: Thursday, September 01, 2005 11:34 AM
>>To: [hidden email]
>>Subject: Re: Yet another optimistic locking question
>>
>>
>>No, that doesn't work.  The "checking" part is executed as part of the
>>database operation.
>>The database "checks" if the value has changed as part of the update
>>statement, not the java code.   We supply the original values as part
>>of the query, and the database does the comparison.  Optimistic
>>locking in general isn't specific to cayenne so the process is well
>>understood and probably as optimized as it can be.  Optimizations to
>>the concept are the timestamp and versioning alternatives of
>>optimistic locking where you only lock on a timestamp (assumes that
>>any database operation must occur at different timestamps) or versions
>>(which forces the caller to maintain versioning).   The downsides of
>>these optimizations are that they take up extra database space (on
>>field per table) and that they consider "differences that make no
>>difference" as a difference.
>>
>>Ie, attribute locking works even if, in the mean time, someone changed
>>a field value then changed it back.   But versioning/timestamping will
>>fail even if the current state is the same as the original perceived
>>state.
>>
>>The downsides of attribute locking is that it requires more bandwidth
>>(multiple where clauses transmitted) and processing on the database
>>(multiple where clauses computed)
>>
>>On 9/1/05, Gili <[hidden email]> wrote:
>>
>>>        Here is an idea for us to further optimize the process. Can we
>>
>>perhaps
>>
>>>detect whether the user ever modified a field without comparing the
>>
>>two
>>
>>>states? For example, if one of my fields is a large BLOB (byte[]) then
>>>when I get() that array I could concievable modify it. So then what
>>
>>I'm
>>
>>>thinking is if the user ever invoked get() or set() on that field, we
>>>toggle the appropriate value in a BitSet to indicate we should look at
>>>it in step 3. If the user never touched a field, we can very quickly
>>>(regardless of its size) know that it has not been modified without
>>>comparing the actual contents.
>>>
>>>        Using a BitSet this would be very cheap to do as well. What do
>>
>>you think?
>>
>>>Gili
>>>
>>>Mike Kienenberger wrote:
>>>
>>>>Yep!
>>>>
>>>>On 9/1/05, Gili <[hidden email]> wrote:
>>>>
>>>>
>>>>>       And I forgot to mention, in step 3 I assume we look at the
>>
>>return value
>>
>>>>>from the DB and if we expected 1 change and got 0 this means we
>>
>>detect
>>
>>>>>that our DB representation was out of date and we throw an
>>
>>exception,
>>
>>>>>correct?
>>>>>
>>>>>Gili
>>>>>
>>>>>Mike Kienenberger wrote:
>>>>>
>>>>>
>>>>>>Yeah, it's basically an atomic db operation that says UPDATE set
>>>>>>values WHERE all fields marked for optimistic locking haven't
>>
>>changed
>>
>>>>>>values from the last time we read them.
>>>>>>
>>>>>>On 9/1/05, Gili <[hidden email]> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>>      Oh my. It all makes so much more sense now... So if I
>>
>>understand it
>>
>>>>>>>correctly,
>>>>>>>
>>>>>>>1) We store the perceived DB value somewhere
>>>>>>>2) We store the cached (maybe modified) value elsewhere
>>>>>>>3) When a commit occurs, we compare the objects in 1 and 2, then
>>
>>issue a
>>
>>>>>>>UPDATE only for fields which have changed.
>>>>>>>
>>>>>>>      Cool :) This also sounds quite efficient to me.
>>>>>>>
>>>>>>>Thank you,
>>>>>>>Gili
>>>>>>>
>>>>>>>Gentry, Michael (Contractor) wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>Optimistic locking never locks the row in the database (it is
>>>>>>>>optimistic).  Read:
>>>>>>>>
>>>
>>>>>>>http://www.objectstyle.org/confluence/display/CAY/Optimistic+Lockin
>>
>>g+Exp
>>
>>>>>>>>lained
>>>>>>>>
>>>>>>>>It explains how Cayenne can ensure that no changes occurred
>>
>>between the
>>
>>>>>>>>SELECT and UPDATE phase.  If you still have questions I'll try to
>>
>>answer
>>
>>>>>>>>them.
>>>>>>>>
>>>>>>>>Thanks,
>>>>>>>>
>>>>>>>>/dev/mrg
>>>>>>>>
>>>>>>>>
>>>>>>>>-----Original Message-----
>>>>>>>>From: Gili [mailto:[hidden email]]
>>>>>>>>Sent: Thursday, September 01, 2005 12:06 AM
>>>>>>>>To: [hidden email]
>>>>>>>>Subject: Yet another optimistic locking question
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>    A question about how optimistic locking is currently
>>>>>>>>implemented. Do we
>>>>>>>>implement it like this?
>>>>>>>>
>>>>>>>>1) Lock row
>>>>>>>>2) Read row
>>>>>>>>3) Compare read row to DataObject version of row
>>>>>>>>4) If values mismatch, unlock the row and throw an exception
>>>>>>>>5) If values match, continue with update and unlock row
>>>>>>>>
>>>>>>>>    or do we not lock the database at all? If we don't lock it
>>
>>at
>>
>>>>>>>>all, how
>>>>>>>>can we ensure that no changes occur after step 3 but before step
>>
>>5?
>>
>>>>>>>>Thank you,
>>>>>>>>Gili
>>>>>>>
>>>>>>>--
>>>>>>>http://www.desktopbeautifier.com/
>>>>>>>
>>>>>>
>>>>>>
>>>>>--
>>>>>http://www.desktopbeautifier.com/
>>>>>
>>>>
>>>>
>>>--
>>>http://www.desktopbeautifier.com/
>>>
>>
>

--
http://www.desktopbeautifier.com/
Reply | Threaded
Open this post in threaded view
|

RE: Yet another optimistic locking question

Gentry, Michael (Contractor)
In reply to this post by Gili
BLOB optimistic locking will always be slow.  The optimization you are
referring to wouldn't help with the WHERE clause generation, it could
only help in the SET clause, which means if you lock on a BLOB, you have
to send it over to the DB regardless of if it has changed or not.

/dev/mrg


-----Original Message-----
From: Gili [mailto:[hidden email]]
Sent: Thursday, September 01, 2005 11:47 AM
To: [hidden email]
Subject: Re: Yet another optimistic locking question



        I think you misunderstand me.

        I'm saying that in part 3 Cayenne compares its internal
representation
of 1 and 2 and it gathers up all fields that the user has changed during

the transaction. Then it generates a "UPDATE SET" statement only
consisting of fields the user has changed. Correct?

        All I am saying is that we can optimize comparison on Cayenne's
part
(when comparing part 1 and 2) because you could use a BitSet to know
whether a field has been touched or not without having to compare the
entire field value.

        This even makes BLOB optimistic locking much cheaper because the
only
time this becomes expensive. If we can wrap the BLOB data in some
read-only structure so the getter() returns an unmodifiable array, then
we can quite easily detect whether a blob has been changed or not
without comparing its entire value.

        We still have send the changed data to the DB and granted this
is slow,
but I'm saying we can optimize the comparison that occurs before that.
Did I miss something? :)

Gili

Mike Kienenberger wrote:

> No, that doesn't work.  The "checking" part is executed as part of the
> database operation.
> The database "checks" if the value has changed as part of the update
> statement, not the java code.   We supply the original values as part
> of the query, and the database does the comparison.  Optimistic
> locking in general isn't specific to cayenne so the process is well
> understood and probably as optimized as it can be.  Optimizations to
> the concept are the timestamp and versioning alternatives of
> optimistic locking where you only lock on a timestamp (assumes that
> any database operation must occur at different timestamps) or versions
> (which forces the caller to maintain versioning).   The downsides of
> these optimizations are that they take up extra database space (on
> field per table) and that they consider "differences that make no
> difference" as a difference.
>
> Ie, attribute locking works even if, in the mean time, someone changed
> a field value then changed it back.   But versioning/timestamping will
> fail even if the current state is the same as the original perceived
> state.
>
> The downsides of attribute locking is that it requires more bandwidth
> (multiple where clauses transmitted) and processing on the database
> (multiple where clauses computed)
>
> On 9/1/05, Gili <[hidden email]> wrote:
>
>>        Here is an idea for us to further optimize the process. Can we
perhaps
>>detect whether the user ever modified a field without comparing the
two
>>states? For example, if one of my fields is a large BLOB (byte[]) then
>>when I get() that array I could concievable modify it. So then what
I'm
>>thinking is if the user ever invoked get() or set() on that field, we
>>toggle the appropriate value in a BitSet to indicate we should look at
>>it in step 3. If the user never touched a field, we can very quickly
>>(regardless of its size) know that it has not been modified without
>>comparing the actual contents.
>>
>>        Using a BitSet this would be very cheap to do as well. What do
you think?

>>
>>Gili
>>
>>Mike Kienenberger wrote:
>>
>>>Yep!
>>>
>>>On 9/1/05, Gili <[hidden email]> wrote:
>>>
>>>
>>>>       And I forgot to mention, in step 3 I assume we look at the
return value
>>>
>>>>from the DB and if we expected 1 change and got 0 this means we
detect
>>>
>>>>that our DB representation was out of date and we throw an
exception,
>>>>correct?
>>>>
>>>>Gili
>>>>
>>>>Mike Kienenberger wrote:
>>>>
>>>>
>>>>>Yeah, it's basically an atomic db operation that says UPDATE set
>>>>>values WHERE all fields marked for optimistic locking haven't
changed
>>>>>values from the last time we read them.
>>>>>
>>>>>On 9/1/05, Gili <[hidden email]> wrote:
>>>>>
>>>>>
>>>>>
>>>>>>      Oh my. It all makes so much more sense now... So if I
understand it
>>>>>>correctly,
>>>>>>
>>>>>>1) We store the perceived DB value somewhere
>>>>>>2) We store the cached (maybe modified) value elsewhere
>>>>>>3) When a commit occurs, we compare the objects in 1 and 2, then
issue a

>>>>>>UPDATE only for fields which have changed.
>>>>>>
>>>>>>      Cool :) This also sounds quite efficient to me.
>>>>>>
>>>>>>Thank you,
>>>>>>Gili
>>>>>>
>>>>>>Gentry, Michael (Contractor) wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>>Optimistic locking never locks the row in the database (it is
>>>>>>>optimistic).  Read:
>>>>>>>
>>>>>>>http://www.objectstyle.org/confluence/display/CAY/Optimistic+Lock
ing+Exp
>>>>>>>lained
>>>>>>>
>>>>>>>It explains how Cayenne can ensure that no changes occurred
between the
>>>>>>>SELECT and UPDATE phase.  If you still have questions I'll try to
answer

>>>>>>>them.
>>>>>>>
>>>>>>>Thanks,
>>>>>>>
>>>>>>>/dev/mrg
>>>>>>>
>>>>>>>
>>>>>>>-----Original Message-----
>>>>>>>From: Gili [mailto:[hidden email]]
>>>>>>>Sent: Thursday, September 01, 2005 12:06 AM
>>>>>>>To: [hidden email]
>>>>>>>Subject: Yet another optimistic locking question
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>    A question about how optimistic locking is currently
>>>>>>>implemented. Do we
>>>>>>>implement it like this?
>>>>>>>
>>>>>>>1) Lock row
>>>>>>>2) Read row
>>>>>>>3) Compare read row to DataObject version of row
>>>>>>>4) If values mismatch, unlock the row and throw an exception
>>>>>>>5) If values match, continue with update and unlock row
>>>>>>>
>>>>>>>    or do we not lock the database at all? If we don't lock it at
>>>>>>>all, how
>>>>>>>can we ensure that no changes occur after step 3 but before step
5?

>>>>>>>
>>>>>>>Thank you,
>>>>>>>Gili
>>>>>>
>>>>>>--
>>>>>>http://www.desktopbeautifier.com/
>>>>>>
>>>>>
>>>>>
>>>>--
>>>>http://www.desktopbeautifier.com/
>>>>
>>>
>>>
>>--
>>http://www.desktopbeautifier.com/
>>
>
>

--
http://www.desktopbeautifier.com/
Reply | Threaded
Open this post in threaded view
|

RE: Yet another optimistic locking question

Gentry, Michael (Contractor)
In reply to this post by Gili
This is why there is a thread just about every week on the Tapestry list
about Hibernate + Lazy Initialization (or whatever -- their version of a
fault) exceptions.  Hibernate doesn't seem to work too well
out-of-the-box in a web-based application.  That second request/response
loop can be a killer.

/dev/mrg


-----Original Message-----
From: Mike Kienenberger [mailto:[hidden email]]
Sent: Thursday, September 01, 2005 11:46 AM
To: [hidden email]
Subject: Re: Yet another optimistic locking question


On 9/1/05, Gentry, Michael (Contractor) <[hidden email]>
wrote:
> Step 2: Yes, Cayenne caches the original DB values.  This is how
Cayenne
> can determine what has changed (if anything).  I don't believe
Hibernate
> has this feature, but I could be wrong.

I think Hibernate has this but only so long as your DataObject is part
of a hibernate Session.
Once you disconnect the object, then you lose the information.

If I am remembering correctly, Gili pointed out to me earlier that a
Session maintains both the backing cache and the database connection
and a Transaction maintains modifications.   Under Cayenne, we
maintain the connections separately, but combine the caching and
transactions in our data context.
Reply | Threaded
Open this post in threaded view
|

Re: Yet another optimistic locking question

Gili
In reply to this post by Gentry, Michael (Contractor)

        Right, I just discussed this with Mike in IRC and I get it now. That's
really too bad :( Still, I think there is value to adding the BitSet I
mentioned. We'll have to try it out to see the performance impact.

Gili

Gentry, Michael (Contractor) wrote:

> BLOB optimistic locking will always be slow.  The optimization you are
> referring to wouldn't help with the WHERE clause generation, it could
> only help in the SET clause, which means if you lock on a BLOB, you have
> to send it over to the DB regardless of if it has changed or not.
>
> /dev/mrg
>
>
> -----Original Message-----
> From: Gili [mailto:[hidden email]]
> Sent: Thursday, September 01, 2005 11:47 AM
> To: [hidden email]
> Subject: Re: Yet another optimistic locking question
>
>
>
> I think you misunderstand me.
>
> I'm saying that in part 3 Cayenne compares its internal
> representation
> of 1 and 2 and it gathers up all fields that the user has changed during
>
> the transaction. Then it generates a "UPDATE SET" statement only
> consisting of fields the user has changed. Correct?
>
> All I am saying is that we can optimize comparison on Cayenne's
> part
> (when comparing part 1 and 2) because you could use a BitSet to know
> whether a field has been touched or not without having to compare the
> entire field value.
>
> This even makes BLOB optimistic locking much cheaper because the
> only
> time this becomes expensive. If we can wrap the BLOB data in some
> read-only structure so the getter() returns an unmodifiable array, then
> we can quite easily detect whether a blob has been changed or not
> without comparing its entire value.
>
> We still have send the changed data to the DB and granted this
> is slow,
> but I'm saying we can optimize the comparison that occurs before that.
> Did I miss something? :)
>
> Gili
>
> Mike Kienenberger wrote:
>
>>No, that doesn't work.  The "checking" part is executed as part of the
>>database operation.
>>The database "checks" if the value has changed as part of the update
>>statement, not the java code.   We supply the original values as part
>>of the query, and the database does the comparison.  Optimistic
>>locking in general isn't specific to cayenne so the process is well
>>understood and probably as optimized as it can be.  Optimizations to
>>the concept are the timestamp and versioning alternatives of
>>optimistic locking where you only lock on a timestamp (assumes that
>>any database operation must occur at different timestamps) or versions
>>(which forces the caller to maintain versioning).   The downsides of
>>these optimizations are that they take up extra database space (on
>>field per table) and that they consider "differences that make no
>>difference" as a difference.
>>
>>Ie, attribute locking works even if, in the mean time, someone changed
>>a field value then changed it back.   But versioning/timestamping will
>>fail even if the current state is the same as the original perceived
>>state.
>>
>>The downsides of attribute locking is that it requires more bandwidth
>>(multiple where clauses transmitted) and processing on the database
>>(multiple where clauses computed)
>>
>>On 9/1/05, Gili <[hidden email]> wrote:
>>
>>
>>>       Here is an idea for us to further optimize the process. Can we
>
> perhaps
>
>>>detect whether the user ever modified a field without comparing the
>
> two
>
>>>states? For example, if one of my fields is a large BLOB (byte[]) then
>>>when I get() that array I could concievable modify it. So then what
>
> I'm
>
>>>thinking is if the user ever invoked get() or set() on that field, we
>>>toggle the appropriate value in a BitSet to indicate we should look at
>>>it in step 3. If the user never touched a field, we can very quickly
>>>(regardless of its size) know that it has not been modified without
>>>comparing the actual contents.
>>>
>>>       Using a BitSet this would be very cheap to do as well. What do
>
> you think?
>
>>>Gili
>>>
>>>Mike Kienenberger wrote:
>>>
>>>
>>>>Yep!
>>>>
>>>>On 9/1/05, Gili <[hidden email]> wrote:
>>>>
>>>>
>>>>
>>>>>      And I forgot to mention, in step 3 I assume we look at the
>
> return value
>
>>>>>from the DB and if we expected 1 change and got 0 this means we
>
> detect
>
>>>>>that our DB representation was out of date and we throw an
>
> exception,
>
>>>>>correct?
>>>>>
>>>>>Gili
>>>>>
>>>>>Mike Kienenberger wrote:
>>>>>
>>>>>
>>>>>
>>>>>>Yeah, it's basically an atomic db operation that says UPDATE set
>>>>>>values WHERE all fields marked for optimistic locking haven't
>
> changed
>
>>>>>>values from the last time we read them.
>>>>>>
>>>>>>On 9/1/05, Gili <[hidden email]> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>     Oh my. It all makes so much more sense now... So if I
>
> understand it
>
>>>>>>>correctly,
>>>>>>>
>>>>>>>1) We store the perceived DB value somewhere
>>>>>>>2) We store the cached (maybe modified) value elsewhere
>>>>>>>3) When a commit occurs, we compare the objects in 1 and 2, then
>
> issue a
>
>>>>>>>UPDATE only for fields which have changed.
>>>>>>>
>>>>>>>     Cool :) This also sounds quite efficient to me.
>>>>>>>
>>>>>>>Thank you,
>>>>>>>Gili
>>>>>>>
>>>>>>>Gentry, Michael (Contractor) wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>Optimistic locking never locks the row in the database (it is
>>>>>>>>optimistic).  Read:
>>>>>>>>
>>>>>>>>http://www.objectstyle.org/confluence/display/CAY/Optimistic+Lock
>
> ing+Exp
>
>>>>>>>>lained
>>>>>>>>
>>>>>>>>It explains how Cayenne can ensure that no changes occurred
>
> between the
>
>>>>>>>>SELECT and UPDATE phase.  If you still have questions I'll try to
>
> answer
>
>>>>>>>>them.
>>>>>>>>
>>>>>>>>Thanks,
>>>>>>>>
>>>>>>>>/dev/mrg
>>>>>>>>
>>>>>>>>
>>>>>>>>-----Original Message-----
>>>>>>>>From: Gili [mailto:[hidden email]]
>>>>>>>>Sent: Thursday, September 01, 2005 12:06 AM
>>>>>>>>To: [hidden email]
>>>>>>>>Subject: Yet another optimistic locking question
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>   A question about how optimistic locking is currently
>>>>>>>>implemented. Do we
>>>>>>>>implement it like this?
>>>>>>>>
>>>>>>>>1) Lock row
>>>>>>>>2) Read row
>>>>>>>>3) Compare read row to DataObject version of row
>>>>>>>>4) If values mismatch, unlock the row and throw an exception
>>>>>>>>5) If values match, continue with update and unlock row
>>>>>>>>
>>>>>>>>   or do we not lock the database at all? If we don't lock it at
>>>>>>>>all, how
>>>>>>>>can we ensure that no changes occur after step 3 but before step
>
> 5?
>
>>>>>>>>Thank you,
>>>>>>>>Gili
>>>>>>>
>>>>>>>--
>>>>>>>http://www.desktopbeautifier.com/
>>>>>>>
>>>>>>
>>>>>>
>>>>>--
>>>>>http://www.desktopbeautifier.com/
>>>>>
>>>>
>>>>
>>>--
>>>http://www.desktopbeautifier.com/
>>>
>>
>>
>

--
http://www.desktopbeautifier.com/