[PEAK] Re: Trellis-fork
Sergey Schetinin
maluke at gmail.com
Tue Jun 23 09:36:29 EDT 2009
Committed the thread-safety fix.
On 2009-06-23, Sergey Schetinin <maluke at gmail.com> wrote:
> Oh yeah, one more thing. We could try to use deadlock-breaking and
> given that automatic retries are OK with Trellis, it would work. But
> lets just not do that. :)
>
>
> On 2009-06-23, Sergey Schetinin <maluke at gmail.com> wrote:
> > Next thing I want to do it introduce thread safety. There's some code
> > to help do that in Trellis already, but I plan to remove that
> > implement a very simple model. If I am not mistaken, Phillip planned
> > for cells to have locks in their manager attribute, so that
> > ctrl.lock(cell) uses that just like other managers, calling __enter__
> > / __exit__. For locks that acquires and releases them so, given
> > ctrl.lock is called in all appropriate places the access to cells is
> > synchronous.
> >
> > The problem with this is that if different cells have different locks
> > as their managers the deadlocks are very likely. There's no way of
> > telling what cells will be used in a transaction and in what order so
> > two concurrent transactions are very likely to get into a deadlock.
> >
> > If all cells have the same lock as their manager there are no
> > deadlocks, but the whole ctrl.lock() business is pointless as there's
> > only one lock to acquire and release.
> >
> > Another possible case is when there are two or more sets of cells that
> > form disjoint networks. All cells in each of these sets would have the
> > same lock. This way there's a guarantee of no deadlocks, but the only
> > reason to use locks in this case is if there will be concurrent
> > transactions on the same cell network.
> >
> > I think the best tradeoff for now is a GIL-like solution -- just
> > disable all this cell.manager / ctrl.lock stuff and just hold the
> > global lock in ctrl._process. This doesn't support the last case
> > described above, but let's first have an actual use case for that and
> > see how it could be supported then. One simple solution that could
> > work for that is to have a controller per cell group / network with
> > its own lock.
> >
> > Another problem is how cells store their values -- they change them
> > in-place and log that for undo. The problem is that if that cell is
> > accessed from another thread, and from *outside* a transaction the
> > changed value will be seen -- that's not good at all. For example see
> > _ReadValue.get_value implementation:
> >
> > def get_value(self):
> > if ctrl.active:
> > # if atomic, make sure we're locked and consistent
> > used(self)
> > return self._value
> >
> > If not atomic there's no guarantee of getting valid value which isn't
> > easily resolved.
> >
> > One way to fix that is to store changes in a different attribute and
> > read that only if inside transaction. A development of this approach
> > is to let the controller store all of cell values. This also makes way
> > for properly concurrent transactions without locking because the
> > controller can store the consistent data snapshot in one location and
> > let the transactions work on a copy. Each transaction would get its
> > own copy so there's no need to lock at this point. At commit time
> > however the changes need to be merged, and because all of value
> > storage is handled by controller it can verify at this point if other
> > transactions changed any inputs used by this transaction since it
> > started. (For more on this see
> > http://code.google.com/p/trellis-fork/wiki/Transactions ) Reads from
> > outside transactions would just get a value from the latest committed
> > snapshot.
> >
> > The unknown in this approach is how fast is it for single threaded
> > applications. I don't see any reasons for it too especially slow, but
> > it could be a bit slower than current implementation. It might be
> > faster too though. If anyone could work on performance measuring for
> > Trellis I'd really appreciate that.
> >
> > Also, one more thing to mention in regards to thread safety -- even
> > with the GIL-like fix in place the controller has to be thread-local
> > because even while there's only one concurrent transaction possible
> > not all operations on controller are transactional: ctrl.active checks
> > in other threads will happen outside transactions and need to return
> > False.
> >
> >
> > On 2009-06-22, Sergey Schetinin <maluke at gmail.com> wrote:
> > > I've just brought the ForkChanges page up to date.
> > > http://code.google.com/p/trellis-fork/wiki/ForkChanges
> > >
> > > Any feedback on changes (also the "future changes" section) is welcome.
> > >
> > >
> > > Also the "Transaction" page have grown into a quite long overview of
> > > what's similar between Trellis rules and concurrent transactions and
> > > some ideas on how Trellis could evolve. Still incomplete, but about
> > > 2/3 done, I think. If you see any mistakes in terminology or factual
> > > errors, please report.
> > >
> > > http://code.google.com/p/trellis-fork/wiki/Transactions
> > >
> > >
> > >
> > >
> > >
> > > On 2009-06-17, Sergey Schetinin <maluke at gmail.com> wrote:
> > > > On 2009-06-12, Sergey Schetinin <maluke at gmail.com> wrote:
> > > > > This is a thread for trellis-fork updates and discussion. At least it
> > > > > should suffice at first.
> > > > >
> > > > > http://code.google.com/p/trellis-fork/
> > > > > http://pypi.python.org/pypi/Trellis-fork
> > > >
> > > >
> > > > I've already finished quite a lot of work and test finally pass (had
> > > > to make a few edits, see commit log) and the changes to the code
> > > > structure are bigger than I imagined at first, but I'm quite happy
> > > > with the state of things overall.
> > > >
> > > > Anyway, I'll work on it some more before reporting on the progress, so
> > > > if you want to watch the changes as they happen see
> > > > http://code.google.com/p/trellis-fork/updates/list
> > > > or the rss feed: http://code.google.com/feeds/p/trellis-fork/updates/basic
> > > >
> > > > The actual reason for this email is that I want to get some feedback
> > > > on an article I started to write, explaining the use of transactions
> > > > in Trellis, it's work in progress, but it's something I think that can
> > > > help a lot with understanding Trellis.
> > > >
> > > > http://code.google.com/p/trellis-fork/wiki/Transactions
> > > > Does it help?
> > > > Should it be more verbose or the maybe the opposite?
> > > > Any factual mistakes, typos, hints on structure, things to mention etc?
> > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Best Regards,
> > > > Sergey Schetinin
> > > >
> > > > http://s3bk.com/ -- S3 Backup
> > > > http://word-to-html.com/ -- Word to HTML Converter
> > > >
> > >
> > >
> > > --
> > > Best Regards,
> > > Sergey Schetinin
> > >
> > > http://s3bk.com/ -- S3 Backup
> > > http://word-to-html.com/ -- Word to HTML Converter
> > >
> >
> >
> >
> > --
> >
> > Best Regards,
> > Sergey Schetinin
> >
> > http://s3bk.com/ -- S3 Backup
> > http://word-to-html.com/ -- Word to HTML Converter
> >
>
>
>
> --
>
> Best Regards,
> Sergey Schetinin
>
> http://s3bk.com/ -- S3 Backup
> http://word-to-html.com/ -- Word to HTML Converter
>
--
Best Regards,
Sergey Schetinin
http://s3bk.com/ -- S3 Backup
http://word-to-html.com/ -- Word to HTML Converter
More information about the PEAK
mailing list