bakwc on master
Bugfix: use == for comparison w… Merge pull request #131 from Cy… (compare)
bakwc on master
Use node.ip instead of host for… Merge pull request #132 from Cy… (compare)
bakwc on master
Refactor syncobj_admin Extract… Address review feedback Introduce UtilityException and … and 1 more (compare)
Hey all, just looking for some pointers getting started with syncobj. First, If I run the example 'kvstorage.py' how can get multiple servers running? I ran 'python kvstorage.py 127.0.0.1:9999 127.0.0.2:9999 127.0.0.3:9999'. Do I have to run a similar command in another terminal with the self address and partner addresses change? When I tried doing that I got an error "OSError: [Errno 49] Can't assign requested address"
Secondly, after running the first command (python kvstorage.py 127.0.0.1:9999 127.0.0.2:9999 127.0.0.3:9999') I tried to do ">>> set name 'matt'" but "get name" returned "None".
Thanks for any pointers. I'm really excited to start using SyncObj for my project if I can just get these little things figured out.
kvstorage.py 127.0.0.1:9999 127.0.0.1:9998 127.0.0.1:9997, in terminal2
kvstorage.py 127.0.0.1:9998 127.0.0.1:9997 127.0.0.1:9999and in terminal3
kvstorage.py 127.0.0.1:9997 127.0.0.1:9999 127.0.0.1:9998
httpServer = HTTPServer(('', httpPort), KVRequestHandler)I'm doing
TCPServer = socketserver.TCPServer((selfHost,int(selfPort)), KVRRequestHandler)If I can get a generic version of this to work I'll gladly make a pull request to add it as en example, but there's also a chance that I'm just approaching this the completely wrong way.
It looks like a lock can only be acquired with the entire cluster is connected, right?
Only when the majority of the cluster is alive
Also, once acquired, does the lock remain with the holder until explicitly released or the holder looses connectivity to the rest of the cluster?
It will remain with the holder until explicitly released or the holder looses connectivity to the rest of the cluster
is does the normal heartbeat process refresh the lock
Exactly, it do it automatically every autoUnlockTime / 4 (autoUnlockTime is specified in ReplLockManager constructor)
Another thing, on the example kvstorage_http.py
./kvstorage_http.py 8080 dump.bin.gz localhost:8081 localhost:8082 localhost:8083
ERROR:root:failed to load full dump
Traceback (most recent call last):
File "/home/quocbao/KhoaLuan/raft/pysyncobj/syncobj.py", line 1295, in loadDumpFile
data = self.serializer.deserialize()
File "/home/quocbao/KhoaLuan/raft/pysyncobj/serializer.py", line 113, in deserialize
File "/home/quocbao/KhoaLuan/raft/pysyncobj/pickle.py", line 54, in load
Regarding locks: I have a replicated object (A) on a cluster of servers that clients will modify. The object has its own object, and that object is a graph (A.graph). Clients will modify attributes in the edges of the graph (A.graph.edge_color := 'blue'). Each edge will have a lock, so clients updating multiple edges will need to acquire all locks before making changes.
The edges of the graph are populated with a method after the object (A) is instantiated. If I want to protect my edges from race conditions at different servers I need assign a lock for each edge as the edge is created.
My question is: Should one lock manager oversee all of the locks, or should there be a lock manager for each lock?
@replicated def changeEdgeColor(self, edgeID = 12345, edgeColor = 'blue'):
Thank you both! I am using replicated sync for all of my graph-changing methods.
My server has multiple replicated classes, so forwarding to the leader is not an option since the leader of the different classes might not be on the same server. Basically, I let the server who caught the request handle responding to it, and make sure that it uses replicated methods for changing anything across the cluster.
The thing is that with the graph object, I want to make updates to multiple edges atomic. For instance if multiple clients are trying to change edges on different paths, and those paths overlap with a common edge, then I only want one client to have access to all of those edges at any time.
@replicated_sync def changeEdgeColor(self, edgeID = 12345, edgeColor = 'blue'): ... return "Success!" @replicated_sync def changeManyEdges(self, list_of_edges = [1, 2, 3] ): sort(list_of_edges) for edge in list_of_edges: self.__LockManager.tryAcquire(edge) for edge in list_of_edges: changeEdgeColor(self, edge, 'blue') for edge in list_of_edge: self.__LockManager.release(edge)
One question here about the lock manager. it seems like
tryAcquire does not require me to create the lock myself. Is that true?
if existingLock is None or existingLock == clientID: self.__locks[lockID] = (clientID, currentTime) return True
Traceback (most recent call last): File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/local/lib/python3.6/dist-packages/gevent/threading.py", line 177, in run super(Thread, self).run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/usr/local/lib/python3.6/dist-packages/pysyncobj/syncobj.py", line 500, in _autoTickThread self._onTick(self.__conf.autoTickPeriod) File "/usr/local/lib/python3.6/dist-packages/pysyncobj/syncobj.py", line 577, in _onTick res = self.__doApplyCommand(entry) File "/usr/local/lib/python3.6/dist-packages/pysyncobj/syncobj.py", line 701, in __doApplyCommand return self._idToMethod[funcID](*args, **kwargs) File "/usr/local/lib/python3.6/dist-packages/pysyncobj/syncobj.py", line 1389, in newFunc return func(self, *args, **kwargs) File "/usr/local/lib/python3.6/dist-packages/pysyncobj/batteries.py", line 419, in prolongate for lockID in self.__locks.keys(): RuntimeError: dictionary changed size during iteration