[PEAK] memory leaks & stuff

Yaroslav Samchuk yarcat at ank-sia.com
Tue Sep 14 13:52:50 EDT 2004


Hello,

On the last week we had discovered, that the application, which is
developed by our team was leaking memory. The problem still is not
solved, but today I saw some interesting stuff - a very simple TCP
server (attached to the mail), which used an OOB data to ping clients
was logging exceptions in case of gc.set_debug(gc.DEBUG_LEAK). At the
startup I had set that GC's debug option and periodically call
gc.collect(). In this case we have a log like:
======
gc: collectable <cell 008FD210>
...
gc: collectable <tuple 007CF148>
log[1396]: Waiting for connection (127.0.0.1:5678)
log[1396]: Connection from 127.0.0.1:3141 accepted
log[1396]: Send PING message
gc: collectable <tuple 00BE4058>
...
gc: collectable <tuple 00BEB590>
peak.events.loop[1396]: Unexpected error in event loop:
peak.events.loop[1396]: Traceback (most recent call last):
peak.events.loop[1396]:   File "E:\Python23\Lib\site-packages\peak\events\io_events.py", line 138, in tick
peak.events.loop[1396]:     doTick(exit)
peak.events.loop[1396]:   File "E:\Python23\Lib\site-packages\peak\events\event_threads.py", line 225, in tick
peak.events.loop[1396]:     self._appointments.pop(0)[1](self,now)
peak.events.loop[1396]:   File "E:\Python23\Lib\site-packages\peak\events\event_threads.py", line 283, in <lambda>
peak.events.loop[1396]:     lambda s,e: func(self,e), self.scheduler.now() + self.delay
peak.events.loop[1396]:   File "E:\Python23\Lib\site-packages\peak\running\daemons.py", line 116, in <lambda>
peak.events.loop[1396]:     lambda src,evt,task=task: self.addTask(task)
peak.events.loop[1396]:   File "E:\Python23\Lib\site-packages\peak\running\daemons.py", line 60, in addTask
peak.events.loop[1396]:     self._taskCount.put()
peak.events.loop[1396]:   File "E:\Python23\Lib\site-packages\peak\events\sources.py", line 805, in put
peak.events.loop[1396]:     self.set(self()+1)
peak.events.loop[1396]:   File "E:\Python23\Lib\site-packages\peak\events\sources.py", line 531, in set
peak.events.loop[1396]:     self.value.set(value,force)
peak.events.loop[1396]:   File "E:\Python23\Lib\site-packages\peak\events\sources.py", line 404, in set
peak.events.loop[1396]:     self._fire(value)
peak.events.loop[1396]:   File "E:\Python23\Lib\site-packages\peak\events\sources.py", line 279, in _fire
peak.events.loop[1396]:     callbacks.pop(0)[0](self,event)
peak.events.loop[1396]:   File "E:\Python23\Lib\site-packages\peak\events\sources.py", line 201, in onceOnly
peak.events.loop[1396]:     return func(self, (source,event))
peak.events.loop[1396]:   File "E:\Python23\Lib\site-packages\peak\events\sources.py", line 107, in cb
peak.events.loop[1396]:     return f(cbr(),src,evt)
peak.events.loop[1396]:   File "E:\Python23\Lib\site-packages\peak\events\sources.py", line 606, in _set
peak.events.loop[1396]:     self._fire(value)
peak.events.loop[1396]:   File "E:\Python23\Lib\site-packages\peak\events\sources.py", line 279, in _fire
peak.events.loop[1396]:     callbacks.pop(0)[0](self,event)
peak.events.loop[1396]:   File "E:\Python23\Lib\site-packages\peak\events\sources.py", line 107, in cb
peak.events.loop[1396]:     return f(cbr(),src,evt)
peak.events.loop[1396]:   File "E:\Python23\Lib\site-packages\peak\events\sources.py", line 518, in _set
peak.events.loop[1396]:     self._fire(evt)
peak.events.loop[1396]: AttributeError: 'NoneType' object has no attribute '_fire'
log[1396]: terminated
log[1396]: Terminated
gc: collectable <MainLoop 00BE8890>
...
gc: collectable <tuple 00BDC6C0>
======

Both implementations of this server (which are zipped) use the component
named `Alarm`, which is used to periodically call method, which pings the
client, using OOB messages. If client sent nothing within some known
time interval, than connection is broken. There are two implementations
of timeout handling in the attached zip:

1. `Alarm` has a method called `ping` which is called each time, client
sends some data to the server. If `ping` was not called within specified
amount of time, timeout handler is called by `Alarm` (implemented in server.py)

2. `Alarm` is used for periodical pings only, timeout is handled by the
`SocketMAC` instance using self.eventLoop.sleep and calling required
handler, when that event occures (implemented in server_old.py).

If I don't use gc.collect(), then `server_old.py` is working normally, but
`server.py` isn't:
======
gc: collectable <cell 008FD210>
...
gc: collectable <tuple 007CF148>
log[1224]: Waiting for connection (127.0.0.1:5678)
log[1224]: Connection from 127.0.0.1:3146 accepted
log[1224]: Send PING message
log[1224]: Send PING message
log[1224]: Send PING message
log[1224]: No data received from the client within 5 seconds
log[1224]: terminated
peak.events.loop[1224]: Unexpected error in event loop:
peak.events.loop[1224]: Traceback (most recent call last):
peak.events.loop[1224]:   File "E:\Python23\Lib\site-packages\peak\events\io_events.py", line 138, in tick
peak.events.loop[1224]:     doTick(exit)
peak.events.loop[1224]:   File "E:\Python23\Lib\site-packages\peak\events\event_threads.py", line 225, in tick
peak.events.loop[1224]:     self._appointments.pop(0)[1](self,now)
peak.events.loop[1224]:   File "E:\Python23\Lib\site-packages\peak\events\event_threads.py", line 283, in <lambda>
peak.events.loop[1224]:     lambda s,e: func(self,e), self.scheduler.now() + self.delay
peak.events.loop[1224]:   File "E:\Python23\Lib\site-packages\peak\events\event_threads.py", line 443, in step
peak.events.loop[1224]:     return self._uncaughtError()
peak.events.loop[1224]:   File "E:\Python23\Lib\site-packages\peak\events\event_threads.py", line 424, in step
peak.events.loop[1224]:     for event in state.stack[-1]:
peak.events.loop[1224]:   File "E:\Python23\Lib\site-packages\peak\events\io_events.py", line 224, in monitor
peak.events.loop[1224]:     fr,fw,fe = self.select(r.keys(),w.keys(),e.keys(),delay)
peak.events.loop[1224]: error: (10038, 'An operation was attempted on something that is not a socket')
Traceback (most recent call last):
...
StopIteration: Nothing scheduled to execute
gc: collectable <Broadcaster 00BD9E18>
...
gc: collectable <tuple 00BDB3F0>
======

and I can't get rid of this exception - socket should be closed to that
moment, events should be destroyed (I hope so - I call
`self._delBinding` to loose references to event objects).

Actually, I know walkarounds for the second problem (don't call timeout
handler from the external component (e.g.`Alarm`), but handle it internally
using sleep events, what is implemented in `server_old.py`, as I had
mentioned), so that isn't the main problem. The main problem is memory
leaks - I'm still trying to write a small app, which will leak. I know,
that I have about 44 new objects every 2 seconds, which must be GC'ed, but
they aren't; tomorrow I'll try to find, which these objects are (I have a
sense, that it could be PEAK's event objects, but I can mistake). From the
first traceback we see
======
peak.events.loop[1396]:     self._fire(evt)
peak.events.loop[1396]: AttributeError: 'NoneType' object has no attribute '_fire'
=====
that PEAK's event became a `NoneType` object! How could it happen? I'll
repeat, that such a situation was achieved if gc.DEBUG_LEAK (actually,
I didn't try other option) GC's debug option was used and gc.collect()
was called (I called it periodically). I really hope, that fixing of this
problem could help me to solve leaks' problem.

--
Best regards,
Yaroslav
-------------- next part --------------
A non-text attachment was scrubbed...
Name: server.zip
Type: application/x-zip-compressed
Size: 7324 bytes
Desc: not available
Url : http://www.eby-sarna.com/pipermail/peak/attachments/20040914/b0b78c4b/server.bin


More information about the PEAK mailing list