|
This page provides information related to the reentrancy problem of pyatspi and D-Bus. |
Contents
Overview
Currently, Python applications that present a GUI will hang if they attempt to access themselves via the D-Bus pyatspi implementation. For example, Orca cannot access its own preferences GUI. Another example is when an application processes device events from itself.
There are several different ways to handle this, with the two most viable ideas being:
- Providing a "sub-main" (or recursive) loop that dispatches events while waiting for a reply
- Having pyatspi detect it is calling itself and convert the DBus call into a local direct call.
These are described below.
Sub-main loop that dispatches events while waiting for a reply
The "sub-main" loop is one where pyatspi ends up making its own main loop for each method call. An example of this loop is currently in at-spi2-atk/atk-adaptor/event.c to handle device events:
static DBusMessage *
send_and_allow_reentry (DBusConnection *bus, DBusMessage *message)
{
DBusPendingCall *pending;
DBusMessage *reply = NULL;
if (!dbus_connection_send_with_reply (bus, message, &pending, -1))
{
return NULL;
}
dbus_pending_call_set_notify (pending, set_reply, (void *)&reply, NULL);
while (!reply)
{
if (!dbus_connection_read_write_dispatch (bus, -1)) return NULL;
}
return reply;
}To accomplish this for AT-SPI method calls that an assisitive technology wants to make on an accessible (e.g., querying the accessible for information), we can potentially do the following:
Provide python bindings (either in D-Bus itself or a C-module specific to pyatspi) for the libdbus method dbus_connection_read_write_dispatch.
When an assistive technology wants to call a method on an accessible, all methods are obtained via the at-spi2-atk/pyatspi/base.py:get_dbus_method method. We'd change at-spi2-atk/pyatspi/base.py:get_dbus_method to call a new method that basically is the implementation of at-spi2-atk/atk-adaptor/event.c in Python. Note that this requires dbus_pending_call_set_notify as well.
NOTE: Simon McVittie suggests we not do this.
Note also that the reentrancy code also exists in at-spi2-core/dbind/dbind.c:send_and_allow_reentry, which ends up getting invoked via a call to at-spi2-core/dbind/dbind.c:dbind_method_call_reentrant. Would it be possible to create a Python module for dbind (i.e., 'import dbind') that ends up exposing dbind_method_call_reentrant for us to use?
2009-05-07 IRC chat between Mark Doffman and Willie Walker
(12:28:55) WillieWalker: doffm: is it basically just a new special main loop?
(12:29:30) doffm: Yes! A very simple one. while (1) { Check the DBus message queue }
...
(12:42:00) WillieWalker: OK - so let's look at a use case - the user presses Tab in the Orca UI.
(12:42:34) WillieWalker: In this case, Orca might want to echo the "Tab" key and it will also get a focus event where it wants to ask the focused object some questions.
(12:43:15) doffm: (Slightly separate note) I got an answer from smcv that doing our own thing in AT-SPI would be OK, as long as the extra functions we wrapped were marked as __private, or something along those lines.
...
(12:43:34) WillieWalker: doffm: mgorse: on the issuing of the Tab event, atk-adaptor is going to block to wait for a 'consumed' reply
(12:44:37) WillieWalker: doffm: mgorse: while processing the 'Tab' event, Orca might ask questions about the current object with focus. Since atk-adaptor is blocked, Orca is blocked, and this whole thing deadlocks. That's example #1, right?
(12:45:00) doffm: WillieWalker: Yes. Thats the deadlock I'm thinking of.
(12:45:07) WillieWalker: doffm: mgorse: OK for the next one...
(12:45:44) WillieWalker: doffm: mgorse: Orca gets a "focus" event on one of its objects. All is cool since the event is delivered async. But now, Orca wants to query the object.
(12:46:49) WillieWalker: doffm: mgorse: so, Orca makes a sync call via pyatspi, blocking it. This query then comes into the atk-adaptor. We now have another deadlock.
(12:46:56) WillieWalker: That's example #2, right?
(12:48:07) doffm: WillieWalker: #2 is the deadlock that should be handled by adding re-entrancy to pyatspi calls.
(12:48:41) doffm: If the pyatspi call were re-entrant the Orca mainloop would be free to handle the message within the atk-adaptor and send a reply.
(12:49:00) doffm: #1 isn't affected by pyatspi re-entrancy. The problem is still there.
(12:49:18) doffm: But it always has been, as far as I can tell.
...
(13:27:28) WillieWalker: doffm: Orca used to do something like this back when we had our own atspi module: http://git.gnome.org/cgit/orca/tree/src/orca/atspi.py?h=gnome-2-18#n152
(13:41:03) WillieWalker: doffm: so...I'm trying to figure out what "{ Check the DBus message queue }" means.
...
(13:57:21) doffm: There is a function call. dbus_connection_read_write_dispatch that does it all.
(13:57:34) doffm: Reads from the socket, puts messages together, dispatches them.
(13:58:30) WillieWalker: doffm: so, would this be done in a separate thread?
(13:58:54) WillieWalker: doffm: i.e., we'd have two main loops?
(14:00:31) doffm: Normally when making a D-Bus method call you would call dbus_send_with_reply_and_block.
(14:01:03) doffm: Which would send the dbus message then loop over the socket waiting for the reply and placing all other d-bus messages on a queue.
(14:01:53) doffm: What we want to do is send the dbus message, then loop over the socket dispatching all messages until a reply is recieved.
...
(14:02:21) doffm: dbus_send_with_reply_and_dispatch if you will. (Although that function doesn't acutall exist.)
...
(14:21:24) doffm: WillieWalker: The reason I keep refering to it as a "main-loop" is that ORBit actually called the GLib main loop to serve the same purpose. It would send the ORBit message. Then it would iterate over the GLib mail loop until it recieved a reply. This way, not only orbit messages were dispatched, but all other events were handled also.
...
(14:31:37) WillieWalker: doffm: back to our mythical dbus_send_with_reply_and_dispatch... So, then we're not doing a main loop replacement. Instead, we're turning all the synchronous calls into calls that basically fire off a little main loop of their own until they get a reply?
...
(14:36:35) doffm: WillieWalker: Yeah. Thats right. We would be faking a syncronous call.
...
(14:42:38) WillieWalker: doffm: would one modify base.py:BaseProxy:get_dbus_method to return a method that does the main loop?
(14:43:02) doffm: WillieWalker: Yes.
(14:46:56) WillieWalker: doffm: so, it can be done more or less in one spot rather than needed to modify each area that has a "func = self.get_dbus_method" call. I guess then that there is some way to distinguish a reply to the function all from all other messages that are coming in?
...
(14:49:53) doffm: WillieWalker: Yeah, there is a reply identifier in the message header.
(14:53:25) WillieWalker: doffm: In 6c2ce8bcec496d34dfe65108104064b13b710cda you write "Despite what people may believe all D-Bus method calls must have a reply, whether the client side is waiting for one or not."....
(14:53:39) WillieWalker: doffm: so from that, I assume we are guaranteed to always get a reply?
(14:54:13) WillieWalker: doffm: In addition, I'm guessing this tricky wrapping only really needs to be done if the source and destination are the same app as mgorse suggests, right?
(14:57:11) doffm: WillieWalker: Yes, I'd probably do it for all the calls ATM though. Just to be safe.
(14:57:45) doffm: WillieWalker: Yes, we are guaranteed to get a reply. I've fixed all the places where we wern't sending one.
...
(15:14:11) ***WillieWalker scouring http://dbus.freedesktop.org/doc/dbus-python/api/
Have pyatspi detect it is calling itself and convert the DBus call into a local direct call
With this solution, pyatspi will detect the target of the call is the current application and will call the method directly instead of using D-Bus. NOTE: Simon McVittie prefers this to the other solution.
Ramblings/Questions:
Does applicationcache.py:self._connection.get_unique_name (available as base.py:self._cache.connection) tell us the unique name for us? Can we get the unique name for the proxy? Is it self._dbus_object.bus_name? Are these the things we compare? Of potential concern is http://dbus.freedesktop.org/doc/dbus-python/api/dbus.proxies.ProxyObject-class.html#get_dbus_method: "If the proxy was instantiated with a well-known name and with follow_name_owner_changes set false (the default), this property is the unique name of the connection that owned that well-known name when the proxy was instantiated, which might not actually own the requested well-known name any more."
- Where do we create this local dictionary to tie the path name to the object? Will this be similar (or the same) as the dictionary we use to handle the MANAGES_DESCENDANTS and infinite space stuff?
- What's the method we use to map the DBus method name to the object's method?
2009-07-29 IRC chat between Simon McVittie and Willie Walker
(12:25:46) WillieWalker: smcv - do you have time for a quick (hopefully) question about D-Bus? It's with respect to the time-favored discussion of reentrancy. I'm wondering if it might be possible to use the dispatch code to send the message directly and locally if it can be detected the client is effectively calling itself.
(12:26:38) mjj29_: WillieWalker: there's still the problem of indirect reentrancy
(12:26:43) smcv: WillieWalker: my opinion: "possible, terrifying, not worth it"
(12:28:10) mvidner left the room (quit: Remote closed the connection).
(12:29:10) WillieWalker: smcv: all things being equal (e.g., we need to make a synchronous call), which evil is worse -- direct/local calling or a custom loop in the client to handle the dispatch/reply?
(12:29:52) WillieWalker: smcv: this is wrt to the AT-SPI/D-Bus work, BTW, which doffm has been doing. reentrancy is a necessary evil for us.
(12:29:58) smcv: WillieWalker: detect that "actually this is me" and bypass D-Bus completely?
(12:30:06) WillieWalker: smcv: yep.
(12:30:14) smcv: that would be my vote
(12:30:18) mjj29_: try it, if you get a deadlock, then send it locally
(12:30:22) mjj29_: s'only way to be sure
(12:30:25) mjj29_ is now known as mjj29
(12:30:38) smcv: mjj29: well, you can know your own unique-name
(12:31:59) WillieWalker: smcv: OK. Cool. So you'd be somewhat in support of the "actually this is me" and bypass D-Bus completely approach.
(12:32:20) WillieWalker: Seems like it would solve perhaps a number of problems for a number of people, not just us crazy a11y folks.
(12:32:22) smcv: WillieWalker: it sounds less terrifying than re-entering the main loop, tbh
(12:32:45) smcv: WillieWalker: my inclination would be to do that in ATSPI and bypass libdbus entirely, I mean
(12:33:10) smcv: WillieWalker: I assume you have a fairly limited vocabulary of "things to do over D-Bus"
(12:33:19) WillieWalker: smcv: aha. OK. I thought it might be possible to detect this easier at a lower level in D-Bus.
(12:33:42) smcv: WillieWalker: I'd prefer it if libdbus didn't cheat
(12:33:51) smcv: WillieWalker: it's hard enough to predict what libdbus will do already :-)
(12:34:05) smcv: WillieWalker: (apart from "abort()" and "_exit()", which are usually a safe bet)
(12:34:58) WillieWalker: heh. doffm - you around? smcv - doffm thought we might need a method exposed or something to take this approach.
(12:35:02) smcv: if you do a direct local method call when necessary, then you're making it explicit in your code that (a) it's safe to block on, (b) you're bypassing the D-Bus message queue, and (c) you're not getting the dbus-daemon message-ordering guarantee
(12:35:46) smcv: whereas if you rely on libdbus to cheat, then libdbus will secretly be bypassing its message queue and stuff, and discarding the message ordering guarantee
(12:35:58) smcv: and, worse than that, it will be doing that in a way not obvious in your code
(12:36:16) smcv: you will need to expose some sort of C API to call back into yourself, yes
(12:36:58) smcv: if you currently use DBusMessage, you could synthesize a DBusMessage, pass it directly to the implementation of the method, and unref it without it ever entering the D-Bus connection
(12:37:29) smcv: or you could have a higher-level approach with actual method arguments and C functions
(12:37:32) smcv: depending
(12:43:00) ssp [n=ssp@c-24-218-20-219.hsd1.nh.comcast.net] entered the room.
(12:43:24) WillieWalker: smcv: many thanks. I'll take this and chew on it.
(12:56:31) J5 [n=quintice@66.187.234.199] entered the room.
(13:00:04) slomo [n=slomo@g228217064.adsl.alicedsl.de] entered the room.
(13:03:39) smcv: WillieWalker: in Python, you can probably just invoke the wrapped @methods directly
(13:19:42) smcv: WillieWalker: that's a client-side proxy
(13:19:52) smcv: WillieWalker: you need an escape hatch through which you can access the service-side objects
(13:20:07) smcv: WillieWalker: like a global registry of { object path => Object }
(13:20:20) smcv: sorry, { object path: Object }, this is Python :-P
(13:28:42) WillieWalker: smcv: so, drawing from http://cgit.freedesktop.org/at-spi2/at-spi2-atk/tree/pyatspi/base.py#n157, I'd use self._acc_path to get the object path and compare it to something so see if I were talking to myself. If so, I'd use a to-be-created dictionary to look up the local object referenced by the object path and then refer to its method from get_dbus_method.
(13:30:54) smcv: WillieWalker: yeah, except object paths are only unique within a process
(13:31:08) smcv: WillieWalker: so it's the unique bus name you want to compare with your own
(13:31:26) smcv: WillieWalker: (unique bus name = connection to a bus ~= process)
(13:32:21) WillieWalker: smcv: yep. I think I may actually understand this. (i.e., I'm not being snarky -- the light is finally dawning on Marblehead right now)
(13:35:13) mmc left the room (quit: Remote closed the connection).
(13:35:25) smcv: WillieWalker: imagine that object paths are directories on ftp servers
(13:35:34) smcv: WillieWalker: and that unique names are IP addresses
(13:35:41) smcv: WillieWalker: and that virtual hosting does not exist :-)
(13:36:00) smcv: WillieWalker: (HTTP-like name-based virtual hosting, I mean)
(13:36:33) smcv: WillieWalker: continuing that analogy, processes are servers
(13:36:44) WillieWalker: smcv: d'oh. right. So, I'd look at something else (not the path) to see if I were talking to myself. Then, I'd use the path to do the dictionary lookup for the actual Object.
(13:37:02) smcv: WillieWalker: so: each server has 1 or more IPs (but nearly always 1), and a directory tree
(13:37:18) smcv: WillieWalker: likewise, each process on the bus has 1 or more unique names (nearly always 1), and a tree of object paths
(13:37:32) smcv: WillieWalker: correct