TODOs, Plans, Crazy Ideas

See Bugzilla for open bugs.

Some hacking is occurring in various branches of the git repo at http://gnome.org/~danw/libsoup.git

HTTP Features

• Content-Encoding (522772 / content-coding branch in git)

  • At least gzip and deflate built-in
  • Possibly pluggable?
  • How to expose to user?
    • On sending side, allow both "this content is already encoded" and "please encode this content for me" declarations

    • On receiving side, allow automatic decoding.

      • Can't automatically autodecode, because that would trip up apps that are currently doing it by hand. :-/

      • How does this interact with soup_message_set_chunk_allocator?

  • Support for gzip/deflate Transfer-Encoding? But no one uses it...
• Caching
  • Client-side

    • Provide a SoupCache object that can be attached to a session

    • Should be transparent to the SoupSession API. Eg, you queue/send a GET, and the session/cache sends a conditional GET instead, and gets back a 304, and then creates a 200 response to return to the app.

    • Where do we store the cache? Single cache for all libsoup apps? How does the user manage it?
  • Server-side

    • Need to make it easier for SoupServers to set useful cache headers

    • Idea: soup_server_add_resource(server, path, ...) which provides a default handler for that path that acts RESTfully, and has information about its cacheability.

• Pipelining
  • Client-side
    • Need to distinguish safe/idempotent and non-s/i requests

    • Blocking on the the SoupConnection work

  • Server-side
    • Probably easier to implement than client-side; just pause and unpause messages as needed to ensure that they are returned in the right order even if the server doesn't finish processing them in the right order.
    • Servers already have to be re-entrant because of multiple connections. So adding automatic pipelining support will not break any contract.
• Proxy support
  • Client-side is mostly done in 2.25

    • Need a regression test for proxies that redirect to HTML login forms (bxc 68531)

    • Proxy connection management is currently broken; each origin server gets its own set of connections to the proxy. They should all share the same connections.
      • Unless you're using NTLM auth...
  • Server-side

    • Have SoupProxy (based on tests/simple-proxy, but better), which is a subclass of SoupServer

    • Handles Via and (optionally) X-Forwarded-For

    • Need to make it possible to implement CONNECT on the server side (by having a way to steal the socket from the SoupServer). This has to happen in such a way as to block further pipelining.

• Cookies
  • Client-side is done in 2.25
  • Server-side
    • Easy automatic session-id support
    • Easy automatic auth-cookie support

Authentication-related Features

• Integration with gnome-keyring in libsoup-gnome (522776, keyring branch in git)

  • When a password is needed, it checks gnome-keyring first, then if it's not found, it signals the application, and the callback allows the app to say whether or not to save the password to the keyring.

  • Make sure our key attributes match epiphany and Firefox

• Support OpenID

  • Not sure how much API this requires beyond normal auth
  • Client and server?

  • OpenID HTTP Authentication may also be relevant

• Support OAuth

  • Not sure how much API this requires beyond normal auth
  • Client and server?
• Support other miscellaneous auth types?

  • Yahoo! BBAuth

  • Flickr Auth

  • AOL OpenAuth

  • Google AuthSub

  • Google ClientLogin

    • evolution-data-server and drivel both already use this
• NTLM Proxy-Authentication does not currently work
• Add GSS-Negotiate support

• Needs to be possible for an application to register new auth types with SoupSession. (SoupAuthManager supports this, there's just no public API yet.)

• SoupAuthDigest/SoupAuthDomainDigest don't support qop="auth-int". (Apparently neither does anything else though...) SoupAuthDomainDigest doesn't support MD5-sess.

• Server auth helpers

  • Add a property to have SoupAuthDomainDigest work directly from a .htdigest file

  • Likewise for SoupAuthDomainBasic and .htpasswd? This is both less useful and harder, since there are several different .htpasswd storage formats

RPC Features

• XML-RPC
  • Client-side

    • soup_xmlrpc_call(), soup_xmlrpc_call_async()

    • SoupXMLRPCRequest

    • Support for extension types, eg these

      • Probably something like: soup_xmlrpc_extension_type_new ("http://ws.apache.org/xmlrpc/namespaces/extensions", "i8", "10000000000"). SoupXMLRPCExtensionType is a boxed type. You give the constructor an (optional) namespace, a type name, and an already-stringified representation of the value.

  • Server-side XML-RPC

    • soup_xmlrpc_add_server_handler(server, ...) ?

    • generated stubs/marshallers? (Can't quite use glib-genmarshal, but something similar. Or use libffi? Ties to GObjectIntrospection?)

      • Registering handlers this way would make it easy to implement the introspection extensions

  • Both client- and server-side

    • Add additional regression tests based on the spec and errata

    • system.multicall support?

  • Allow use of expat instead of libxml2? (No XML is actually exposed in the API, so this would be transparent to apps.) Does anyone want that?

• JSON (also RFC 4627) (json branch in git)

  • Parse/generate

  • Use GValue and soup-value-utils like the XML-RPC code. This is a bit more annoying than something like JSON-GLib, but

    • It makes things easy for language bindings (shared with XML-RPC, and can reuse existing GValue-handling code)

    • It makes it easy for SoupServers to provide both XML-RPC and JSON implementations of an API

  • JSON-RPC? Is anyone actually using this?

• AtomPub?

• XmlHTTPRequest / Ajax / "Comet"

  • Not clear that there's anything in particular we need to do to support these...

SSL

• The Fedora Crypto Consolidation project is trying to standardize applications on Mozilla's NSS library, for FIPS 140 compliance.

  • Red Hat Bug 347491: Port libsoup to use NSS library for cryptography

  • It may not be possible to make NSS work with the current libsoup SSL API; see the bugs that the above bug is blocking on.
  • Other distros/embedded platforms may prefer to stick with GnuTLS

• Need to allow the application to accept/reject server certificates itself. (Eg, 507802)

• Need to allow the application to provide client certificates, including certificates from crypto tokens. (Eg, 334021)

• Need to allow the application to provide certificate-related data from sources other than PEM-encoded files. (Eg, 507801)

• The above features should integrate with the gnome-keyring certificate store (GnomeKeyring/Cryptoki)

• 507802 has the most up-to-date discussion on the probable future SSL API.

Language bindings

• GObjectIntrospection. There are bindings in gir-repository.

• The use of g_object_new-style constructors is a problem for some languages, and we need to provide other non-varargs ones.

Connection handling

• Client-side I/O is currently a mess, because SoupSession, SoupConnection, and SoupMessage all take an active role in at least part of the process. This needs to be cleaned up. The 2.4 API prepared for this by hiding SoupConnection and some SoupMessage fields/methods. 2.25 does some internal cleanup. What remains:

  • SoupSession should handle CONNECT messages itself

  • SoupConnection at that point becomes just a very thin wrapper around SoupSocket, and maybe goes away entirely.

  • soup-message-io is rearranged to belong to SoupSession, not SoupMessage. (SoupMessage should be a passive object, which is acted upon by SoupSession.)

• At this point the I/O code will generally make a lot more sense than it does now, and it will be easier to fix certain bugs / add certain features:
  • Pipelining
  • Making connections time out eventually / noticing when the remote end has closed a connection before we try to write to it
  • A real pull API rather than the "I'll tell you when to push" hacks we have now.

General/API stuff

• We need a simple high-level API
  • basic HTTP stuff
  • basic WebDAV stuff?

    • evolution-exchange's E2kContext has a lot of Exchange-specific assumptions and exceptions, and it might not be possible to turn it into a generic API

  • jamesh's HttpResource object (baz repo), or something like it (to monitor an http resource).

• I/O filters

  • Occasionally libsoup has to deal with someone else's broken HTTP implementation. Eg, 475169, 502325. It would be nice to let application authors work around these things themselves rather than having to wait for a new libsoup release.

  • There could be low-level filters on the SoupSocket that would let you rewrite the data immediately after it was read (to let libsoup parse erroneous data) or before it was written (to let libsoup send erroneous data if another implementation somehow requires it)