332 lines
10 KiB
Plaintext
332 lines
10 KiB
Plaintext
Aug 2003, Jorge Arellano Cid,
|
|
Ferdi Franceschini --
|
|
Last update: Nov 2009
|
|
|
|
|
|
------
|
|
dpid
|
|
------
|
|
|
|
-------------
|
|
Nomenclature:
|
|
-------------
|
|
|
|
dpi:
|
|
generic term referring to dillo's plugin system (version1).
|
|
|
|
dpi1:
|
|
specific term for dillo's plugin spec version 1.
|
|
at: https://dillo-browser.github.io/old/dpi1.html
|
|
|
|
dpi program:
|
|
any plugin program itself.
|
|
|
|
dpi framework:
|
|
the code base inside and outside dillo that makes dpi1
|
|
working possible (it doesn't include dpi programs).
|
|
|
|
dpip:
|
|
dillo plugin protocol. The HTML/XML like set of command tags
|
|
and information that goes inside the communication sockets.
|
|
Note: not yet fully defined, but functional.
|
|
Note2: it was designed to be extensible.
|
|
|
|
dpid:
|
|
dillo plugin daemon.
|
|
|
|
server plugin:
|
|
A plugin that is capable of accepting connections on a socket. Dpid will
|
|
never run more than one instance of a server plugin at a time.
|
|
|
|
filter plugin:
|
|
A dpi program that reads from stdin and writes to stdout, and that
|
|
exits after its task is done (they don't remain as server plugins).
|
|
Warning, dpid will run multiple instances of filter plugins if requested.
|
|
|
|
-----------
|
|
About dpid:
|
|
-----------
|
|
|
|
* dpid is a program which manages dpi connections.
|
|
* dpid is a daemon that serves dillo using IDS sockets.
|
|
* dpid launches dpi programs and arranges socket communication
|
|
between the dpi program and dillo.
|
|
|
|
The concept and motivation is similar to that of inetd. The
|
|
plugin manager (dpid) listens for a service request on a socket
|
|
and returns the socket/port pair of a plugin that handles the
|
|
service. It also watches sockets of inactive plugins and starts
|
|
them when a connection is requested.
|
|
|
|
|
|
-----------------------------------------------------------
|
|
What's the problem with managing dpi programs inside dillo?
|
|
-----------------------------------------------------------
|
|
|
|
That's the other way to handle it, but it started to show some
|
|
problems (briefly outlined here):
|
|
|
|
* When having two or more running instances of Dillo, one
|
|
should prevail, and take control of dpi managing (but all
|
|
dillos carry the managing code).
|
|
* If the managing-dillo exits, it must pass control to another
|
|
instance, or leave it void if there's no other dillo running!
|
|
* The need to synchronize all the running instances of
|
|
dillo arises.
|
|
* If the controlling instance finishes and quits, all the
|
|
dpi-program PIDs are lost.
|
|
* Terminating hanged dpis is hard if it's not done with signals
|
|
(PIDs)
|
|
* Forks can be expensive (Dillo had to fork its dpis).
|
|
* When a managing dillo exits, the new one is no longer the
|
|
parent of the forked dpis.
|
|
* If Unix domain sockets for the dpis were to be named
|
|
randomly, it gets very hard to recover their names if the
|
|
controlling instance of dillo exits and another must "take
|
|
over" the managing.
|
|
* It increments dillo's core size.
|
|
* If dillo hangs/crashes, dpi activity is lost (e.g. downloads)
|
|
* ...
|
|
|
|
That's why the managing daemon scheme was chosen.
|
|
|
|
|
|
----------------------
|
|
What does dpid handle?
|
|
----------------------
|
|
|
|
It solves all the above mentioned shortcomings and also can do:
|
|
|
|
* Multiple dillos:
|
|
dpid can communicate and serve more than one instance
|
|
of dillo.
|
|
|
|
* Multiple dillo windows:
|
|
two or more windows of the same dillo instance accessing dpis
|
|
at the same time.
|
|
|
|
* Different implementations of the same service
|
|
dpi programs ("dpis") are just an implementation of a
|
|
service. There's no problem in implementing a different one
|
|
for the same service (e.g. downloads).
|
|
|
|
* Upgrading a service:
|
|
to a new version or implementation without requiring
|
|
patching dillo's core or even bringing down the dpid.
|
|
|
|
|
|
And finally, being aware that this design can support the
|
|
following functions is very helpful:
|
|
|
|
SCHEME Example
|
|
------------------------------------------------------------
|
|
* "one demand/one response" man, preferences, ...
|
|
* "resident while working" downloads, mp3, ...
|
|
* "resident until exit request" bookmarks, ...
|
|
|
|
* "one client only" cd burner, ...
|
|
* "one client per instance" man, ...
|
|
* "multiple clients/one instance" downloads, cookies ...
|
|
|
|
|
|
--------
|
|
Features
|
|
--------
|
|
* Dpi programs go in: "EPREFIX/dillo/dpi" or "~/.dillo/dpi". The binaries
|
|
are named <name>.dpi as "bookmarks.dpi" and <name>.filter.dpi as in
|
|
"hello.filter.dpi". The ".filter" plugins simply read from stdin
|
|
and write to stdout.
|
|
* Register/update/remove dpis from list of available dpis when a
|
|
'register_all' command is received.
|
|
* dpid terminates when it receives a 'DpiBye' command.
|
|
* dpis can be terminated with a 'DpiBye' command.
|
|
* dpidc control program for dpid, currently allows register and stop.
|
|
|
|
|
|
-----
|
|
todo:
|
|
-----
|
|
|
|
These features are already designed, waiting for implementation:
|
|
|
|
* dpidc remove // May be not necessary after all...
|
|
|
|
|
|
-----------------
|
|
How does it work?
|
|
-----------------
|
|
|
|
o on startup dpid reads dpidrc for the path to the dpi directory
|
|
(usually EPREFIX/lib/dillo/dpi). ~/.dillo/dpi is scanned first.
|
|
|
|
o both directories are scanned for the list of available plugins.
|
|
~/.dillo/dpi overrides system-wide dpis.
|
|
|
|
o next it creates internet domain sockets for the available plugins and
|
|
then listens for service requests on its own socket,
|
|
and for connections to the sockets of inactive plugins.
|
|
|
|
o dpid returns the port of a plugin's socket when a client (dillo)
|
|
requests a service.
|
|
|
|
o if the requested plugin is a 'server' then
|
|
1) dpid stops watching the socket for activity
|
|
2) forks and starts the plugin
|
|
3) resumes watching the socket when the plugin exits
|
|
|
|
o if the requested plugin is a 'filter' then
|
|
1) dpid accepts the connection
|
|
2) maps the socket fd to stdin/stdout (with dup2)
|
|
3) forks and starts the plugin
|
|
4) continues to watch the socket for new connections
|
|
|
|
|
|
|
|
|
|
---------------------------
|
|
dpi service process diagram
|
|
---------------------------
|
|
|
|
These drawings should be worth a thousand words! :)
|
|
|
|
|
|
(I)
|
|
.--- s1 s2 s3 ... sn
|
|
|
|
|
[dpid] [dillo]
|
|
|
|
|
'--- srs
|
|
|
|
The dpid is running listening on several sockets.
|
|
|
|
|
|
(II)
|
|
.--- s1 s2 s3 ... sn
|
|
|
|
|
[dpid] [dillo]
|
|
| |
|
|
'--- srs ------------------'
|
|
|
|
dillo needs a service so it connects to the service request
|
|
socket of the dpid (srs) and asks for the socket name of the
|
|
required plugin (using dpip).
|
|
|
|
|
|
(III)
|
|
.--- s1 s2 s3 ... sn
|
|
| |
|
|
[dpid] | [dillo]
|
|
| | |
|
|
'--- srs '---------------'
|
|
|
|
then it connects to that socket (s3, still serviced by dpid!)
|
|
|
|
|
|
(IV)
|
|
.--- s1 s2 s3 ... sn
|
|
| |
|
|
.[dpid] | [dillo]
|
|
. | | |
|
|
. '--- srs '---------------'
|
|
.
|
|
.............[dpi program]
|
|
|
|
when s3 has activity (incoming data), dpid forks the dpi
|
|
program for it...
|
|
|
|
|
|
(V)
|
|
.--- s1 s2 (s3) ... sn
|
|
|
|
|
[dpid] [dillo]
|
|
| |
|
|
'--- srs .---------------'
|
|
|
|
|
[dpi program]
|
|
|
|
... and lets it "to take over" the socket.
|
|
|
|
Once there's a socket channel for dpi and dillo, the whole
|
|
communication process takes place until the task is done. When
|
|
the dpi program exits, dpid resumes listening on the socket (s3).
|
|
|
|
|
|
--------------------------------
|
|
So, how do I make my own plugin?
|
|
--------------------------------
|
|
|
|
Maybe the simplest way to get started is to understand a few
|
|
concepts and then to use the hands-on method by using/modifying
|
|
the hello dpi. It's designed as an example to get developers
|
|
started.
|
|
|
|
---------
|
|
Concepts:
|
|
---------
|
|
|
|
* Dillo plugins work by communicating two processes: dillo
|
|
and the dpi.
|
|
* The underlying protocol (DPIP) has a uniform API which is
|
|
powerful enough for both blocking and nonblocking IO, and
|
|
filter or server dpis.
|
|
* The simplest example is one-request one-answer (for example
|
|
dillo asks for a URL and the dpi sends it). You'll find
|
|
this and more complex examples in hello.c
|
|
|
|
First, you should get familiar with the hello dpi as a user:
|
|
|
|
$dillo dpi:/hello/
|
|
|
|
Once you've played enough with it, start reading the well
|
|
commented code in hello.c and start making changes!
|
|
|
|
|
|
---------------
|
|
Debugging a dpi
|
|
---------------
|
|
|
|
The simplest way is to add printf-like feedback using the MSG*
|
|
macros. You can start the dpid by hand on a terminal to force
|
|
messages to go there. Filter dpis use sdterr and server dpis
|
|
stdout.
|
|
|
|
Sometimes more complex dpis need more than MSG*. In this case
|
|
you can use gdb like this.
|
|
|
|
1.- Add an sleep(20) statement just after the dpi starts.
|
|
2.- Start dillo and issue a request for your dpi. This will
|
|
get your dpi started.
|
|
3.- Standing in the dpi source directory:
|
|
ps aux|grep dpi
|
|
4.- Take note of the dpi's PID and start gdb, then:
|
|
(gdb) attach <PID>
|
|
5.- Continue from there...
|
|
|
|
|
|
------------
|
|
Final Notes:
|
|
------------
|
|
|
|
1.- If you already understand the hello dpi and want to try
|
|
something more advanced:
|
|
|
|
* bookmarks.c is a good example of a blocking server
|
|
* file.c is an advanced example of a server handling multiple
|
|
non-blocking connections with select().
|
|
|
|
2.- Multiple instances of a filter plugin may be run
|
|
concurrently, this could be a problem if your plugin records data
|
|
in a file, however it is safe if you simply write to stdout.
|
|
Alternatively you could write a 'server' plugin instead as they
|
|
are guaranteed not to run concurrently.
|
|
|
|
3.- The hardest part is to try to modify the dpi framework code
|
|
inside dillo; you have been warned! It already supports a lot of
|
|
functionality, but if you need to do some very custom stuff, try
|
|
extending the "chat" command, or asking in dillo-dev.
|
|
|
|
|
|
|
|
>>>>>>>>>>>>>>>>>>>>> <<<<<<<<<<<<<<<<<<<<<
|
|
|