Initial import of Dillo
This commit is contained in:
331
devdoc/Dpid.txt
Normal file
331
devdoc/Dpid.txt
Normal file
@ -0,0 +1,331 @@
|
||||
Aug 2003, Jorge Arellano Cid,
|
||||
Ferdi Franceschini --
|
||||
Last update: Nov 2009
|
||||
|
||||
|
||||
------
|
||||
dpid
|
||||
------
|
||||
|
||||
-------------
|
||||
Nomenclature:
|
||||
-------------
|
||||
|
||||
dpi:
|
||||
generic term referring to dillo's plugin system (version1).
|
||||
|
||||
dpi1:
|
||||
specific term for dillo's plugin spec version 1.
|
||||
at: https://dillo-browser.github.io/old/dpi1.html
|
||||
|
||||
dpi program:
|
||||
any plugin program itself.
|
||||
|
||||
dpi framework:
|
||||
the code base inside and outside dillo that makes dpi1
|
||||
working possible (it doesn't include dpi programs).
|
||||
|
||||
dpip:
|
||||
dillo plugin protocol. The HTML/XML like set of command tags
|
||||
and information that goes inside the communication sockets.
|
||||
Note: not yet fully defined, but functional.
|
||||
Note2: it was designed to be extensible.
|
||||
|
||||
dpid:
|
||||
dillo plugin daemon.
|
||||
|
||||
server plugin:
|
||||
A plugin that is capable of accepting connections on a socket. Dpid will
|
||||
never run more than one instance of a server plugin at a time.
|
||||
|
||||
filter plugin:
|
||||
A dpi program that reads from stdin and writes to stdout, and that
|
||||
exits after its task is done (they don't remain as server plugins).
|
||||
Warning, dpid will run multiple instances of filter plugins if requested.
|
||||
|
||||
-----------
|
||||
About dpid:
|
||||
-----------
|
||||
|
||||
* dpid is a program which manages dpi connections.
|
||||
* dpid is a daemon that serves dillo using IDS sockets.
|
||||
* dpid launches dpi programs and arranges socket communication
|
||||
between the dpi program and dillo.
|
||||
|
||||
The concept and motivation is similar to that of inetd. The
|
||||
plugin manager (dpid) listens for a service request on a socket
|
||||
and returns the socket/port pair of a plugin that handles the
|
||||
service. It also watches sockets of inactive plugins and starts
|
||||
them when a connection is requested.
|
||||
|
||||
|
||||
-----------------------------------------------------------
|
||||
What's the problem with managing dpi programs inside dillo?
|
||||
-----------------------------------------------------------
|
||||
|
||||
That's the other way to handle it, but it started to show some
|
||||
problems (briefly outlined here):
|
||||
|
||||
* When having two or more running instances of Dillo, one
|
||||
should prevail, and take control of dpi managing (but all
|
||||
dillos carry the managing code).
|
||||
* If the managing-dillo exits, it must pass control to another
|
||||
instance, or leave it void if there's no other dillo running!
|
||||
* The need to synchronize all the running instances of
|
||||
dillo arises.
|
||||
* If the controlling instance finishes and quits, all the
|
||||
dpi-program PIDs are lost.
|
||||
* Terminating hanged dpis is hard if it's not done with signals
|
||||
(PIDs)
|
||||
* Forks can be expensive (Dillo had to fork its dpis).
|
||||
* When a managing dillo exits, the new one is no longer the
|
||||
parent of the forked dpis.
|
||||
* If Unix domain sockets for the dpis were to be named
|
||||
randomly, it gets very hard to recover their names if the
|
||||
controlling instance of dillo exits and another must "take
|
||||
over" the managing.
|
||||
* It increments dillo's core size.
|
||||
* If dillo hangs/crashes, dpi activity is lost (e.g. downloads)
|
||||
* ...
|
||||
|
||||
That's why the managing daemon scheme was chosen.
|
||||
|
||||
|
||||
----------------------
|
||||
What does dpid handle?
|
||||
----------------------
|
||||
|
||||
It solves all the above mentioned shortcomings and also can do:
|
||||
|
||||
* Multiple dillos:
|
||||
dpid can communicate and serve more than one instance
|
||||
of dillo.
|
||||
|
||||
* Multiple dillo windows:
|
||||
two or more windows of the same dillo instance accessing dpis
|
||||
at the same time.
|
||||
|
||||
* Different implementations of the same service
|
||||
dpi programs ("dpis") are just an implementation of a
|
||||
service. There's no problem in implementing a different one
|
||||
for the same service (e.g. downloads).
|
||||
|
||||
* Upgrading a service:
|
||||
to a new version or implementation without requiring
|
||||
patching dillo's core or even bringing down the dpid.
|
||||
|
||||
|
||||
And finally, being aware that this design can support the
|
||||
following functions is very helpful:
|
||||
|
||||
SCHEME Example
|
||||
------------------------------------------------------------
|
||||
* "one demand/one response" man, preferences, ...
|
||||
* "resident while working" downloads, mp3, ...
|
||||
* "resident until exit request" bookmarks, ...
|
||||
|
||||
* "one client only" cd burner, ...
|
||||
* "one client per instance" man, ...
|
||||
* "multiple clients/one instance" downloads, cookies ...
|
||||
|
||||
|
||||
--------
|
||||
Features
|
||||
--------
|
||||
* Dpi programs go in: "EPREFIX/dillo/dpi" or "~/.dillo/dpi". The binaries
|
||||
are named <name>.dpi as "bookmarks.dpi" and <name>.filter.dpi as in
|
||||
"hello.filter.dpi". The ".filter" plugins simply read from stdin
|
||||
and write to stdout.
|
||||
* Register/update/remove dpis from list of available dpis when a
|
||||
'register_all' command is received.
|
||||
* dpid terminates when it receives a 'DpiBye' command.
|
||||
* dpis can be terminated with a 'DpiBye' command.
|
||||
* dpidc control program for dpid, currently allows register and stop.
|
||||
|
||||
|
||||
-----
|
||||
todo:
|
||||
-----
|
||||
|
||||
These features are already designed, waiting for implementation:
|
||||
|
||||
* dpidc remove // May be not necessary after all...
|
||||
|
||||
|
||||
-----------------
|
||||
How does it work?
|
||||
-----------------
|
||||
|
||||
o on startup dpid reads dpidrc for the path to the dpi directory
|
||||
(usually EPREFIX/lib/dillo/dpi). ~/.dillo/dpi is scanned first.
|
||||
|
||||
o both directories are scanned for the list of available plugins.
|
||||
~/.dillo/dpi overrides system-wide dpis.
|
||||
|
||||
o next it creates internet domain sockets for the available plugins and
|
||||
then listens for service requests on its own socket,
|
||||
and for connections to the sockets of inactive plugins.
|
||||
|
||||
o dpid returns the port of a plugin's socket when a client (dillo)
|
||||
requests a service.
|
||||
|
||||
o if the requested plugin is a 'server' then
|
||||
1) dpid stops watching the socket for activity
|
||||
2) forks and starts the plugin
|
||||
3) resumes watching the socket when the plugin exits
|
||||
|
||||
o if the requested plugin is a 'filter' then
|
||||
1) dpid accepts the connection
|
||||
2) maps the socket fd to stdin/stdout (with dup2)
|
||||
3) forks and starts the plugin
|
||||
4) continues to watch the socket for new connections
|
||||
|
||||
|
||||
|
||||
|
||||
---------------------------
|
||||
dpi service process diagram
|
||||
---------------------------
|
||||
|
||||
These drawings should be worth a thousand words! :)
|
||||
|
||||
|
||||
(I)
|
||||
.--- s1 s2 s3 ... sn
|
||||
|
|
||||
[dpid] [dillo]
|
||||
|
|
||||
'--- srs
|
||||
|
||||
The dpid is running listening on several sockets.
|
||||
|
||||
|
||||
(II)
|
||||
.--- s1 s2 s3 ... sn
|
||||
|
|
||||
[dpid] [dillo]
|
||||
| |
|
||||
'--- srs ------------------'
|
||||
|
||||
dillo needs a service so it connects to the service request
|
||||
socket of the dpid (srs) and asks for the socket name of the
|
||||
required plugin (using dpip).
|
||||
|
||||
|
||||
(III)
|
||||
.--- s1 s2 s3 ... sn
|
||||
| |
|
||||
[dpid] | [dillo]
|
||||
| | |
|
||||
'--- srs '---------------'
|
||||
|
||||
then it connects to that socket (s3, still serviced by dpid!)
|
||||
|
||||
|
||||
(IV)
|
||||
.--- s1 s2 s3 ... sn
|
||||
| |
|
||||
.[dpid] | [dillo]
|
||||
. | | |
|
||||
. '--- srs '---------------'
|
||||
.
|
||||
.............[dpi program]
|
||||
|
||||
when s3 has activity (incoming data), dpid forks the dpi
|
||||
program for it...
|
||||
|
||||
|
||||
(V)
|
||||
.--- s1 s2 (s3) ... sn
|
||||
|
|
||||
[dpid] [dillo]
|
||||
| |
|
||||
'--- srs .---------------'
|
||||
|
|
||||
[dpi program]
|
||||
|
||||
... and lets it "to take over" the socket.
|
||||
|
||||
Once there's a socket channel for dpi and dillo, the whole
|
||||
communication process takes place until the task is done. When
|
||||
the dpi program exits, dpid resumes listening on the socket (s3).
|
||||
|
||||
|
||||
--------------------------------
|
||||
So, how do I make my own plugin?
|
||||
--------------------------------
|
||||
|
||||
Maybe the simplest way to get started is to understand a few
|
||||
concepts and then to use the hands-on method by using/modifying
|
||||
the hello dpi. It's designed as an example to get developers
|
||||
started.
|
||||
|
||||
---------
|
||||
Concepts:
|
||||
---------
|
||||
|
||||
* Dillo plugins work by communicating two processes: dillo
|
||||
and the dpi.
|
||||
* The underlying protocol (DPIP) has a uniform API which is
|
||||
powerful enough for both blocking and nonblocking IO, and
|
||||
filter or server dpis.
|
||||
* The simplest example is one-request one-answer (for example
|
||||
dillo asks for a URL and the dpi sends it). You'll find
|
||||
this and more complex examples in hello.c
|
||||
|
||||
First, you should get familiar with the hello dpi as a user:
|
||||
|
||||
$dillo dpi:/hello/
|
||||
|
||||
Once you've played enough with it, start reading the well
|
||||
commented code in hello.c and start making changes!
|
||||
|
||||
|
||||
---------------
|
||||
Debugging a dpi
|
||||
---------------
|
||||
|
||||
The simplest way is to add printf-like feedback using the MSG*
|
||||
macros. You can start the dpid by hand on a terminal to force
|
||||
messages to go there. Filter dpis use sdterr and server dpis
|
||||
stdout.
|
||||
|
||||
Sometimes more complex dpis need more than MSG*. In this case
|
||||
you can use gdb like this.
|
||||
|
||||
1.- Add an sleep(20) statement just after the dpi starts.
|
||||
2.- Start dillo and issue a request for your dpi. This will
|
||||
get your dpi started.
|
||||
3.- Standing in the dpi source directory:
|
||||
ps aux|grep dpi
|
||||
4.- Take note of the dpi's PID and start gdb, then:
|
||||
(gdb) attach <PID>
|
||||
5.- Continue from there...
|
||||
|
||||
|
||||
------------
|
||||
Final Notes:
|
||||
------------
|
||||
|
||||
1.- If you already understand the hello dpi and want to try
|
||||
something more advanced:
|
||||
|
||||
* bookmarks.c is a good example of a blocking server
|
||||
* file.c is an advanced example of a server handling multiple
|
||||
non-blocking connections with select().
|
||||
|
||||
2.- Multiple instances of a filter plugin may be run
|
||||
concurrently, this could be a problem if your plugin records data
|
||||
in a file, however it is safe if you simply write to stdout.
|
||||
Alternatively you could write a 'server' plugin instead as they
|
||||
are guaranteed not to run concurrently.
|
||||
|
||||
3.- The hardest part is to try to modify the dpi framework code
|
||||
inside dillo; you have been warned! It already supports a lot of
|
||||
functionality, but if you need to do some very custom stuff, try
|
||||
extending the "chat" command, or asking in dillo-dev.
|
||||
|
||||
|
||||
|
||||
>>>>>>>>>>>>>>>>>>>>> <<<<<<<<<<<<<<<<<<<<<
|
||||
|
||||
Reference in New Issue
Block a user