Additional features:

Spider: 
* offer to include any cookies received (present all cookies received, in order from first to last. Use select-multi, but make most recent override any older ones if selected.)
* offer to include any authentication headers sent previously (proxy auth, Basic Auth, etc)
* be able to supply parameters for login forms - would need a config file, because there could be a number of forms and a number or parameters. Identify by URL containing the FORM, the form identifier, and each INPUT field. Support wildcard matches?
* be able to supply an exclude list to prevent the spider from logging out!
* retrieve all pages for which a cookie was sent without the cookie, and check for any differences.






Requirements for a "mangle" interface:

Start and stop mangle from the GUI
tail the mangle log in real time, sort, insert and update entries.
Monitor the mangle debug log in real time.
Show tabular listing, as follows:
Request no, Method, Server, Path, Parameters (GET or POST), Result Code, Cookie set

Have ability to filter on certain attributes:
Exclude/include images (gif|jpg|png)
Exclude/include other file types (css,js,etc. User specified regex)
Exclude/include requests with parameters
Exclude/include responses with "Set-Cookie"
Exclude/include responses with user specified header and parameters (regex)

Have ability to colour highlight requests with above attributes.

Show "from client", "to server", "from server", "to client" logs.
Workbook interface. Tab is disabled if no changes made at proxy level.
Updates while scrolling?
Ability to search within above log windows.

Edit window for "mangle.inc"

Ability to "% decode" URL's and parameters, showing characters, and decimal, and hex.

Ability to search "text" file types for strings across the log base.

Maintain persistent Sequence number, survive across reboots, so long as the same log file is used. Get from logfile?

Ability to edit and update description of the action being done, linked to the sequence number of the next URL to be fetched.

Fix mangle to use a directory per session
Then log file is dir/log
description file is dir/descriptions
indiv files are: 
dir/seq-fromclient, 
dir/seq-toserver, 
dir/seq-fromserver, 
dir/seq-toclient
Run uses "random" temp dir for files, deleted on exit if not saved.
"save" is simply "rename temp dir"

IT IS PROBABLY EASIEST AT THE MOMENT TO RUN MANGLEUI AS A "TAIL" OF A PRE-
EXISTING MANGLE.PL PROCESS.

Mangle proxy will operate as follows (and can be run standalone):
Mangle is passed various parameters, including
- port
- interface
- debug
- log file
- log images
etc.

The mangle UI will execute mangle as follows, using a normal pipe open:
$logdir=`mktemp -d $ENV{'TMP'}/mangle.XXXXXX`
if (-x $pathto/mangle.pl) {
  $pid=open(DEBUG,"$pathto/mangle.pl -d -f $logdir/log -i 2>&1 |");
  $DEBUGfd=fileno(DEBUG);
  $err_Gtk=Gtk->add_input($DEBUGfd,updateDEBUG());
}
open(LOG,"<$logdir/log");
"read all currently available, leave remnants in a $var"
$LOGfd=fileno(LOG);
$log_Gtk=Gtk->add_input($LOGfd,updateLOG());

updateLOG() will consider $var, and add any new data to it, until it gets a 
CR, at which point it will extract the line, much as we do for the socket.

and kill it as follows:
Gtk->input_remove($err_Gtk);
Gtk->input_remove($log_Gtk);
waitpid (see man perlipc)
kill ("TERM",$pid);

Periodically, one should kill(0,$pid) to make sure it is still around . . .


Opening an existing logfile can also tail it, once it has been read in.

Will have to convert to sysread, but this seems to be compatable with 
magic open.  Necessary for use with Gtk::gdk add_input

