February 3, 2002

Version 1.8604

Sigh. Integrating CAB's cookie management may have been a bad idea, it
doesn't handle domain cookies at all. Had to revert to always using the
OVL's management. Also fixed a bug with redirects causing a previously
cached URL to be overwritten. Also a minor tweak, the "Resolving Host"
status message really means something now. The OVL used to jump straight
to the "Connecting to Host" status message, even though it was just
starting the gethostbyname() call. Now if you see "Resolving Host" for
a long time, that probably means the destination's domain is down.

February 3, 2002

Version 1.8603

Fixed a memory leak in the gzip support code. Fixed an unterminated string
in the authentication code. Fixed the make_unix_time function, which was
using local time instead of GMT. Added support for CAB's cookie functions.
When the version of CAB supports cookies, the OVL gives them to CAB instead
of storing them itself. Exception - the OVL always keeps single-session
cookies to itself (cookies that have no expiration date, and thus expire
when the browser terminates). When CAB does not support cookies, the OVL
does all the cookie management, as before. Tweaked the Makefile to make
my life easier building SSL vs non-SSL binaries...

December 5, 2001

Version 1.8602

Fixed three unrelated bugs. One caused redirects to sometimes be processed
incorrectly. Another one caused some web servers to hang instead of
responding; this was because the overlay was sending lines terminated by '\n'
instead of the expected "\r\n" line terminator. Finally, the gzip handling
was changed to only decompress files of type text/*. This should fix the
problems with downloading .tar.gz files and such.

November 23, 2001

Version 1.8601

The overlay only works if drive U is the current drive. Added code to
set the current drive and path if it's not starting on U:.

November 21, 2001

Version 1.86

Now uses libz instead of spawning external gzip app. Greatly improves
HTML download speed when the server supports it.

November 11, 2001

Version 1.8406

Updated to work with MiNTlib 0.57. My entire system has been upgraded with
the SpareMiNT distribution. Also a minor improvement, for HTTP/1.1 servers
assume the connection is kept open by default. Connection re-use is much
more effective now but there is still room for improvement.

Note - because of the particular way that CAB loads modules, this code is
still very much dependent on how the C library initializes itself. The code
in init.c is extremely library-dependent and there's really no way to avoid
this. The linking process also cannot use the standard C runtime startup,
so things are still inconvenient there.

Since the SpareMiNT distribution only includes a 68020 version of the
OpenSSL libraries, this binary is 68020-only. When I get a chance to rebuild
the OpenSSL library I will build a 68000 binary.

February 13, 1998

Version 1.8404 (1.8403 was only sent to one beta tester.)

Fixed a bug in redirect processing, would resolve the wrong path if the
"http:" prefix was missing. Fixed a bug in the main request handler, it
didn't allow POSTing form results via mailto. Fixed a refresh bug, the
reloadflag that CAB gave us was declared as int but needed to be short.
Fixed an SSL bug, have to run the library in bug-compatibility mode to
talk to certain broken SSL servers.

Doc note - if you have an accelerated machine (Nemesis, Afterburner, etc.)
you will probably experience lots of crashes unless you turn off the Fast
RAM flags on the CAB.OVL binary. Don't know why, but turning off the flags
makes things more stable.

December 27, 1997

Fixed bug with CAB 1.5, wasn't giving back HEADer information. Removed
code to map HTTP errors to system error codes; we just report "success"
and let CAB display the error message. Added X-Mailer header to the
mail handler. This is version 1.8402. The last two digits of the version
number are now used as a fix/patch counter. The December 20 release was
1.8401.

December 20, 1997

Added a first attempt at general documentation (ovldoc.txt). Fixed a bug in
the debug(!) printing. Reduced frequency of byte count updates, they were
actually slowing down transfers with a fast connection.

December 11, 1997

Found several bugs in the authorization code. It now passes all of the basic
auth tests on http://digest-test.agranat.com. (I'm pretty happy with this,
even the latest version of Netscape fails the multiple-realm test.) Also
found bugs in the config file handling; it now looks for "modules\cabovl.cnf"
in addition to "cabovl.cnf", since you will probably want the OVL in your
modules subdirectory. It occurred to me that since you could have multiple
copies of the OVL loaded, depending on use of other CAB modules, this OVL
ought to be a shared-text binary. We'll try that for 1.85. 

Found a problem with NCSA web servers, they freak out if you send them an
HTTP/1.1 request; they send back a response without a valid HTTP version ID.
The OVL now recognizes this case. Also found a problem with Netcache 3.11d
proxy server on Solaris, it can insert extra bytes into Chunked text transfers.
(God knows if it also happens on binary files; I can't verify the content!)
The OVL now tries to detect these extra inserted bytes and save them into
the output file.

Re-enabled ftp, gopher, wais, and nntp via HTTP-based proxy. Silly to have
turned that off in the first place. Added nil-pointer checking back in for
CAB proxy table. The 2.7 beta is giving nil-pointers where it didn't used
to, and shouldn't...

December 7, 1997

Finished Basic authorization. I was going to work on Digest auth, but the
HTTP/1.1 spec hasn't really been finalized yet. A new 1.1 draft was issued
on November 21 and there are still a couple of open issues unresolved in
the spec. Also I'm sure it will be some months before anyone has web servers
that actually use this stuff.

Fixed a bug with redirects, they needed to be fully parsed before handing
them back to CAB. Also, redirects used to involve nested recursive calls
to the GetUrl function, potentially consuming a lot of stack space. They
now use a longjmp into GetUrl, so no additional stack space is consumed.

Added maxhosts and maxnodes keywords to the config file. This lets you tune
the size of the in-memory URL info cache. Lower bound is "2" for both of
these values. See the CABOVL.CNF file.

I'm still waiting on supporting the CAB 2.7 cookie management, it looks like
the current draft is missing some functionality. So, the OVL continues to do
all cookie management by itself.

At this point I need to stop and take care of other things in my life. The
features I see that still need to be added are:
	internal Gopher - later
	internal WAIS - much later. I haven't used WAIS in ages...
	piped downloads - it can be done in the OVL, clumsily, or it
		could be done in the filesystem, elegantly. Also, it
		could be done just in the minixfs driver, or it could
		be done in the MiNT kernel, thus supporting all filesystems.
		I'm beginning to lean toward changing the main kernel.

December 6, 1997

Finished cookie support. Preliminary authentication/authorization is in, but
it's not really decent yet. This is version 1.84. Note - Alex has added some
kind of cookie caching mechanism to CAB 2.7. I have not added support for
that yet. Currently the overlay caches cookies in memory only, and it doesn't
check for the new functions in CAB 2.7. That'll come later.

Also, I got rid of the CMGZIP and MAILUSER environment variables. I now have
a CABOVL.CNF config file. Currently there are only 3 keywords recognized for
this file: mailfrom, gzip, and cookies. See the sample file for descriptions.
The CABOVL.CNF file goes in the same folder as CAB.APP and CAB.OVL.

November 30, 1997

Did some major rewriting and reorganizing here. I guess that'll be obvious if
you examine the source files. There still isn't much in the way of user-
visible changes. But underneath, the thing is fully HTTP/1.1 compliant, and
now supports reusable connections. It also fixes some parts where I hadn't
followed the HTTP/1.1 spec carefully enough before.

Cookies and Authentication/Authorization are still missing; other work took
too long to complete. Maybe by Christmas...

PS: I'm calling this version 1.83 now; the July code with SSL was 1.82 and
was never publically distributed.

July 6, 1997

The overlay can now support HTTPS for secure connections using SSL. The
support is all conditionally compiled in the network.c file; you will have
to build the SSLeay library yourself to use it. I used SSLeay 0.80, 0.81 is
already out and there are mmany patches to that as well. I haven't built
with 0.81 yet. I cannot distribute the SSL binaries due to stupid US laws
about exporting encryption technology. Some kind soul needs to contact me
about providing SSL-capable binaries from outside the US.

June 17, 1997

I received a new CAB 2.5 beta from Alexander Clauss which fixed the parameters
of get_url_if_modified to agree with the documentation. So, I had to change my
definition again. No other functional changes.

June 14, 1997

OK, I swear this is the real, final, 1.81.

Fixed the bytes-received reporting. Before, you could sometimes see status
like "received 300/250" or other nonsense, because of the way the overlay
had sent the CAB parent a status message, and then continued processing.
Since CAB gives the overlay pointers to work with and update, the overlay
could have updated the pointers multiple times before CAB got around to
processing all of the status messages. Now it will not update the filesize
pointer until after all status messages have been processed. (Gee, never
thought a program could work *too* fast now, did you...)

Broke out the time/date stamp version stuff into a separate file, for easier
updating. Stripped down the child startup code. Rewrote gzip spawning code.
On aborts, return 0 (success) instead of -1. CAB seems to like this better.

June 13, 1997

I believe that I've finally wiped out the STOP button crashes. There was
also an error in the CAB 2.5 documentation, the last argument to
get_url_if_modified is the time itself, not a pointer to the time. The
get_url_if_modified function now works with the latest CAB 2.5 beta. I've
changed my mind about the Cookie management, and have decided to release
this 1.81 as-is. Further additions will have to wait.

One more tweak - if a HEAD/get_url_info is redirected, the info is saved so
that the subsequent GET/get_url on the same url doesn't do a redundant query.

June 11, 1997

This is version 1.81. 1.80 was accidentally released with 68020 libraries
linked in, thus causing it to crash on unmodified STs. There was also a bug
where hitting the STOP button during a transfer would very likely crash the
overlay. Also, the overlay was calling menu_register unconditionally, when
it should have only done it if MultiTOS was active. The overlay now includes
Accept and Host headers in its requests, to allow it to talk to HTTP 1.1
servers. I'm still leaving "HTTP/1.0" for my request version, since I haven't
fully digested the HTTP 1.1 spec. I have also added support for the new
get_url_if_modified call in CAB 2.0. At this time, CAB 2.0 is broken and
does not use the new call, but that should be fixed soon.

The overlay also uses a new CMGZIP environment variable, which should be set
to the path of your gzip executable. If set, the overlay will add an
	Accept-Encoding: x-gzip, x-compress, gzip, compress
header to all of its requests. Any data that is received with a header of
	Content-Encoding: gzip
or whatever will be piped through gzip and uncompressed before storing to disk.

There is a separate bug in the CAB 2.0 beta, where pressing the STOP button
during a transfer corrupts CAB's memory. This will cause the overlay to
crash. The bug has not appeared in CAB 1.5 (so far).

This is not the final 1.81 release. I plan to add Cookie management before
making a general 1.81 release available. Please bash on this some more and
let me know what breaks.

June 4, 1997

This code is derived from Andrew Vesperman's version 1.64 source release.
However, I have radically restructured a lot of the code to fit it into
a multi-threaded model. I have fixed old bugs in the 1.64 code, and no
doubt introduced several new ones of my own.

This module is numbered version 1.80, just to keep it separate from the
releases that Andrew did. His last release was 1.77, which included partial
FTP support. That code is not present in this release.

This release includes mailto support as well as FTP support via an FTP
proxy. Real FTP isn't done yet. Since I didn't have access to the 1.7x
source code, I have to write it again from scratch.

DETAILS:

This overlay module works with CAB 1.5 and the CAB 2.0 demo. It is known to
have problems with the CAB 2.0 beta that is currently out. This module is
different from others because it is implemented as a threaded subprocess,
and does most of its work concurrently with the main CAB process.

The overlay has been tested with MiNTnet 1.01 and multiple MiNT versions
from 1.12 up. It also works with MultiTOS (AES 4.1) other AES replacements,
but is known to crash under Geneva. (If this is a problem for you, feel
free to dig through the source code and fix it. I don't have Geneva, so
I won't be looking for the problem myself.)

The overlay uses two environment variables to alter its behavior. If the
CMDEBUG variable is set, the overlay will log its activity into a file
called cmdebug.log. The other variable is the MAILUSER variable, which
should be set to your email address. This setting is used by the mailto
processing to set your return address on outgoing messages.

FEATURES:

	Faster URL-info caching
	Faster redirect handling
	Faster transfers
	Supports HTTP 1.1 error codes
	...did I mention that it's faster?
	Better proxy handling - uses the CAB proxy variables instead of
	  environment variables

Thanks to my beta testers who volunteered to try this code in its early
stages and suffered through its evolution over the past couple weeks:

	Yves Pelletier <ypell@CAM.ORG>
	Anders Eriksson <ae@wombat.ludvika.se>
	Katherine L Ellis <kellis@primenet.com>
	Jo Even Skarstein <joska@nuts.edu>
	Rodolphe Pineau <rodolphe.pineau@InfoRoute.CGS.Fr>
	atack <lang@eryx1.zcu.cz>
	Mario Becroft <mb@tos.pl.net>
	Bill Hallman <whallman@flash.net>

This code could not have been released without their input.

  -- Howard Chu  hyc@locus.com, chu@platinum.com

PS: since this overlay spawns a child process that accesses the parent's
memory, you must set the program flags on CAB.APP to be Global, or run
MiNT without any memory protection at all. Btw, I'm using a fairly ancient
release of MiNT (1.12, with patches) and MiNTnet 1.01, and pppd 2.1.2,
patched to support 115200bps connections, on HSModem0.7.

PPS: the included source code was built with the patchlevel 44 MiNT library.
The code in init.c is highly dependent on the library version, and will only
compile under GCC. Yes, it's ugly, and sane programmers don't go mucking
about in their C library like this, but I felt it was necessary to obtain
the fastest performance with the smallest memory consumption.

Future: I figure it needs a real FTP client. Will also add support for the
get_url_if_modified function in CAB 2.0; this will speed up response time
even more. Still working out how to parallelize things more, and boost the
network efficiency some more. Unfortunately I have no HTTP 1.1 servers
available to test with. gzip/compress decoding might be worth adding too.
