What would it take to archive LBO

Chuck Grimes cgrimes at tsoft.com
Mon Jul 12 13:35:06 PDT 1999


On Sun, 11 Jul 1999, Michael Pollak wrote:


> how much it would take to set up a real archive for LBO.

The skill demands are quite minor; the hypermail package suffices quite admirably for several mailing lists that I'm on. However, it's a unix (windows ports do exist, but they're old, and unix boxes are much better for this kind of thing) program; I'd certainly be willing to volunteer my personal, 24/7 connected linux box, a few 100 MB of hard drive space, and a half hour of my time to host the LBO-talk archives (assuming LBO-talk-ers are in favour of creating a web archive - which is hardly settled).

So, first things first: what does LBO-talk, as a whole, want?

marco

--------------

You guys have got my vote. I looked through the ports directory and found hypermail already ported. For fun I downloaded and compiled it in two minutes, thirty seconds flat. For FreeBSD'ers out there, see /usr/ports/www. Anybody whose interested can look at:

http://www.landfield.com/hypermail/

to see what the archive system looks like and how it works. The format is both unix mailbox and html. According to the related e-mail lists, you can send attachments that can be decoded through hypermail. For specific directions on setting up the archive the latest entries (July) in the Hypermail Development List give a detailed script. See below (downloaded through lynx and pasted here into emacs--very fast).

It's an experiment to see what this looks like once it goes through the old unix 'mail' program and off to lbo-talk at list panic dot com.

Chuck Grimes

-------------------------------------------------------------------

Re: Archiving Question

---------

New Message Reply About this list Date view Thread view Subject view

Author view Other groups

Kent Landfield (kent at landfield.com)

Wed, 7 Jul 1999 20:23:42 -0500 (CDT)

* Next message: Bernhard Reiter: "minor patches for 2a23"

* Previous message: hypermail at voodoobox.net: "Archiving Question"

_________________________________________________________________

#

# My apologies if this has already been asked...

# but I have read all the documentation I can

# find and didn't see a FAQ anywhere.

#

# Does hypermail have an option to automatically

# sort messages into folder by date? (like

# 1999/jan, 1999/feb, etc.)

#

# I noticed that all of the hypermail archives

# I've seen were set up that way... so I thought

# it might be an option. Are you just doing

# it manually or with a script? If it is a

# script, could I beg a copy?

#

Take a look at the hypermail distributions since 2.x. There you will

find a directory named archive/. The README for the archive directory

utilities follows. Some of this will be integrated into hypermail

proper in the future.

==========================

This directory contains the archive specific information for

maintaining

mailing list archives. It is expected that the msg2archive.c

functionality

will be added to hypermail in a future release. Until then... This is

what

is here and how it works....

README - This file.

1998 - script for recreation of the hypermail archives

from the mailbox version

Makefile.in - configure Makefile input

msg2archive.c - Individual message archiver.

mbox2hypermail.c - Read messages from a mailbox and calls hypermail.

The contents of this directory support automatic generation of

hypermail

archives from inbound messages *as well as* the generation of a

hypermail

archive from a Unix mailbox of stored messages.

This is how I do it so... Your mileage might vary...

1. First you will need to decide where you want the archive stored at.

2. Then you will need to edit the file lists.h with the appropriate

info.

Simply add a section such as is shown below. I have a section for

each list. If you are not familiar with 'C' might want to get someone

else to assist. Notice that HYPERMAIL is different in the two entries.

This is because in the case of the wu-ftpd list, it is using a stock

hypermail executable to save the messages. The nfr-announce

list is using a customized version (very useful for testing too.)

#ifdef WU_FTPD

#define HYPERMAIL "/usr/local/bin/hypermail"

#define ARCHIVE "/ftp/wu-ftpd/mail-archive"

#define MAILBOXDIR "/ftp/wu-ftpd/mail-archive/mailbox"

#define LABEL "WU-FTPD Development Mailing List"

#define ABOUT_LINK "/wu-ftpd"

#define LISTNAME "wu-ftpd"

#define CONFIGFILE "NONE"

#else

#ifdef NFR_ANNOUNCE

#define HYPERMAIL "/usr/local/bin/nfrhypermail"

#define ARCHIVE "/ftp/nfr/mail-archive/nfr-announce"

#define MAILBOXDIR "/ftp/nfr/mail-archive/nfr-announce/mailbox"

#define LABEL "Network Flight Recorder Announcements"

#define ABOUT_LINK "/nfr/mail-archive"

#define LISTNAME "nfr-announce"

#define CONFIGFILE "/usr/local/lib/hypermail/nfr-announce.rc"

...

If you specify a CONFIGFILE path, it will be used to set hypermail

options and is call as

"HYPERMAIL -u -i -c CONFIGFILE".

If you do not specify a CONFIGFILE path, hypermail is called using

the other information as

"HYPERMAIL -u -i -d ARCHIVE/year/month -l \"LABEL\" -b

ABOUT_LINK"

3. Next make sure that you have a properly compiled version of

hypermail

installed on the system.

4. Edit the Makefile and add the appropriate entries. For example:

wuftpdmail: msg2archive.c

$(CC) -DWU_FTPD -o $@ msg2archive.c

wu-rdmsg: mbox2hypermail.c

$(CC) -DWU_FTPD -o $@ mbox2hypermail.c

5. Compile and install the new programs.

Now everything software-wise is in place. Time to setup your mailer

to process the inbound messages.

6. You need to add an entry to your /etc/aliases file such as

wuftpd: "|/usr/local/bin/wuftpdmail"

7. Assure that your mailer is aware of the new alias, i.e. might need

to type 'newaliases'.

Now you are ready to start receiving messages. Time to subscribe your

new alias to the list you want to archive. If you are running the list

then this is easy. In some cases you have to play games to get an

alias

subscribed with other than you normal account. What ever the case get

the new alias subscribed.

======================

So what happens now ?

======================

Messages sent to a list and are resent to the members of the list.

The alias you subscribed is now a member of the list. Messages are

sent to the list archive alias. The "msg2archive" process is started

for each message and the messages are stored into the location

specified

in the MAILBOXDIR define. Messages are stored in the directory in a

"month' file or as specified in the configuration file. These are Unix

mailbox formatted files. Once the message is archived in mailbox

format,

it is also then archived in hypermail format. The appropriate

hypermail

is called with the inbound message sent to it to add to the hypermail

archive.

In this manner each article is available to those who want to read

them online and to those who want to download an entire month's worth

of traffic to read with their favorite mail Unix reader.

If there is ever a problem with the hypermail archives, be it a new

look and feel wanted or a bug fix that requires regeneration of the

archive, the mailbox versions will be used as input to the process.

The script `1998' and the `mbox2hypermail.c' program in this directory

are being used for testing and archive re-creation.

If you want to change the look and feel of the individual hypermail

archive pages, that can be done easily by creating the list specific

header and footer HTML template files as described in the hypermail.1

and hmrc.4 man pages.

===========================

Integrating with Majordomo:

===========================

If you are running a majordomo list and want to integrate this with

majordomo then make sure that the ftp mailbox directory is symlinked

to ~majordomo/Archives/listname directory so that this scheme does

not break majordomo. DO NOT HAVE MAJORDOMO DO ANY ARCHIVING. The

symlink'ed directory is so people will be able to request archives

and index listing from majordomo as normal. Now the list archives

are reachable via the web, ftp or email.

-- Kent Landfield Phone: 1-817-545-2502 Email: kent at landfield.com http://www.landfield.com/ Email: kent at nfr.net http://www.nfr.net/ Search the Usenet FAQ Archive at http://www.faqs.org/faqs/ Search the RFC/FYI/STD/BCP Archive at http://www.faqs.org/rfcs/

_________________________________________________________________

* Next message: Bernhard Reiter: "minor patches for 2a23"

* Previous message: hypermail at voodoobox.net: "Archiving Question"

New Message Reply About this list Date view Thread view Subject view

Author view Other groups

---------

This archive was generated by hypermail 2.0b4.



More information about the lbo-talk mailing list