Check out the FREE chatroom mod for IPB 3.x

Invision Board NNTP Gateway

Category: Convertors & Patches

Compatibility: All versions of Invision Board

Demo: news://forums.ru.ac.za/

Description:

This is not strictly a mod - it is a standalone Perl program that provides NNTP (usenet, news, etc) access to the forums.

It is intended to be run through inetd (or tcpserver or similar) and bind the nntp port (119). It speaks a limited subset of RFC 977 and RFC 2980 compliant NNTP to any NNTP client. It has been tested with slrn(1), tin(1), pine(1), Mozilla News, Netscape Communicator, and Microsoft Outlook Express.

There is support of NNTPv1 authentication to allow a user to post using their username and password.

The program interfaces directly with the SQL database behind the forums board. It makes every attempt to be sane about queries, and to match the real queries that would be performed by the web equivelent.

View This Mod

This is a great idea, however, I would like it converted to php if possible. I really don't like perl. What happens when you synch the newsgroup with the forum? Does it download every single post ever made in the database? That's a definite problem in my opinion since my forums have more than 100,000 posts. Is the newsgroup stored in the forum database, does it allow posts into topics? Does this present any problems since a user may be posting using the actual forum and a user with an unsynch'd usenet client might cause a db error?

How many known security issues are there that remain unfixed?

Can we get a better description of what exactly it does when synch is done?

I'm really trying to get as much information as I can about this.

This post has been edited by Skye: May 30 2003, 11:02 AM

It wouldn't be a conversion so much as a complete re-write. I've never really found PHP to be a useful scripting language. It is great for web pages, but the I've always found that the command line version doesn't work so well. If someone else has different experiences with it and wants to re-write the code in PHP, I'll give them whatever advice I can from what I've learnt doing this.



Your client will ask the server for a list of available articles and it'll return a list of article numbers in the current group (forum). What your client does after that depends on your client. There is support in the server for article expiry, so you won't necessarily see every article ever posted to the board - only ones that are considered current.



Yes and no. There are no changes to the database structure if that is what you mean. Each forum is represented by a newsgroup, and the articles in each one are retrieved from the database in the same way the web page version retrieves it. The name of each newgroup is worked out dynamically from the forum name, and all information (message ids, etc) are created in the same way. As a result, there is no need for the gateway to store any state information of its own.

For example, a forum named "Night Life" would become ru.forums.night_life as a newsgroup. The prefix is configurable. All articles in the ru.forums.night_life newsgroup would directly be retrieved from the database.



It allows authenticated users to post into topics, and create new topics. In each case, the users access rights for the forum in question are checked before this is allowed.



The syncing of your client is irrelevent, in the same way as my web page being out of date is irrelevent. Posts are based on the article number you are following up to, and if there are intervening posts by other users in the mean time, your post will end up at the bottom -- exactly as it does in the web version. The only requirement is that you have a news reader that correctly implements the (RFC specified) References: header. I haven't yet found one that doesn't.



The main issue is that un-authenticated users may be given access. This is however controlled in the configuration. Any un-authenticated user inherits the permissions of a specified forums group (say "Members"), and has the same permissions (with respect to reading articles) as any member of that group. Once the user has authenticated, they get their normal group permissions.

You can restrict access completely by creating a group with no permissions whatsoever and assigning it as the default group for non authenticated users.

The only other issue that I know about is a client-dependent problem. The NNTP server honours a users request for their e-mail address to be kept private. If, however, a user has set their e-mail address as publically readable, it is used as the From: field in the article.

When some clients (for example tin(1)) post followups, they often quote the e-mail address in the body. This leads to a situation where you get a forum post similar to the following



In other words the user's e-mail address is displayed in a post. This isn't really a server problem, but is caused by a difference in the way usenet clients and the forums interpret things.

I don't know of any other problems -- we've been using this version of the software for about six months now and we've yet to notice anything amiss.



There is no sync as such. The whole thing is completely stateless insomuch as every NNTP command is evaluated without making any assumptions about the state of the forums. As such, it is always in sync.

When your client requests an article, it has a Message-ID header which uniquely identifies it. These Message-IDs are generated by the news server, and a guarenteed to be unique for each article. They are, however, consistant -- so all users downloading the same article at any time get the same Message-ID. This is because the Message-ID is based on the name of the forum and the article number. This allows things to be completely stateless. Your client can cache copies of articles, and they'll always be in sync because the message id doesn't change.

For example, an NNTP LIST command allows you to list the article numbers in a group. If you issue two LIST commands, you may get different results -- for the simple reason that there may be new articles that have been posted between the two commands being issued. All that happens is that when the LIST implementation does a select on the database, it gets a different set of article numbers returned.




I hope this helps you some?

Is it possible to run this script with a hosted server without a services directory ?

Yes, it can be run under inetd(8).

I find the easiest way to do this is to create a little shell script that calls it -- that way you can
redirect the STDERR logging to any file you want. Something like

#!/bin/sh
exec /path/to/invgateway.pl 2> /path/to/logfile

will do the trick. This can be called from inetd(8) with a line like

nntp stream tcp nowait news /path/to/shell-script.sh shell-script.sh

where nntp is the nntp port (from /etc/services), news is the user running your daemon, and
shell-script.sh is the script you created above.

I suspect this can be done with xinetd too, in much the same way.

I may add a command line option to handle the logging better at some stage.

Well - I believe I wrongly expressed myself.

I have no own server - all my files and of course the forums too, are on a hosted server of a provider - in this case "Host Europe" but that doesnt matter I think. So it IS a Linux server but I think I dont have any access to INETD(8) or any other Linux progs. / commands ?

Is it true ?

Hey.

I've installed this, but I'm getting some weird errors:



I can see the forums/newsgroups (though it seems to cut off at some point and I haven't figured out what that point is), but I can't seem to post anything back up to the server. Also, if I authenticate (i.e. provide username/password), I don't see the extra forums that my normal user would see on the web version.

Any thoughts?

What Perl version, and what NNTP client are you using?

You can check that authentication is working properly by looking in the log files. You should see the username field (they are CIF format logs) changing from a - to the username. It'll do this as you authenticate, but if authentication fails it'll switch back to - on the next request.

WRT to the prototype errors: I use Perl 5.005, so its possible (likely even) that errors slip in when you use a later version of Perl. If you let me know what version you're using, I'll try get things to work better.

i don't suppose there is any planes to update this 1.2? i installed it and reading forums works. when i tried to post a reply i got:

DBD::mysql::st execute failed: Column count doesn't match value count at row 1 at /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi/Mysql.pm line 175, <> line 24.

Perl 5.6.0 built for i386-linux, according to perl -v

I can see that I'm being authenticated (in the logs), but I'm assuming this is based on the member group we configure in invgateway.pl -- I'm using my Guest group, so I'm only seeing the forums the Guest group can see, regardless of what my authenticated user account (and group) can see.

Also, the dates are wrong. I'm seeing completely the wrong dates in Outlook Express for each post.

And lastly, I'm not seeing all the posts. I have some groups with well over 20,000 posts and I'd like to know how to configure invgateway.pl to allow me to see all of them.

Thanks for your help!

This post has been edited by Djelibeybi: Jul 6 2003, 05:03 PM

I get the following errors when trying to post, btw:

Use of uninitialized value in concatenation (.) at /usr/local/invgateway-1.3/invgateway.pl line 1048, <> line 18.
DBD::mysql::st execute failed: Column count doesn't match value count at row 1 at /usr/lib/perl5/site_perl/5.6.0/i386-linux/Mysql.pm line 172, <> line 18.

I'm running via xinetd now and those prototype errors have disappeared, btw.

I have solved the mystery! Though, I require your help in fixing it.

When you build your SQL statements to add posts (in sub news_post), you're not specifying the field names in the tables. Now, my field order is obviously different to yours, and I've got additional fields in my post table to support some modifications, thus the post is not being submitted correctly.

I've tried to work out what the fields you're referencing are and in which order, but I have to admit that some of them have me stumped!

crap i guess the author has a disapeared

anyone get this to work?

PS. how do you get it to work with xinitd

I'll take a closer look and see if I can work out the SQL statements properly.

To get it to work with xinetd, create a file in /etc/xinetd.d with the following content:



Make sure you have a user created on your server called "invnntp" or change that field to something else. DO NOT run it as root.

ok thanx for that xinetd help. whats in your rungateway.sh? i just pointed the xinetd to the invgateway.pl but that gives wierd errors on the client.

do you run invision 1.2rc1?

wonder if this would work better with 1.1

This post has been edited by vexorg: Jul 24 2003, 10:50 PM