moderated >> [NEW MODULE] IO::Filter

by Andrew Stribblehill » Tue, 03 Aug 2004 19:22:54 GMT

I've written a module that acts as a wrapper around filehandles.

So, you might use it as an input filter:

use IO::Filter;
IO::Filter::make {s/foo/bar/} *STDIN, *WRAPPED;
while (<WRAPPED>) {
print;
}

or an output filter:

use IO::Filter;
IO::Filter::make {s/foo/bar/} *STDOUT, *WRAPPED;
while (<>) {
print WRAPPED $_;
}


Of course, wrappers stack too:

IO::Filter::make {s/foo/bar/} *STDIN, *X1;
IO::Filter::make {$_=uc} *X1, X2;

This way, you can implement a basic shell-like pipelining infrastructure.

Is the attached module suitable for CPAN?

--
Andrew Stribblehill
Systems programmer, IT Service, University of Durham, England


moderated >> [NEW MODULE] IO::Filter

by nadim » Wed, 04 Aug 2004 20:58:57 GMT


Hi, IMHO, 'make' is rather bad name. A bunch of examples (real ones best)
would make the chances of your module being used, by someone else than you, a
bit higher. Too many modules on CPAN just collect dust because of their
documentation (while the module is good).

Some better names for the 'wrapped' filehandle in the doc would be good
ex: while(<FOO_TO_BAR>)

An untie (and re-tie) example in the middle of a stream would be good.

The question is if your module is better than other ways to filter streams
(maybe other could give examples of how the usually do it.

Cheers, Nadim.

PS: perl module authors is often a better place to post this kind of questions

moderated >> [NEW MODULE] IO::Filter

by Terrence Brannon » Fri, 10 Dec 2004 08:55:47 GMT

Andrew Stribblehill < XXXX@XXXXX.COM > writes:


I think this is a great addition to CPAN. However, you should check
the structure of CPAN module documentation --- your POD section is
missing a NAME section and that should be first.

Similar Threads

1. New Module planned - MTA policy server, content filter interface, log parser

Hello all

i tend to release a module-bundle of mine. It is rather an
application, such as Catalyst or Mail::SpamAssassin (of course smaller
then those), then a single purpose module. Before i release this, i'd
like to ask whether anybody is interested in this.

In short what it does:
* It can be used with MTAs, such as postfix (only tested with postfix,
so far).
* It combines a policy server, a SMTP content filter (interface) and a
log parser. All run as separate servers, but able to communicate and
based on the same core modules.
* It does already include the "common" features (greylisting, DNSBL,
other white- and blacklists, SPF support, honeypot, throttling, DKIM
support, interfaces to multiple spam- and virus filters, archiving,
mime header manipulation and so on.. most of them based on existing
CPAN Modules) but the general idea is to be an extendable platform.
* It uses Cache::* modules for caching and DBD::* and MongoDB for
database support.
* The configuration is based on YAML files.
* Can be deployed either on a single mail server or in a distributed
environment.

Before you point out: there are already a bunch of policy servers (eg
policyd-weight..) and content filter interfaces (eg amavis), all
written in Perl -> True, but none on CPAN (afaik) and all that i know
are single purpose, self-sustained, in matters of interacting with
each other.
This approach is not entirely different, but has a "global point of
view" design. The initial goal for me was to simplify and generalize
the spam fighting measurements on my own mail servers. The interaction
between the multiple stand alone solutions i used before, was always
kind of a patchwork and lead to unnecessary errors and to much time
spend (i'm lazy) on configuration and adapting issues.

It is based on POE and uses Moose a lot. I'm writing the final tests,
wrap up the documentation and try to verify it's stability in a live
environment.

So:
* Is anybody interested in this ?
* What name-space would you suggest (it actually has a "project" name,
but i am not sure where in the hierarchy it would fit best.. maybe
Mail:: or Postfix::) ?


Greets from Berlin
Ulrich

2. Hyperlinks - Open new window or keep filter values on return - Excel

3. Can't locate object method "new" via package "IO::socket::INET"

4. IO::Socket Stop listening for new connections after initial connection - Perl

5. can't locate method new IO::Socket::INET

Sometimes, not always, an attempt to create a new socket connection
fails with this message:
Socket::INET" (perhaps you forgot to load "IO::Socket::INET"?) at
/usr/local/lib/perl5/5.6.1/IO/Socket/INET.pm line 32

Any ideas why or how INET.pm fails saying it can't find new in INET.pm?
The script runs on many servers many times a day, but only fails on one
and generally about the same time of the day.   Okay.  Seems kind of
obvious that is something specific to the server and the time of day.
But, within the same script there are multiple calls to create sockets
to different servers and they complete successfully.

I'm kind of out of my element on the internals of perl here, but my
thoughts:
Is it trying to find IO::Socket::INET.pm in @INC in order to identify
the super class, which is IO::Socket?
Does that mean that @INC has somehow become 'corrupted' during runtime?

any other thoughts.
Or has perl's symbol table become corrupted so it doesn't know what
"new" is?


nothing fancy in the script:

use IO::Socket;
use IO::Socket::INET;

$sock = IO::Socket::INET->new(
                                                PeerAddr => $host,
                                                PeerPort => $port,
                                                Proto    => 'tcp',
                                               );

for reference : in INET.pm:
@ISA = qw(IO::Socket);
...
sub new {
    my $class = shift;
    unshift(@_, "PeerAddr") if @_ == 1;
    return $class->SUPER::new(@_);      ###   Here's line 32

}

6. can't locate method new IO::Socket::INET - Perl

7. incorrect errno/perror with IO::socket->new

 I have been having fits getting meaningful errno/perror values out of
Perl from Socket->new() and I am hoping someone here might know what
is going on. The errno/perror values seem to work fine on my Linux
servers but AIX, HP, and Sun are not giving me a correct failure
value. I have remote servers set up so that I can get various types of
failures (no daemon, firewall blocking, no route, etc.) and I can test
all these conditions with telnet to see what the perror string should
be, and my Linux servers match that.

 It should be just a matter of getting extern int errno in the
interpreter when I ask for it with $!, no?

 Here is my sample code :

------------------------------------------------

#!/usr/bin/perl

use IO::Socket;
use strict;

my($server_ip, $server_port) = ("172.16.18.96", "10");
my($timeout) = 5;
my($firsterr) = "";
my($socket);

print "\$! = $!\n";
$! = 0;
print "\$! = $!\n";

$socket = IO::Socket::INET->new(
        Proto    => "tcp",
        PeerAddr => $server_ip,
        PeerPort => $server_port,
        Timeout  => $timeout
);

if ( $firsterr eq "" ) { $firsterr = $!; }
print "\$! = $!\n";

if ( $socket ) {
        close $socket;
        print " OK : Port $server_port is OPEN\n";

else {
#       Other error detected
        print "   Use OS problem determination procedures to determine
the \n";
        print "   source of the perror (errno) string : \n";
        print "   \t $firsterr\n";
}

exit 0;


----------------------

 In this particular case (IP/port) on my subnet, this server exists
and has no process listening on the port so I fully expect to get
(some OS specific form of) a perror string : "connection refused".
This is what I get on all the OS's when I telnet to this server and
port, and is also what I get from this script on Linux. For the other
OS's I am getting errno/perror values that apparently do not have
anything to do with the fact there is no listening process :

OS / Ver	perl -v		result

RH WS2u1	v5.6.1		"Connection refused"
RH WS3u5	v5.8.0		"Connection refused"
RH AS4u3	v5.8.5		"Connection refused"
AIX 5.3         v5.8.0		"A system call received a parameter that is
not valid."
AIX 5.3		v5.8.2		"A system call received a parameter that is not
valid."
AIX 5.1		v5.6.0		"A system call received aparameter that is not
valid."
AIX 5.2		v5.8.0		"A system call received aparameter that is not
valid."
AIX 4.3         v5.005_03       "A file descriptor does not refer to
an open file."
AIX 4.3         v5.005_03       "A file descriptor does not refer to
an open file."
HP 11.11        v5.8.0          "Invalid argument"
HP 11.00	v4.0		<problem w/ use>
HP 10.20	v4.0		<problem w/ use>
Sun 5.7		v5.8.5		"Connection refused"
Sun 5.8         v5.005_03       "Bad file number"
Sun 5.9		v5.6.1		"Connection refused"
Sun 5.10	v5.8.4		"Connection refused"

 If I change nothing other than the IP and port values in the script
(to something that will allow a connection), I get a successful socket
creation. This is just an example of the script I am developing, the
original also writes to and reads from the socket, and that is working
splendidly on all the test boxes. I need to use the errorno/perror to
point the user towards what needs to be checked on the host, client,
or network if there is a failure however.

 Also, I have also noticed similar behavior opening ports sucessfully.
The above server had telnet running and I can open port 23
successfully but I get various inexplicable $! errors after the
new().

 Thanks, Brandon

8. New module Filesys::DfPortable & new version Filesys::Df