 |
| Introduction |
|
Scope is a procedure engine for Xmail ESMTP server written in PERL.
It provides XMail with an out of process facility for running perl driven
filters, mailprocs, cmdaliases and custom domain mail processing in a
persistent, high-performance perl environment. Scope is designed to be
fully platform independent and can be installed as a Win32 service out
of the box.
|
| |
|
Via a minimal Scope client, XMail communicates with the Scope engine over
the TCP/IP network layer, meaning Scope is not limited to the same box as
XMail at all. Although it can theoretically sit on any box connected to
the internet, the idea behind this is that you can distribute the often
resource intensive filter/mailproc processing to a Scope enabled backend
deep inside your LAN and leave the XMail box upfront dedicating its
CPU-cycles on providing SMTP/POP3 services only.
|
| |
|
Scope procedures can be added or removed in a true plug n' play manner,-
to bring in new functionality and by using the exposed API anyone with
a fair understanding of writing a PERL module can easily knock up a new
procedure with a just few lines of code and hook it right in and have
XMail start using it pretty much the same way you would add a local
filter-script.
|
| |
|
Performance. With Scope two of the major performance bottlenecks when
dealing with perl has been cut off; Loading the perl interpreter for each
initiation of a script and converting it into internal perl bytecode before
the actual execution can take place. Scope itself while self hosted by a
perl-process, it exposes means to dynamically import external code
(procedures) into the its own process, storing them in semi-compiled form
and reusing them over and over again throughout the life-span of the server.
This approach actually makes procedures executed from the Scope procedure
cache nearly equivalent in performance as a compiled language.
|
| |
|
| Prerequisites |
|
Until we have managed to put together a CPAN/PPM style distribution
You gotta see to that the following requirements are fulfilled before
you can Scope up and running. We recommend you use the CPAN (PPM for
ActiveState Win32 PERL distribution) for a quick module installation
|
| |
|
First off, RAM. A fairly normal Scope process will occupy up to ~25 Mb RAM,
so make sure your machine got some to spare.
|
| |
| Perl 5.6.0 or newer is recommended |
|
|
| |
| Win32 Note: ActiveState's PERL |
|
If you intend to install and run the server as a Win32 service you
need three external dependencies, module Archive::Zip, module
Win32::Daemon by Roth Consulting and the srvany.exe utility from
Microsoft. However, you don't need to bother about installing these
manually since the server is kind enough to download and install
these from the respective author's ftp server if they arent present
already. Just issue the -install switch and follow the instructions
and you'll have the service up and running in a minute. If your box
has an internet connection up that is.
|
| |
|
| Installing |
For now, just extract the archive into the [MAIL_ROOT]/bin directory.
The server is just vanilla PERL so you should be getting it running right
away; Enter the newly created [MAIL_ROOT]/bin/scope directory and jump into
console mode and start up the Scope engine in debug mode.
This is done by giving it the -d[ebug] flag
#perl ./scoped.pl -d
|
| |
|
The server might die at first startup, complaining about missing modules.
Then I am afraid there's no shortcut, you'll need to install them before
you can continue. I highly recommend you use the CPAN module (PPM for Win32)
for installing the missing ones listed prerequiste modules mentioned above.
|
| |
|
Once you get them all in place re-issue the server start and now it should
be all go after you set a administrator email address and the MAIL_ROOT path
(needed, if it can't be resolved from the environment or registry (Win32))
A successfull startup will give a console crust looking something like this:
|
| |
debug: Scope Engine v1.00 initializing... [id: 0xDEADBEEF]
debug: ---------------------------------------
debug: Configuration loaded....... /var/.../scope/scopeconf.xml
debug: Logging enabled............ /var/.../scope/scope_log
debug: XMail MAIL_ROOT directory.. /var/MailRoot
debug: Procedure root directory... /var/.../scope/procroot
debug: Local temp directory....... /tmp
debug: Remote temp directory...... None
debug: Regeneration interval...... 24 hours
debug: Administrator.............. postmaster@bogusdomain.tld
debug: Preloading procedure.....ok vanguard
debug: Preloading procedure.....ok spamassassin
debug: Preloading module........ok MIME::Parser
debug: TCP socket ready........... 127.0.0.1:7887
debug: --------------------------------------
|
| |
|
If Scope has problems resolving a path or comes across any malformalies,
it will report this and exit. Otherwise, the engine is basically go for
requests. You probably want to tweak some of the main server settings or
any procedure specific configurations over time. These are all available
in the scopeconf.xml file, best edited in your favorite XML/text editor.
The configuration is dynamically reloaded upon change, so you don't
have to restart the server for the changes to take effect.
|
| |
| Ok, the server is running, now what? |
In order to talk with the server you'll need to utilize the supplied
client which can be found in the scope/client dir. The Win32 version comes
as a pre-compiled 4kb executable, if you are on another OS you will have to
compile the client from source. (see scope/client/src) Like so;
# cd ./client/src
# make -f Makefile.lnx
# cd ..
# ls -al
Currently there's only a makefile available for Linux, since I have no
access to OS'es besides that one currently. I was hoping someone outthere
using *BSD might be willing to supply additional makefiles...
|
| |
|
As an option here you may want to create a symbolic link to the scope client
executable, or if you are on Win32 copy 'scope.exe' into your windows system
folder. (or some other directory part of the path env varible) This way you
don't have to care about the full path the client every time you write a new
filter or mailproc tab, but just the name of the scope executable.
|
| |
|
The full path "/var/MailRoot/bin/scope/client/scope
...then can be replaced with just...
scope"
|
| |
To create a symlink you issue (depending on system and path setting)
# ln -s /var/MailRoot/bin/scope/client/scope /usr/local/bin/scope
|
| |
|
The client works a little different to what you might be used to, by the
fact it does not produce any visible output whatsoever. Huh, you might say..
Well, since the client is to be used by XMail and as far as XMail concerns
output means squat, the only thing important are exitcodes. Any application
you run, besides doing the job it's supposed to, goes with an code when it
ends, an exitcode. This code, if the application is executed from a parent
process can be picked up in order to determine if the child-process did ok
or not. If you are somewhat into the XMail documentation, then 96 thru 99
and 100 may ring a bell... those are exitcodes.
|
| |
|
In order to delegate a filter/mproc job (read procedure) the Scope engine,
the client is trigged by XMail in a vanilla "external" directive or as a
filter. The client when run, takes any supplied argument and reformats them
into a HTTP style URL basically, (not quite, but I wont into details here)
and then connects to the Scope server and passes the request uri over the
network. The server, as it sits listening on the same box, another box on
the local LAN or on the other side of the world, picks up this uri and
carries out the specific procedure requested and returns an integer
specifying the result to the client. The client inturn then simply makes
the response code into its own exitcode and thereby let XMail now how
to proceed. It's simple really, your webrowser does this every time you
request a document from the internet. The only thing that differs is the
lack of a visible output in the former case. However, as I said before,
XMail couldn't care less about output, the only thing that matters
are exitcodes.
|
| |
Now, a good idea to begin with, before involving XMail is to make sure
your Scope server is answering calls from the client. You can still run
the client independendly in order to make sure the connection works etc.
To be able to view the output though, you need to wrap the client call
inside a host application in order to catch the exitcode. And while we're
at it why don't use PERL for the task. To make things comfortable I have
supplied two wrapper scripts for this purpose "wrap.pl" and "loop.pl"
(found in scope/client). So if you haven't yet, I suggest you check it
out as it may makes a good view how the client/server interaction works.
If you have started the server in debug mode, you may run the "wrap.pl"
script to trig a procedure manually. The script takes three arguments,
in order,all optional:
- A procedure-path (default : "/")
- IP or name of server to connect to. (default : 127.0.0.1)
- Port -""- (default : 7887)
Example:
#perl ./wrap.pl /noop 192.168.0.1 666
|
| |
|
By switching to the server console/command prompt window, you'll see
detailed info of what came in, was being performed and what was sent
back to the client. On the wrap.pl side you can confirm this info.
The /noop procedure, as its name implies does nothing, a no-operation
is performed and exitcode 96 gets returned. The way for a filter to
tell XMail the message is ok as-is and nothing has been modified,
is by returning a noop-code, 96.
|
| |
| ...more to come... |
|
| |
|
| Configuring |
|
Scope uses a global configuration file in XML format for all server-settings
including any procedure level configurations. The supplied config-file comes
with most stuff pre-defined to get you started right away, but there are
settings that might need your attention though such as logging, temp dirs,
listening IP/port and what IP/host that are allowed to access the Scope service
etc. The configuration is stored dynamically meaning any modification is
immediately applied. This is very handy since you never have to restart the
server for the changes to take effect.
|
| |
|
All main server settings are documented inline for now on. Consult the
configuration file for details, but please read on here first...
|
| |
| Tag: init |
|
Here you can supply a list of modules and/or procedures to automatically
load at server start-up. It may significally improve the procedure
first-run performance, since the script in question is already compiled
into bytecode and cached at start-up and thus the overhead of
loading/interpreting at request is gone.
|
| |
| Tag: procedure-map |
|
This map holds one or more reserved section for each procedure, whereas in
each section you can finetune how and for whom the procedure should be
accessible and message max-size limits. There is also a 'config' sub-section
which is tied to the procedure API as sub-hashes, so you can access any
custom configuration settings from its corresponding procedure just
like a vanilla perl %hash.
|
| |
|
Although a reserved section is not required, most procedures explicitly
requires one.
|
| |
|
Each section is identified by its request path and is considered as a
namespace sortof. A procedure with a path of /some_procedure will conform
to the rules and settings by the section defined by a path attribute with
the same name. You can also define procedure redirections and alternative
configurations in situations where you might need multilple confs for a
specific procedure.
|
| |
| Access control |
Now with Scope eliminating the overhead of spawninig a new perl process
for each filter/mailproc call. Here you can with confidence create
domainwide tabs without concern of hogging down the CPU or HDD and instead
filter what users should access the procedure at the Scope level. Say you
want to apply filter Foo for all users at domain bogus.tld except for
recipient u1@bogus.tld and u2@bogus.tld, then you just need to create
a domainwide filter filters/bogus.tld.tab with a scope call for /Foo and
supply the @@RCPT macro. Secondly set up an access control for u1 and u2 like so;
<procedure path="/Foo" access-order="allow,deny"> <allow>.+@bogus.tld</allow> <deny>u1@bogus.tld</deny> <deny>u2@bogus.tld</deny> </procedure>
** Regexpressions are ok, so the rule creation limit is your imagination.
|
| |
| Dynamically Decline processing |
The max-msg-size attribute can prove a very good performance tweaker in
the case you use spamassassin for example. The job Mail::SpamAssassin
performs is a very resource intensive operation, and by the assumption
a spam message rarely exceeds the 256kb limit, why let SA have an vastly
expensive go at a 25MB message, while it with 99.999% probability can be
foretold the message is spam-ngative. With this setting you can avoid
this easily, by declining a spamscan action right at the server-level.
Following will instruct Scope not to let messages larger than 1024 kb through
to procedure /Foo, instead tell XMail to pass it on to the next handler
of any (exitcode 96).
<procedure path="/Foo" max-msg-size="1024">
</procedure>
In the case of declining messages with the means described above, the CPU
overhead is hardly measureable. A request for filter-procedure, declined
at the server level are handled in the matter of 10-tops milliseconds and
that includes the network transport between XMail and the Scope engine.
|
| |
| Tag: config |
This section is reserved for any procedure specific settings and
is accessible in real-time via the API method 'config'. Basically
any value can go in here as long can be stored in XML format, though
its recommended when writing a procedure to keep it as slim as possible.
The configuration structure is automatically converted into a hash table
so a config block layed out as this for example:
<procedure path="/Foo" max-msg-size="1024">
<config my-attribute="baz">
<my-list>
<my-value>foo</my-value>
<my-value>foo</my-value>
</my-list>
</config>
</procedure>
Is accessible via the procedure API as;
my $attribute = $self->config->{'my-attribute'};
...and in a list-context...
my @my_list = $self->config->{'my-list'};
print "my-value : $_" for (@my_list);
|
| |
|
| Procedure Reference |
|
|
| |
| |
Copyright(c) 2001-2005 Saltstorm.
All rights reserved.