Intercepting, and Completing,
Requests for Third Party Services

Have you ever want to intercept a request for a third party service, write the information to a file, and forward the original request to the service provider? This tutorial will describe the use of a Perl script to do just that. We'll use Netmind's URL minder as an example.

URL Minder is a great service. It keeps track of Web pages (and other Internet resources) and sends you e-mail when the pages you register change. The URL-Minder is your own personal Web robot: it keeps tabs on pages that are important to you, so you can spend your time doing other things! For more information check out their home page:

Net Mind Home Page

I have a What's New page on most of the sites I manage. As the name implies, new and modified items are announced on this page. If a viewer registers the What's New page they know whenever a change to the site is implemented. I supply the form, the viewer supplies their e-mail address, and Net Minder does the work.

However, I would like to know the viewers e-mail address -- and other information -- when they register the page. This is accomplished with a two part process: the form and a Perl script.

A Practical Example

Looks just like what one would normally expect. However, click on the submit button doesn't send the user's e-mail address to Netmind, it send it to a Perl script for processing. View the actual HTML with comments.

Next the actual Perl script:

#!/usr/bin/perl

# This is program url-mind.pl Version 0.9.
# Created and copyright 1997 by Urb LeJeune, lejeune@usats.com
# Initially created on 7/20/97. May only be used with permission.
# Last modified July 20, 1997.

# The following is the path -- relative to your cgi directory --
# of the file to store the information.

$DataFile = "data/url-mind.dat";

# The following is the destination URL.
$URL_MINDER = "http://www.netmind.com/cgi-bin/uncgi/url-mind?";
$DATE = `date +"%B %d, %Y"`;
chop($DATE);

# Read in and parse user supplied e-mail address and hidden variable
&Parse;

$NEW_URL = $URL_MINDER . $in{'QUERY'}; # QUERY is original parameter string

&SaveInformation($QUERY_STRING); # Write user info to file or whatever

print "Location: $NEW_URL\n\n";  # Send URL off to original location.

exit;

sub SaveInformation{
# Do whatever you like with user information in this subroutine.
   $EMAIL= $in{'required-email'};
   $URL =  $in{'url'};
   open (DATA, ">>$DataFile"); #  Open data file for appending
   # I saving the supplied URL (more than one form being processed)
   # date of entry, user's qualified domain name,
   # calling document, and browser type.
   print DATA "$URL $EMAIL $DATE $ENV{'REMOTE_HOST'} " .
             "$ENV{'HTTP_REFERER'} $ENV{'HTTP_USER_AGENT'}\n";
   close (DATA);
} # End of subroutine SaveInformation

sub Parse {
  # Slightly modified version of ReadParse from cgi-lib.pl
  # It's needed to save original query string.
  local (*in) = @_ if @_;
  local ($i, $key, $val);

  # Read in text
  if ($ENV{'REQUEST_METHOD'} eq "GET") {
     $QUERY = $ENV{'QUERY_STRING'};
  } elsif ($ENV{'REQUEST_METHOD'} eq "POST") {
     read(STDIN,$QUERY,$ENV{'CONTENT_LENGTH'});
  }

  @in = split(/[&;]/,$QUERY);

  foreach $i (0 .. $#in) {
    # Convert plus's to spaces
    $in[$i] =~ s/\+/ /g;

    # Split into key and value.
    ($key, $val) = split(/=/,$in[$i],2); # splits on the first =.

    # Convert %XX from hex numbers to alphanumeric
    $key =~ s/%(..)/pack("c",hex($1))/ge;
    $val =~ s/%(..)/pack("c",hex($1))/ge;

    # Associate key and value
    $in{$key} .= "\0" if (defined($in{$key})); # \0 is the multiple separator
    $in{$key} .= $val;

  }
  $in{'QUERY'} = $QUERY; #Save original query string in @in array
  return scalar(@in);
} # End of subroutine Parse

Downloadable Perl Source

Change the path and name of the file to hold the user information in the first executable statement.

$DataFile = "data/url-mind.dat";

A call to subroutine SaveInformation writes to the data file.

print DATA "$URL $EMAIL $DATE $ENV{'REMOTE_HOST'} " . "$ENV{'HTTP_REFERER'} $ENV{'HTTP_USER_AGENT'}\n";

These are the URL being monitored, the requester's e-mail address, the time of the request, and the user's host, referring document, and browser. Fell free to modify the format as you see fit.

This is a beta release. Any feedback would be appreciated via e-mail