[wplug] pnc history
bpmedley at 4321.tv
bpmedley at 4321.tv
Mon Aug 6 14:14:12 EDT 2001
Hi,
I have made a perl program to login to PNC's web page and retreive an
account's history. I have attached it in case someone else might find it
useful.
Unfortunately, it requires some perl modules that are usually not installed
by default. Even more unfortunate, is that I can't remeber the exact
modules that I needed to install to write it. Sorry. Here are a few that
I remeber:
LWP - Library for WWW access in Perl
AppConfig - Module for reading configuration files and parsing command
line arguments
Maybe Net::SSL for https urls.
If this is a problem for you, and you want to use this, please let me know
and I'll put some more time into figuring out what modules were required.
This program has the ability to output the history in both plain text and
qif format. The qif format is not tested. I don't have any good software
for linux that imports them. Therefore, I would greatly appreciate any
comments on this functionality.
You comments about the program are welcome. Please run
./grab_pnc_history.pl -man to figure out the usage.
~'`^`'~=-.,__,.-=~'`^`'~=-.,__,.-=~'`^`'~=-., \|/ (___) \|/ _,.-=~'`^`
Brian Medley @~./'O o`\.~@
"Knowledge is Power" brian.medley at verizon.net /__( \___/ )__\ *PPPFFBT!*
-- Francis Bacon `\__`U_/'
_,.-=~'`^`'~=-.,__,.-=~'`^`'~=-.,__,.-=~'`^`'~= <____|' ^^`'~=-.,__,.-=
~`'^`'~=-.,__,.-=~'`^`'~=-.,__,.-=~'`^`'~=-.,__,.-==--^'~=-.,__,.-=~'`^`
-------------- next part --------------
#! /usr/bin/perl -w
#
# This program will goto PNC's web page and retrieve the user's account
# history.
#
use strict;
use AppConfig;
use HTTP::Cookies;
use HTTP::Request::Common;
use HTML::Form;
use LWP::UserAgent;
use Pod::Usage;
# use Data::Dump qw(dump);
# use LWP::Debug qw(+);
my %cmdargs;
my $ua;
#
# An array of hashes:
#
# transactions[x]{date}
# transactions[x]{check_num}
# transactions[x]{withdrawal}
# transactions[x]{deposit}
# transactions[x]{desc} <- array of lines
#
my @transactions;
#
# hash describing the user's requested account
#
# account{type}
# account{url_snippet}
# account{number}
# account{balance}
#
my %account;
sub init;
sub get_history;
sub parse_history;
sub print_history_text;
sub print_history_qif;
MAIN:
{
init;
get_history;
no strict 'refs';
&{"print_history_$cmdargs{output}"};
use strict 'refs';
exit 0;
}
sub verify;
#
# Setup our environment
#
sub init
{
my $cookie;
my ($config, $filepth);
my $userid;
my @acceptable_output;
$filepth = "$ENV{HOME}/.grab_pnc_historyrc";
# don't print out error messages (internal to AppCongif):
# (like config file not found)
$config = AppConfig->new({ERROR => sub {;}, GLOBAL => {DEFAULT => 0}});
#
# print out error messages:
# $config = AppConfig->new({GLOBAL => {DEFAULT => 0}});
$config->define ("debug", {ARGS => "!"});
$config->define ("date", {ARGS => "=s"});
$config->define ("userid", {ARGS => "=s", DEFAULT => undef});
$config->define ("passwd", {ARGS => "=s", DEFAULT => undef});
$config->define ("output", {ARGS => "=s", DEFAULT => "text"});
$config->define ("account", {ARGS => "=s", DEFAULT => "Interest Checking"});
$config->define ("help");
$config->define ("man");
# give command line options precedence over config file
# the nobundling was required because -account wasn't working. don't
# know why...
$config->file($filepth);
$config->getopt(qw(nobundling), \@ARGV) or pod2usage(1);
pod2usage(1) if $config->help;
pod2usage(-verbose => 2) if $config->man;
pod2usage(-exitval => 1, -verbose => 1, -msg => "You must specify a userid") if not defined $config->userid;
pod2usage(-exitval => 1, -verbose => 1, -msg => "You must specify a passwd") if not defined $config->passwd;
# i like the ease-of-use of AppConfig, but I would rather access the options like
# variables, not subroutine calls.
%cmdargs = ();
%cmdargs = $config->varlist(".");
# the javascript would have stripped out the dashes and spaces
$userid = $config->userid;
$userid =~ s/-//g;
$userid =~ s/\s//g;
$config->userid ($userid);
@acceptable_output = ("text", "qif");
verify "output", $cmdargs{output}, @acceptable_output;
$cookie = './cookie_jar.txt';
unlink $cookie;
#
# setup our "browser"
#
$ua = LWP::UserAgent->new;
$ua->cookie_jar(
HTTP::Cookies->new(
file => "$cookie",
autosave => 1 )
);
$ua->agent('Mozilla/4.73');
} # end init()
#
# This routine makes sure that the value we are given is acceptable.
#
sub verify
{
my $option = shift;
my $value = shift;
my $is_ok;
$is_ok = "no";
#
# see if what the user gave us is allows
#
foreach (@_) {
if ($_ eq $value) {
$is_ok = "yes";
}
}
if ("yes" eq $is_ok) {
return;
}
# else the user specified an invalid argument
print "Value '$value' for the '$option' option is invalid.\n";
print "We currently support:\n";
foreach (@_) {
print "\t$_\n";
}
exit 2;
} # end verify()
sub parse_history;
#
# This subroutine goes to pnc's web page and dl's the user's history.
#
# It's highly sensitive to pnc's setup. One reason for this is that pnc uses javascript
# in their forms, which is not easily used in perl.
#
sub get_history
{
my ($request, $response);
my ($which_request);
my $form;
my $url;
my $acct_info;
my $i;
#
# login
#
if ($cmdargs{debug}) {
print "Trying to login to pnc with userid: $cmdargs{userid}.\n";
}
$request = HTTP::Request->new(GET =>
'https://www.accountlink.pncbank.com/logon.jsp?HttpLevel=128');
$response = $ua->request($request);
$form = HTML::Form->parse( $response->content, $response->base());
# give them the username/password
$form->value( 'UserID', "$cmdargs{userid}" );
$form->value( 'Password', "$cmdargs{passwd}" );
# the form does not have a submit button. it uses javascript. we have to add a submit
# button.
$form->push_input ( 'submit', {value=>"submit",name=>"submit"} );
$response = $ua->request( $form->click('submit') );
#
# get the accounts
#
if ($cmdargs{debug}) {
print "Retrieving information for your $cmdargs{account} account.\n";
}
$request = HTTP::Request->new(GET =>
'https://www.accountlink.pncbank.com/alservlet/DepositAccountListServlet');
$response = $ua->request ($request);
${$response->content_ref} =~ /($cmdargs{account}.*)\)\);/m;
die "Can't find account info for $cmdargs{account}.\n" if not defined $1;
$acct_info = $1;
$acct_info =~ s/'//g;
($account{type}, $account{url_snippet},
$account{number}, $account{balance}) = split /, /, $acct_info;
#
# get the history
#
if ($cmdargs{debug}) {
print "Obtaining history from pnc..\n";
}
$url = 'https://www.accountlink.pncbank.com/Accounts/Deposit';
$url .= '/depositDetail.jsp?selectedPage=0&accountID=';
$url .= $account{url_snippet};
$url .= '&More=INIT&Sort=DEFAULT&Page=0';
$request = HTTP::Request->new(GET => $url);
$response = $ua->request ($request);
parse_history $response->content_ref;
} # end get_history()
#
# This routine takes the raw history data from pnc (i.e. the HTML) and turns each entry
# into an element in our transactions array.
#
sub parse_history
{
my $history_data = shift;
my $entry;
my $date;
my $i;
while ($$history_data =~ /document.writeln\(buildRow\((.*)\)\);/mg) {
$entry = $1;
$entry =~ s/'//g;
# convert dates into iso 8601 date format
$entry =~ s/(.*?),\s*(.*)/$2/;
$date = $1;
$date =~ m#(\d\d)/(\d\d)/(\d\d\d\d)#;
$date = "$3-$1-$2";
# only give the user the date range they want
if ($cmdargs{date}) {
return if $date lt $cmdargs{date};
}
# initialize our data structure
$i = scalar @transactions;
$transactions[$i] = {};
$transactions[$i]{desc} = [];
# store the values in our data structure
$transactions[$i]{date} = $date;
($transactions[$i]{check_num}, $transactions[$i]{withdrawal}, $transactions[$i]{deposit},
$transactions[$i]{desc}[0], $transactions[$i]{desc}[1]) = split /, /, $entry;
# if there was only one line in the description
$#{$transactions[$i]{desc}}-- if $transactions[$i]{desc}[1]=~ /\&\#160/;
# tidy up check output
$transactions[$i]{desc}[0] =~ s/^CHECK\s*(\d+)\s+(\S+)/CHECK $1 $2/;
} # end parsing the html from pnc
} # end parse_history()
#
# This routine prints out the transactions array as text.
#
sub print_history_text
{
my $line;
foreach (@transactions) {
# we don't print check_num b/c it's in the description
printf "%s %-46s%-15s%-s\n", $_->{date}, $_->{desc}[0], $_->{withdrawal}, $_->{deposit};
# if the description is multi-line this takes care of it
foreach $line (1 .. $#{$_->{desc}}) {
print " $_->{desc}[$line]\n";
}
# this is so the user can add their own description. the spaces are so they can
# hit 'A' in vi and be lined up with the others.
print " \n";
}
} # end print_history_text()
#
# This routine generates a qif file from our transactions array. It's not tested a
# great deal, because I can't find a good program in Linux that will import qif
# files.
#
# example for qif files:
# http://www.intuit.com/quicken/technical-support/quicken/old-faqs/dosfaqs/60006.html
#
sub print_history_qif
{
my $QIF = "./output.qif";
my $orig_fh;
my $amt;
open QIF, "> $QIF" or die "Opening $QIF: $!";
$orig_fh = select;
select QIF;
foreach (@transactions) {
print "!Type:Bank\n";
print "D$_->{date}\n";
$_->{check_num} =~ s/\s*//g;
if ($_->{check_num} =~ /^\d+$/) {
print "N$_->{check_num}\n";
}
$amt = $_->{withdrawal};
if ($amt =~ /^\$/) {
# we have a withdrawal
$amt =~ s/\$/-/;
} else {
# we have a deposit
$amt = $_->{deposit};
$amt =~ s/\$//;
}
print "T$amt\n";
print "P$_->{desc}[0]\n";
print "M$_->{desc}[1]\n" if defined $_->{desc}[1];
print "^\n";
}
close QIF;
select $orig_fh;
} # end print_history_qif()
__END__
=head1 NAME
grab_pnc_history.pl - Goto PNC's web page and retrieve the user's account history
=head1 SYNOPSIS
B<grab_pnc_history.pl> [options]
=head1 OPTIONS
=over 4
=item B<-help>
Prints some help and exits.
=item B<-man>
Prints the manual page and exits.
=item B<-date>
Used to specify an ending date when producing output. No transaction after
this date will be output.
date should be specified in the ISO 8601 format (e.g. 2001-08-04).
=item B<-debug>
Prints some (hopefully) helpful messages during execution to figure out
what's going on.
=item B<-userid>
The user id you login to pnc with.
=item B<-passwd>
The password for that user id.
=item B<-output>
What format you want the output in. Currently we accept text and qif.
The default is text. qif will save the output to "output.qif" and overwrite
what is already there.
=item B<-account>
The account you want to get information for. The default is "Interest Checking".
Known values are:
Interest Checking
Regular Checking
Statement Savings
=back
=head1 DESCRIPTION
This program will goto PNC's web page and retrieve the user's account history.
=head1 CONFIGURATION
This program accepts information from the command line and from a
configuration file. The command line overrides values placed inside the
configuration file. At present the configuration file is:
~/.grab_pnc_historyrc
An example is:
# my account
userid = 478661344
passwd = 1234
account = Interest Checking
More generally, this is:
option [=] [value]
Where "option" is a command line option. This allows the user to specify
their own defaults and makes sure that the password does not show up in ps(1)
and the like.
NOTE: Passwords are stored in plaintext.
=head1 EXAMPLES
Use the values in the configuration file and get "Statement Savings" history.
grab_pnc_history.pl -account="Statement Savings"
Use the values in the configuration file and get "Statement Savings" history.
Send the output to the screen. However, only transactions that happened
AFTER and INCLUDING Wed, August 15th, 2001 will be printed.
grab_pnc_history.pl -account="Statement Savings" -output=text -date=2001-08-15
Use the values in the configuration file and get "Statement Savings" history.
Send the output to a qif file.
grab_pnc_history.pl -account="Statement Savings" -output=qif
=head1 NOTES
This program needs the following perl modules (which, to the best of my
knowledge are not installed by default):
LWP - Library for WWW access in Perl
AppConfig - Module for reading configuration files and parsing command line arguments
(there are probably others, but I've forgotten what I had to install)
I don't know the minimum perl version that is usable. I use 5.6.0.
Don't forget that the qif file will be overwritten.
In addition to everything said above, this program also uses a file called
./cookie_jar.txt. As you might expect, this is used to store cookies that pnc
gives us. However, as you might not expect, this file is deleted at startup and
then re-created.
Therefore, make sure you don't have a file called cookie_jar.txt in the same
directory as this program. In the future, this should probably be a temporary
file.
This program prints out the message "Input 'UserID' is readonly at...". This has
something todo with the form options for UserID. I'm not really sure how to make
this go away. Suggestions are welcome.
=cut
More information about the wplug
mailing list