Nothing beats Perl to solve a data manipulation requirement quickly

Archive - Originally posted on "The Horse's Mouth" - 2009-04-14 06:54:58 - Graham Ellis
OK - I admit it. It might be slightly out of fashion but I still love Perl for that quick hack script which is a one-off data manipulation need.

I have two files - the first containing records of who signed our pledge for an appropriate TransWilts train service last year, and another containing records in the same format for those who have signed this year, and I wanted a list of people who haven't reconfirmed their support.

Here's a "break the rules of good programming" piece of code that I slapped together in a few minutes ...

open (FH,"08.cooked");

while (<FH>) {

  chop;

  $key = lc;

  $signed{$key} = 1;

  $lastyear{$key} = 1;

  }

  

open (FH,"09.cooked");

while (<FH>) {

  chop;

  $key = lc;

  $signed{$key} = 2;

  }

  

foreach $previous (keys %signed) {

  # last year only

  if ($signed{$previous} == 1) {

    print "$previous\n" ;

    $awaited += 1;

    }

  if ($signed{$previous} == 2) {

    # this year only

    $newcomer += 1 if (! $lastyear{$previous}) ;

    # this year and last year

    $both += 1 if ($lastyear{$previous}) ;

    }

  }

print ("$awaited last year only\n");

print ("$both both years\n");

print ("$newcomer new this year\n");

... and after a couple of syntax errors and a couple of validation tests, it ran for me and fulfilled its job.

Coding rules broken.

1. Cut and paste the file read code - un-necessary duplication
2. Poorly (un-named!) variable - use of $_
3. Combined data and report output of STDOUT needed cutting about later
4. Lack of descriptive comment block

but job done, and I can move on with my once-only result set and take a look through the output, considering and acting on the data provided which was the purpose in the first place.

P.S. Had I been working from database tables, a left join looking for orphan records would have let me do the whole data thing as a one-liner:

mysql> select pledges.email from pledges left join pledge09 on pledges.email = pledge09.email where pledge09.pid is NULL;

with slight modifications for each of the counts

We offer courses in both Perl Programming and in MySQL. One of the most popular pages on our web site explains the difference between the types of join in MySQL and how to extract all the different data combinations you might want.

Main Content

Nothing beats Perl to solve a data manipulation requirement quickly