Using ModPerl to Keep Zencart and phpBB2 Hackers Away

I’ve seen a bunch of attempts to hack at my zencart admin area and an old phpBB2 install that is no longer there. Both of these don’t work, and just return a 404 error, but it’s kind of annoying to keep spending compute cycles on this sort of behavior. Besides, now that I have installed the statsd and graphite packages, I am set up to provide a graph of whatever I want. So why not track the hackers, lock them out, and provide a graph of activity at the same time? So here goes.

2016-12-28 edit: see my github repo at https://github.com/dminear/apache_hack_check

I also run mod_perl in the Apache2 server, so I wanted to hook in to the PerlAccessHandler and do the work. The following is the Apache2 config lines:

        PerlModule ModPerl::DanHandler
        <Location />
                #SetHandler perl-script
                PerlAccessHandler ModPerl::DanHandler
        </Location>

And the following is the Perl module that is in the /usr/local/lib/site_perl/ModPerl directory on my Ubuntu machine:

package ModPerl::DanHandler;
use strict;
use warnings;
use FileHandle;
use IO::Socket::INET;

use Apache2::Log;
use Apache2::RequestRec ();
use Apache2::Connection ();

use Apache2::Const -compile => qw(FORBIDDEN OK :log);

BEGIN {
mkdir "/tmp/bad_ips";
chmod 0777, "/tmp/bad_ips"
}

my $sock = IO::Socket::INET->new(PeerPort => 8125,
				PeerAddr => '127.0.0.1',
				Proto => 'udp');

sub handler {
	my $r = shift;

	my $str = $r->connection->remote_ip();

	# if there is an attempt to access "zencart/admin", then put the ip on the block list
	if ($r->unparsed_uri() =~ /zencart\/admin$/ ||
		$r->unparsed_uri() =~ /zencart\/+admin\/+/ ||
		$r->unparsed_uri() =~ /phpbb2/i ) {
		#$r->log_error("BAD IP: $str");
		$sock->send( "hacker.unparsed_uri." . $r->unparsed_uri() . ":1|c\n" ) if defined $sock;
		my $fh = FileHandle->new( "> /tmp/bad_ips/$str");
		if (defined $fh) {
			print $fh "dummy";
			$fh->close;
		}
	}

	# check the block list
	if (-e "/tmp/bad_ips/$str") {
		$sock->send( "request.blocked:1|c\n" ) if defined $sock;
		return Apache2::Const::FORBIDDEN;
	} else {
		$sock->send( "request.allowed:1|c\n" ) if defined $sock;
		$sock->send( "request.hostname." . $r->hostname() . ":1|c\n" ) if defined $sock;
		return Apache2::Const::OK;
	}
}
1;

So now when someone comes in and tries to access /zencart/admin (or some gyrations thereof), the IP address gets stored in the tmp directory as a filename. On every request, the remote IP address is checked, and if found returns a 403 Forbidden response. The nice thing is that this happens for any request thereafter.  Because it’s early in the request stage, there’s not too much overhead. Plus I get the satisfaction of watching the banned IP addresses grow.

Then there’s some logic to update the statsd server based on good or bad requests. Here’s a screen capture of it in action (click on the image to enlarge):

Monitoring Graph

Processing Quicken QFX Files

I recently wanted to download my bank transactions from E*Trade, but they do not support Excel format for the bank statements (only brokerage).  You can, however, download 3 months of Quicken QFX data at a time.  I downloaded several files spanning the year-to-date, but then needed a program to parse up the files and output a tab delimited text file.  So, for a try in formatting code, here’s what I got:

#!/usr/bin/perl -w
# Dan Minear
# 2011-09-01
#
# process Quicken QFX files and output tab delimited text file
#
# call 

use strict;
use FileHandle;
use Data::Dumper;

my $data = {};
my $capture = 0;		# don't capture
my %txntypes = ();

if (@ARGV < 1) {
	die "Syntax:  $0 \nExample:  $0 09*.QFX\n";
}

my $fo = FileHandle->new( "> out.txt");
if (! defined $fo) {
	die "Cannot write file";
}

while (my $fname = shift) {
	print $fname . "\n";
	my $fi = FileHandle->new("< $fname");
	if (defined $fi) {
		my $txn = {};
		while(<$fi>) {
			chomp;
			chop;
			# look for  
			if (/^/) {	#start of record
				$capture = 1;
				$txn = {};
				next;
			}
			if (/<\/STMTTRN>/) {	# end of record
				$capture = 0;
				# add to data hash
				if (defined $data->{$txn->{FITID}}) { 
					print "FITID $txn->{FITID} already defined\n";
				} else {
					$data->{$txn->{FITID}} = $txn;
					$txntypes{$txn->{TRNTYPE}} = 1;
				}
			}
			if ($capture) {
				/<(\w+)>(.+)$/;
				$txn->{$1} = $2;	
			}
		}
	} else {
		die "cannot open $fname for reading";
	}
	$fi->close;
}

print "there are " . keys(%$data) . " transactions in files\n";

#print $fo Dumper( $data );
print $fo "DATE\tTYPE\tCREDIT\tDEBIT\tNAME\tMEMO\n";
foreach my $i (keys(%$data)) {
	my $t = $data->{$i};	# ref to hash
	my $date = $t->{DTPOSTED};

	if ($t->{TRNAMT} < 0) {	# it's a debit
		print $fo substr($date,0,4) . "-" . substr($date,4,2) . "-" . substr($date,6,2) . "\t" .
			$t->{TRNTYPE} . "\t" .
			"\t" .
			$t->{TRNAMT} . "\t" .
			$t->{NAME} . "\t" . 
			$t->{MEMO} . "\n";
	} else {	# it's a credit
		print $fo substr($date,0,4) . "-" . substr($date,4,2) . "-" . substr($date,6,2) . "\t" .
			$t->{TRNTYPE} . "\t" .
			$t->{TRNAMT} . "\t" .
			"\t" .
			$t->{NAME} . "\t" . 
			$t->{MEMO} . "\n";
	}
}

$fo->close;

=head1 uncomment to print out transaction types
foreach (keys(%txntypes)) {
	print $_ . "\n";
}
=cut

Automating my sprinklers

I have been spending a few hours creating a sprinkler controller.  The basic structure is to have a Perl program do the hard core logic, and then issue a webservice request to trigger the sprinkler controller on for a watering cycle.  So far, I have the embedded controller ready to interface to a hacked sprinkler controller .  The controller just takes a simple command to water a zone for so many seconds.  There is no higher level sequencing that is in place yet.  In reality, I probably need to send a sequence to the embedded controller so the power supply only powers one water solenoid at a time. The alternative is  to put that logic in the Perl program, so that’s where I am now.

I have been working on the Perl program to figure out my watering algorithm.  So far, I just make a request to a NOAA weather site for an airport  about 2 miles from me. The document is slurped as XML data that XML::Simple throws into a hash.  From there, I take the relative humidity and integrate the “dryness”,  or 100 minus the humidity, for the given time span.  I have decided to water my grass every 3 days as a ballpark until I see how things are working. The program just prints out a trigger message for now.  In the future, this will spawn off a sequence of webservice requests to water the lawn.

So far, we have been having some interesting weather.  There have been off-and-on rain storms coming through, and this week is setting up for a rapid change into hotter weather.  A weather description that matches “Rain” will cause a reset of the integration so the lawn is not watered.  The script logs some pertenant data to the end of the script so I can rerun a scenario in the future with real world data.

A side product of this project is to tie it to my Doggie Dumpcam, which captures realtime image changes on my front lawn.  The tree is growing bigger lately, which is creeping outside of the mask area and triggering a lot of image captures.  Some interesting things have been captured, including dogs, birds, and hornets.  This Perl program will have the capability to trigger a particular sprinkler zone between, say, the 6am and 8am hours for a small timespan.

Garden watering system

Garden with drip lines
Garden with drip lines

Over the Memorial Holiday, I finished installing an automatic watering system for the new garden. I planted Indian corn in the back row, and Cindy will be sowing some seeds in the other rows (hopefully today). I mixed in some garden soil into the rather clay-like soil, so we shall see if things actually grow. The most difficult part was digging the trench from the back of the house over to the wall.  It is only about 25 feet, but the ground is so hard that it takes a lot of effort to dig.  I had to presoak the ground which helped, but it was still hard.

Finished trench
Finished trench

This is the finished trench where it meets the house.  Luckly for me, there was a previous sprinkler from a time long past, so I was able to just use the outlet from the existing gate valve and run that over into the trench.  It is only a matter of time before the kids somehow manage to break that PVC pipe, so I want to put some backing underneath it.  On the other hand, maybe they will not mess with it.  Lucky thinking.

Supply side
Supply side

The next picture is a close up of the supply line from the hose outlet. There is a few inches of clearance below the pipe.  I also need to do something to prevent the sunlight from damaging the pipe.  I think a nice paint job of the sage house color should help.

Fixed Slab Leak

I finally got the leak contained by sealing off both ends of the pipes that used to go into the slab. Here’s the manifold near the front door where the water was being run from the slab back down in the slab to the kitchen.

front door manifold
front door manifold

I had to do the second smaller cutout to get more room for the soldering.front door manifold closeup You can see the cut and capped off pipe in the second picture below. I was lucky to find the supply line was a 3/4 inch pipe, and then a 1/2 inch teed off and went back down.  Since there was a 1/2 inch in the kitchen, then that was the one.  I would have had to guess otherwise.

The next set is where the pipe comes up for the kitchen and upstairs bathroom supply. kitchen manifoldkitchen manifold closeup The pipes on the left are cold water, and the hot water is on the right.  There is a previous cap from an earlier slab leak a few years ago, so now both pipes are capped. My job is the left solder work, which I think is a better job!

The next sequence of pictures is where I dug into the backside of the upstairs bathroom and teed in the hot water supply line.

Back of upstairs water lines behind sink
Back of upstairs water lines behind sink

It was kind of a tricky job getting the copper tee just right.  Basically you have to cut enough to be able to slide the thing in, but then slide if half a length up the other way and solder it.  I chose to connect this pipe to the upstairs bathroom lines with PEX tubing.  It’s a whole lot simpler than sweating copper pipes together.  You can see my wonderful amateur plumber work up close.

 

bathroom line after tee installation
bathroom line after tee installation

 

PEX tubing attached
PEX tubing attached
Close-up of the PEX tubing
Close-up of the PEX tubing

So that’s about it.  I still want to put up a picture of the  other side of the PEX tubing.  I just hacked into the existing hot line and even installed a ball valve!

Hot Water Pipe Slab Leak

It’s 3 days before Aidan’s First Communion, and reception afterwards at our house.  Last night I was hearing a water sound from the pipes, and yes, it’s a slab leak!  The hot water pipe under the entryway area seems to have a leak.  I still have to call the insurance and see what they can do. I expect nothing, so I think it will be time to dig into some walls and reroute the water pipes.  Conveniently, there were some water pipes run through the upstairs attic when the new bathroom was added during the addition.  I’m thinking those pipes are going to be supplying water to other areas of the house. Continue reading “Hot Water Pipe Slab Leak”

Santiago Trip – April 18th, 2009

dsc05059 Well, I was up at 6:00am to meet Chris and Brian in Lake Forest at 7am.  We started up via Maple Springs Road at 8am, and it was a nice drive.  The road was in good condition, and I always forget how long the road goes until the pavement ends. I also forget the smell of being outdoors.  I’m not sure if it’s all the sagebrush, but it just makes me breathe easier.  It’s hard to believe that I am still in Southern California when I’m traveling these mountains.

Here I am on top of the tower after lowering an antenna down.  We switched it out for a different antenna and redid the receive system.  It seems that the repeater is working better.  I know the transmit power has less reflected (50W forward, 1/2W reflected), and the receive is  a little more sensitive — it works better with a handheld.

The next picture is a pullback of the building and shows the layout.  The tower is a couple of stories tall, and this day it was bug free up there.

dsc05069That was not the case on the ground. I thought that it would be too cold, but I was wrong.  It was the busiest  I have ever seen on Santiago, both with the bugs and people.  I don’t know how many hikers, bikers, and joyriders there were.  When I was up on the tower, a motorcycle rider came up, got off his bike, and asked for Dan.  “What? Was that ITD?”  Yup, it sure was.  I have not seen Eric Robitaille for about 10 years! And now were talking on the top of Santiago.  Weird.  There was also a rescue going on to the north about 6 miles away on Skyline Road of a body in a car over the side.  While that was going on, a guy rolls up in a pickup to the emergency crew saying he just got bit by a rattlesnake.  There was a lot of coordination trying to figure out if they could helicopter the guy out.  I never found out how things ended.  When stuff happens, it happens.

How to increase gas milage

For a few years now before green was the thing, I’ve had this idea.  It seems kind of obvious to me, and I’m a bit puzzled why it has not happened.  See, the real solution for increasing vehicle MPG is not with the vehicle, it’s with the system. By my tracking for over 5 years (another topic for another post), my 4WD Isuzu Rodeo gets about 13 MPG in the city, and 22 MPG on the highway. Most cars are like that.  Our Toyota Sienna gets around 16 MPG city and 22 MPG highway (3 years worth of data).

The thing that really kills the MPG is stoplights.  You waste all the energy slowing down, then dump all the gas accelerating back up to the speed limit, where you hit another red light and the cycle repeats.  What’s really needed is a way to give the driver some feedback on what the optimum speed is to make the next green light.

So here’s how it works.  You cannot really change the stoplights — it just costs too much to synchronize them.  Sure it can be done, but then you have to squabble over city borders, etc. What you do is add a device to the existing signals.  So here goes.

The invention consists of the Monitoring Segment and the User Segment.  The Monitoring Segment consists of devices that are installed in existing stoplights that ideally interface with the signal timers and contain GPS receivers.  They are configured to know their nearest neighbor stoplights, the contact information for these stoplights, and the distance to those neighbor stoplights. They transmit their own timing information, and also the nearest neighbor contact information and distance.

By using CDMA technology, you can create efficient frequency reusage.  Heck, the whole thing could probably be implemented without any FCC registrations by using 802.11 or some ISM band. The important part is that by using a CDMA architecture, you can have low powered local signals that overlay eachother, streaming the contact information (code) for the next signal. To everyone else, including the incumbent user, there’s a higher noise floor.

The user segment is implemented as a receiver in the car.  This could be a built-in device, or an aftermarket add-on;  think GPS navigation device.  It  should have a GPS receiver anyway, as the car needs to know where it is in relation to the approaching stoplight.  The basic jist is that the car scans for a stoplight RF signal, locks when found, and begins to build a table of neighbors. As the car travels the city, it is picking up timing information for the next stoplight, and is far enough for the stoplight to transmit its timing information to the approaching vehicle.  By using the timing information and distance from the stoplight, the device can display the optimum speed that is necessary to make the next green light.

So there. If you do make a device, all I ask is that you send a receiver to me so I don’t have to think about these things while sitting at a red light.

And it’s not my fault my MPG is lousy.  See, the improvement has to come from the system side.  The government has to solve the problem. Heck, the have an obligation to solve this problem.  It’s probably cheaper than synchronization.

Another Long Ride

Back in Oct 2008 I took a business trip on my 2003 Yamaha YZF-600.

dsc04688

It was about a 400 mile round trip that started out at 7am and ended at 11:00pm with about 3 hour rides inside both ends.  I was trying to see how things would fair on a 600cc class street bike, and whether or not a long distance trip would be possible.

I remember for about 2 weeks after that my hands were just constantly numb and my fingers tingling.  I thought it was from the vibration — you kind of have that feeling in the feet when you go on a long ride, say 30 miles at a time.  However, this tingling just kept on going for days.

After doing some seaching, it appeared that I was just gripping the handlebars too tightly.  So I was interested in doing a long distance ride again, trying to stay loose and remain relaxed in the arms and hands.

So yesterday was the day.  Another 400 mile round trip, this time on my terms.  I started off at about 9am, cranked out 200 miles, and then arrived at my destination for a couple hours of work.  I didn’t seem to have any problems with my feet or hands.  In fact, the only problem was my skrunched legs. I started back on the road at 2pm and took the back way through Buellton, Solvang (I’m not a fan), and then stopped at the Gainey Vineyard (I am a fan) to pick up a couple bottles of wine.  Picked up highway 154 east past Lake Cachuma, back over the hills to Santa Barbara, then continued home. Today all is fine, except for some sore inner thighs — lots of peg weighting and tank pushing with the legs. But the good news: no hand tingling at all!

FEMA Training Online

This is a mental note for me. For RACES (Radio Amateur Civil Emergency Service) and EOC (Emergency Operations Center) training, take the FEMA IS-100, IS-200, and IS-700 courses available at ISP Courses. This is a introduction to the Incident Command System and such. It might even be more, but I have not taken the training yet.