Patrick Michael Kane's Technical Diary

This is a collection of technical discoveries and musings. It's useful to me as a reference source, maybe it'll be useful for you too.

Fri Aug 25 13:18:25 2000 Need detailed resource accounting for a Linux system? Check out the Beancounter patch which adds detailed accounting to the Linux kernel. Tasty!

Thu Jul 27 19:34:27 2000 No updates for a while. For folks looking for really, really speedy DNS resolution on web access logs (or anything else), check out Fastresolve. It uses ADNS, an async DNS stub resolver, and is blazingly fast. Highly recommended for large sites.

Thu Dec 30 21:58:59 1999 I've been toying around with Dents. Dents is a new DNS server, built to be modular and easily reconfigurable, without restarting the server. Making Dents pull zones from a SQL database, algorithmically generate zones, etc. is trivial, due to the well-defined module API. One can easily imageine a mod_perl-type integration for this DNS server.
While BIND is fantastic, and perhaps the new 9.0 release will be easier to modify than 4.x and 8.x, 9.0 is still a ways off. Unfortunately, it looks like Dents may be a dead project now -- No meaningful updates or discussion since October. I'm going to see what happened.

Wed May 26 10:28:10 1999 The end result of the row selection problem described below just went live at Cyberscratch. Terrible name, but I just do the architecture and code, so don't blame me.:-)

Wed May 26 10:26:40 1999 The first public release of lsh, a free ssh version 2 client/server, is available over at psst.

Mon Apr 19 20:30:58 1999 Released a new version of Authen::Smb today -- Hopefully squashing the last few bugs that were causing segfaults on odd platforms.

Wed Apr 14 10:18:23 1999 The San Francisco Perl Mongers are an interesting bunch.

Wed Mar 31 07:35:50 1999 An interesting RDBMS problem: How to quickly select a random row from a table, update that row in some way and return it to the user.
There are a number of situations where you might want to do something like this. You might have a table full of invoices that you want to ship from, a table full of lottery cards that you want to hand out to users, etc.
It's actually quite a difficult problem, since you need to find an efficient way to hold a lock on the table, even though you are only doing a select. My first attempt at the problem used a while() loop and rand(), selecting a random ID at the top of the loop and then trying to update that row inside the loop. If @@rowcount equaled 1 after the update, we broke out of the while loop. Otherwise, we tried again.
This was cute, but suffered from a number of problems. First, the entire thing was massively non-deterministic. If we were really unlucky with our random number generator, the stored procedure might not return. Secondly, the solution had a tendency to not return the bottom rows in the table -- Since we were using a >= in the select to get the nearest ID to our picked random ID, IDs towards the bottom of the table would have a low chance of being picked.
After some extensive conversations with Anthony Mandic and some other folks from comp.database.sybase (My target database), I've come up with the following solution:
Create an update trigger on the table that returns the updated row. Then, just have our stored proc set rowcount 1 and issue an update on the table. This is a nice, simple solution. It has some concurrency-related performance issues (We are always going to be hitting the same page for the update requests), but it still peforms well (100+ transaction/sec on a PII/450 running Linux). It does break down when you have to scan the index fairly deeply to find a matching row to update, but we mitigate that problem by moving the rows into a historical table.

Sun Feb 21 11:42:21 1999 Another one from the "all the good ideas have already been coded" department. If you open up any sort of Internet access to the desktop, including through a web proxy, you've just potentially extended your network perimeter to the entire Internet. Lars from nocrew released httptunnel which runs under Linux (and would port easily to any other *NIX with decent PTY support) and gives you a two-way communication channel via http or proxied http. I successfully ran a PPP connection over a squid cache -- no problems. Security practitioners have known this was possible for a long time, but someone finally sat down and coded it.
I can't wait for this to get some broader attention. They'll be unauthorized network connections galore to exploit during A&P work.

Mon Feb 15 08:28:56 1999 When netbooting a sparc box attached to a Cisco Catalyst 5000 switch, you need to manually configure the Ethernet port the Sparc is connected to on the switch. The ethernet card in the sparc doesn't seem to do enough negotiation with just the PROM running to get the switch to believe that there is a box attached to it, so the box will never boot.

Sun Jan 31 18:54:47 1999 Damn, someone beat me to distributed dictionary password cracker. Check out Slurpie.

Sat Jan 30 09:54:44 1999 Buried in the Apache 1.3 support directory is a little program called 'ab'. It's an Apache benchmarking program and is fantastic for benchmarking and stress-testing systems. You can control the total number of requests and the concurrency.
If you need to do a mix of pages or anything more complex, check out crashme (same name, different function from the program that feeds syscalls random garbage). It's written in perl and uses LWP::ParallelUserAgent, so it's a bit slower than ab, but is quite useful for simulating an actual browser session (or 500 simultaneous browser sessions).

Sat Jan 30 09:51:00 1999 The *Log directives in the Apache config files do not honor 'None' to disable log output entirely. Instead, they create a logfile called None in your root partition and try to fill it up. You can point them to /dev/null, but the logging code path is still followed, using up CPU.

Mon Jan 18 20:38:52 1999 The idea behind granitecanyon.com is good, but they apparently have lots of problems with people attacking their servers and links. I don't think I'd recommend using them, unless you don't have any other alternative.

Wed Jan 13 19:17:33 1999 Duh. john the ripper already contains code to do this kind of algorithmic dictionary generation. It can do a resume, given the last password tested, so I can use the same internals to divide up the keyspace.

Wed Jan 13 17:25:16 1999 I started work today on a distributed dictionary cracking system, built on top of john the ripper. Dictionary distribution is a pretty hard problem, I've found. I guess Alec Muffet was right.
The main problem that I've run into so far is that CPUs are fast, so the network becomes a bottleneck for the CPU if you can't get dictionary information fast enough. I think a little compression will help out here greatly. A 1,000,000 word uncompressed dictionary, enough to keep a PII/300 busy for about a minute, takes well over a minute to transfer, even over the loopback. Since a mode like this would be most useful for incremental password cracking, pretransferring the entire dictionary isn't a real option. Plus, we'd like to be able to have clients come in and drop out at will.
I wonder how distributed.net handles the dictionary problem ... do they have the entire keyspace in a database somewhere? That would be a fairly large amount of data.
Hmm. On the other hand, if we are only interested in doing incremental cracking, it might be possible to distribute a "block", ala distributed.net. Instead of transferring the entire dictionary, just transfer a range. So you'd give one client [0-9] through [0-9][0-9][0-9][0-9], etc. Just break up your keyspace into arbitrary blocks and go nuts. That might work.

Wed Jan 13 12:38:55 1999 Granite Canyon provides free primary and secondary DNS services. The entire thing is web-based, with MAIL-FROM confirmation to any changes that are made.
One of the guys over there, Steve Senator, was extremely helpful in resolving a problem when someone used Granite Canyon's free service to try and do something nefarious. Thanks Steve! Check these folks out.

Tue Jan 12 11:58:30 1999 You cannot change the size of a Linux ramdisk if you've compiled ramdisk support as a module.

Mon Jan 11 21:21:49 1999 One last tidbit:
Disappearing web pages. So you've got a portion of a website, where some pages should only be accessible by a subset of your userbase. Pick your authentication method (Basic, MD5, personal certs, fingerprints, retinal scan, whatever).
Typically you'll give your users a slap on the hand when they go someplace they shouldn't, with a 'authorization failed' or 'you're not allowed here'. But this gives away that there is even an authenticated area to get to, encouraging the curious.
Instead, why don't we just serve up 404 pages? I just did this with HTML::Mason templates, but you could do it with anything. If the user presents the correct credentials, let them in, otherwise pass a 404 up through the server to the client. The page doesn't exist, unless you are the right person!

Mon Jan 11 21:14:45 1999 When building HTML::Mason components, you have to be real careful to use <%perl_init>, not <%perl>, when you have components that shouldn't produce any output, just return values.
I hunted down a bug today in HTML::Mason that was causing it to return duplicate headers in some situations, only to discover that I was using a <%perl> section in one of my components that should have used a <%perl_init>. The component produced some output and sent a header to the client, so my real header became visible in the URL. No bug in HTML::Mason, just a bug in the programmer.

Mon Jan 11 21:01:51 1999 I don't think enough people realize how useful the Apache 1.3.x log filtering system is. Unlike NCSA and previous versions of Apache, you can safely put a filter program between the Apache server and the access_log file, without worrying about the process dying and screwing up your server or losing data.
I used this filtering for a little download counter project. I needed to be able to access the download count for a group of files under a certain directory in real-time. I considered writing a mod_perl module to do it, which would have been easy, but also would have bloated my httpd's on this mod_perl-less server. Instead, with half a dozen lines of code, I'm able to safely tally this stuff. Awesome!

Mon Jan 11 18:03:18 1999 If you are using mod_php3's mail() function and qmail together, you need to be sure to set MAILUSER and possibly MAILHOST so that qmail-inject correctly sets your Return-Path and envelope From.

Mon Jan 11 16:07:35 1999 Must have software: A Java ssh client, with source, that is GPL'd. The terminal emulation on this thing is great. It emulates an xterm-color almost perfectly and its vt102 emulation seems flawless. Perfect for getting access to your systems securely when a native ssh client isn't available.
They also have a java ssh server over there, although it only does port forwarding and isn't GPL'd. Source is available.

Mon Jan 11 13:40:46 1999 Finished reading the Perl Cookbook a few days ago. Most of it is rehash and recipes that any decent perl hacker would invent on their own, but the object oriented/module sections were useful and I learned this cute bit of perl syntactical sugar:
@hash{keys %hash} = ();
Which will initialize all of %hash's keys. This is cuter than it seems, because you can put anything in the braces, which will save you a foreach loop here and there.

Mon Jan 11 12:39:37 1999 Discovered a bug in HTML::Mason today. If you tell your handler.pl to ignore everything except text, to avoid inadvertantly trying to parse binary data and screwing everything up, you also ignore requests with no mime-type at all.
This means that HTML::Mason will never process requests that would normally give you a 404 error, which breaks dhandlers. I just wrapped the regex that checks the mime-type in an if statement that checks to see if there is a mime-type at all. Seems to work. I've submitted this to Jon Swartz.


Fri Aug 25 13:18:25 2000	Need detailed resource accounting for a Linux system? Check out the Beancounter patch which adds detailed accounting to the Linux kernel. Tasty!

Thu Jul 27 19:34:27 2000	No updates for a while. For folks looking for really, really speedy DNS resolution on web access logs (or anything else), check out Fastresolve. It uses ADNS, an async DNS stub resolver, and is blazingly fast. Highly recommended for large sites.

Thu Dec 30 21:58:59 1999	I've been toying around with Dents. Dents is a new DNS server, built to be modular and easily reconfigurable, without restarting the server. Making Dents pull zones from a SQL database, algorithmically generate zones, etc. is trivial, due to the well-defined module API. One can easily imageine a mod_perl-type integration for this DNS server. While BIND is fantastic, and perhaps the new 9.0 release will be easier to modify than 4.x and 8.x, 9.0 is still a ways off. Unfortunately, it looks like Dents may be a dead project now -- No meaningful updates or discussion since October. I'm going to see what happened.

Wed May 26 10:28:10 1999	The end result of the row selection problem described below just went live at Cyberscratch. Terrible name, but I just do the architecture and code, so don't blame me.:-)

Wed May 26 10:26:40 1999	The first public release of lsh, a free ssh version 2 client/server, is available over at psst.

Mon Apr 19 20:30:58 1999	Released a new version of Authen::Smb today -- Hopefully squashing the last few bugs that were causing segfaults on odd platforms.

Wed Apr 14 10:18:23 1999	The San Francisco Perl Mongers are an interesting bunch.

Wed Mar 31 07:35:50 1999	An interesting RDBMS problem: How to quickly select a random row from a table, update that row in some way and return it to the user. There are a number of situations where you might want to do something like this. You might have a table full of invoices that you want to ship from, a table full of lottery cards that you want to hand out to users, etc. It's actually quite a difficult problem, since you need to find an efficient way to hold a lock on the table, even though you are only doing a select. My first attempt at the problem used a while() loop and rand(), selecting a random ID at the top of the loop and then trying to update that row inside the loop. If @@rowcount equaled 1 after the update, we broke out of the while loop. Otherwise, we tried again. This was cute, but suffered from a number of problems. First, the entire thing was massively non-deterministic. If we were really unlucky with our random number generator, the stored procedure might not return. Secondly, the solution had a tendency to not return the bottom rows in the table -- Since we were using a >= in the select to get the nearest ID to our picked random ID, IDs towards the bottom of the table would have a low chance of being picked. After some extensive conversations with Anthony Mandic and some other folks from comp.database.sybase (My target database), I've come up with the following solution: Create an update trigger on the table that returns the updated row. Then, just have our stored proc set rowcount 1 and issue an update on the table. This is a nice, simple solution. It has some concurrency-related performance issues (We are always going to be hitting the same page for the update requests), but it still peforms well (100+ transaction/sec on a PII/450 running Linux). It does break down when you have to scan the index fairly deeply to find a matching row to update, but we mitigate that problem by moving the rows into a historical table.

Sun Feb 21 11:42:21 1999	Another one from the "all the good ideas have already been coded" department. If you open up any sort of Internet access to the desktop, including through a web proxy, you've just potentially extended your network perimeter to the entire Internet. Lars from nocrew released httptunnel which runs under Linux (and would port easily to any other *NIX with decent PTY support) and gives you a two-way communication channel via http or proxied http. I successfully ran a PPP connection over a squid cache -- no problems. Security practitioners have known this was possible for a long time, but someone finally sat down and coded it. I can't wait for this to get some broader attention. They'll be unauthorized network connections galore to exploit during A&P work.

Mon Feb 15 08:28:56 1999	When netbooting a sparc box attached to a Cisco Catalyst 5000 switch, you need to manually configure the Ethernet port the Sparc is connected to on the switch. The ethernet card in the sparc doesn't seem to do enough negotiation with just the PROM running to get the switch to believe that there is a box attached to it, so the box will never boot.

Sun Jan 31 18:54:47 1999	Damn, someone beat me to distributed dictionary password cracker. Check out Slurpie.

Sat Jan 30 09:54:44 1999	Buried in the Apache 1.3 support directory is a little program called 'ab'. It's an Apache benchmarking program and is fantastic for benchmarking and stress-testing systems. You can control the total number of requests and the concurrency. If you need to do a mix of pages or anything more complex, check out crashme (same name, different function from the program that feeds syscalls random garbage). It's written in perl and uses LWP::ParallelUserAgent, so it's a bit slower than ab, but is quite useful for simulating an actual browser session (or 500 simultaneous browser sessions).

Sat Jan 30 09:51:00 1999	The *Log directives in the Apache config files do not honor 'None' to disable log output entirely. Instead, they create a logfile called None in your root partition and try to fill it up. You can point them to /dev/null, but the logging code path is still followed, using up CPU.

Mon Jan 18 20:38:52 1999	The idea behind granitecanyon.com is good, but they apparently have lots of problems with people attacking their servers and links. I don't think I'd recommend using them, unless you don't have any other alternative.

Wed Jan 13 19:17:33 1999	Duh. john the ripper already contains code to do this kind of algorithmic dictionary generation. It can do a resume, given the last password tested, so I can use the same internals to divide up the keyspace.

Wed Jan 13 17:25:16 1999	I started work today on a distributed dictionary cracking system, built on top of john the ripper. Dictionary distribution is a pretty hard problem, I've found. I guess Alec Muffet was right. The main problem that I've run into so far is that CPUs are fast, so the network becomes a bottleneck for the CPU if you can't get dictionary information fast enough. I think a little compression will help out here greatly. A 1,000,000 word uncompressed dictionary, enough to keep a PII/300 busy for about a minute, takes well over a minute to transfer, even over the loopback. Since a mode like this would be most useful for incremental password cracking, pretransferring the entire dictionary isn't a real option. Plus, we'd like to be able to have clients come in and drop out at will. I wonder how distributed.net handles the dictionary problem ... do they have the entire keyspace in a database somewhere? That would be a fairly large amount of data. Hmm. On the other hand, if we are only interested in doing incremental cracking, it might be possible to distribute a "block", ala distributed.net. Instead of transferring the entire dictionary, just transfer a range. So you'd give one client [0-9] through [0-9][0-9][0-9][0-9], etc. Just break up your keyspace into arbitrary blocks and go nuts. That might work.

Wed Jan 13 12:38:55 1999	Granite Canyon provides free primary and secondary DNS services. The entire thing is web-based, with MAIL-FROM confirmation to any changes that are made. One of the guys over there, Steve Senator, was extremely helpful in resolving a problem when someone used Granite Canyon's free service to try and do something nefarious. Thanks Steve! Check these folks out.

Tue Jan 12 11:58:30 1999	You cannot change the size of a Linux ramdisk if you've compiled ramdisk support as a module.

Mon Jan 11 21:21:49 1999	One last tidbit: Disappearing web pages. So you've got a portion of a website, where some pages should only be accessible by a subset of your userbase. Pick your authentication method (Basic, MD5, personal certs, fingerprints, retinal scan, whatever). Typically you'll give your users a slap on the hand when they go someplace they shouldn't, with a 'authorization failed' or 'you're not allowed here'. But this gives away that there is even an authenticated area to get to, encouraging the curious. Instead, why don't we just serve up 404 pages? I just did this with HTML::Mason templates, but you could do it with anything. If the user presents the correct credentials, let them in, otherwise pass a 404 up through the server to the client. The page doesn't exist, unless you are the right person!

Mon Jan 11 21:14:45 1999	When building HTML::Mason components, you have to be real careful to use <%perl_init>, not <%perl>, when you have components that shouldn't produce any output, just return values. I hunted down a bug today in HTML::Mason that was causing it to return duplicate headers in some situations, only to discover that I was using a <%perl> section in one of my components that should have used a <%perl_init>. The component produced some output and sent a header to the client, so my real header became visible in the URL. No bug in HTML::Mason, just a bug in the programmer.

Mon Jan 11 21:01:51 1999	I don't think enough people realize how useful the Apache 1.3.x log filtering system is. Unlike NCSA and previous versions of Apache, you can safely put a filter program between the Apache server and the access_log file, without worrying about the process dying and screwing up your server or losing data. I used this filtering for a little download counter project. I needed to be able to access the download count for a group of files under a certain directory in real-time. I considered writing a mod_perl module to do it, which would have been easy, but also would have bloated my httpd's on this mod_perl-less server. Instead, with half a dozen lines of code, I'm able to safely tally this stuff. Awesome!

Mon Jan 11 18:03:18 1999	If you are using mod_php3's mail() function and qmail together, you need to be sure to set MAILUSER and possibly MAILHOST so that qmail-inject correctly sets your Return-Path and envelope From.

Mon Jan 11 16:07:35 1999	Must have software: A Java ssh client, with source, that is GPL'd. The terminal emulation on this thing is great. It emulates an xterm-color almost perfectly and its vt102 emulation seems flawless. Perfect for getting access to your systems securely when a native ssh client isn't available. They also have a java ssh server over there, although it only does port forwarding and isn't GPL'd. Source is available.

Mon Jan 11 13:40:46 1999	Finished reading the Perl Cookbook a few days ago. Most of it is rehash and recipes that any decent perl hacker would invent on their own, but the object oriented/module sections were useful and I learned this cute bit of perl syntactical sugar: @hash{keys %hash} = (); Which will initialize all of %hash's keys. This is cuter than it seems, because you can put anything in the braces, which will save you a foreach loop here and there.

Mon Jan 11 12:39:37 1999	Discovered a bug in HTML::Mason today. If you tell your handler.pl to ignore everything except text, to avoid inadvertantly trying to parse binary data and screwing everything up, you also ignore requests with no mime-type at all. This means that HTML::Mason will never process requests that would normally give you a 404 error, which breaks dhandlers. I just wrapped the regex that checks the mime-type in an if statement that checks to see if there is a mime-type at all. Seems to work. I've submitted this to Jon Swartz.