Who is knocking at your web site door? Are you well set up to deal with allcomers?
Archive - Originally posted on "The Horse's Mouth" - 2011-10-21 11:15:49 - Graham Ellis
Anyone who advertises a public facing web server / web site is telling everyone about a door to their resources ... and even those people who have web sites which they don't advertise are likely to be talked about and so discovered by a potentially wide audience. So it's rather important for the people who are looking after the web server and web site to be very careful about that door:
* What can be taken out of the door?
* What people can bring in through the door and leave with you?
* Whether the door is big enough for everyone to pass through.
* Whether the property behind the door is big enough to handle all the customers.
* Can you keep customers queueing at the door at busy times, or will they give up?
* Will there always be at least someone inside the door to look after customers?
* Indeed - will there be enough staff to handle all the customers coming through the door?
* Whether the Landlord's going to get upset if there's too many people coming to your door.
* Is the door always going to be accessible?
* How will you know if something malfunction in the door's operation?
In the "real" world, there are checks and controls and common sense on all of these issues. Think of a shop, of a train, of a private house (or a public house) and you'll see how we administer each issue in day to day life. It may be through signage and good common sense, as in these instructions from the railway carriage telling you how to open the door. And people can make mesaured decisions too, based on how long a queue they find when they turn up, or they can come back later if they're advised that something will take a bit of a while.
In the world of the Internet, and web sites, all the aspects need to be considered, but handled automatically. The traffic level is much higher than in the real world, the visitors less forgiving if things don't work right (and obviously) for them, and many of those visitors will be looking to take advantage of you.
• If you were to walk into each of the shops in Melksham and try to leave a pile of leaflets advertising [something] without permission, you would soon find they were chucked out, and that you came to the attention of the shopkeepers and perhaps the police. But online, there are enormous numbers of automated programs looking to leave things on your web server ... and they are constantly knocking on your door, trying out your staff daemons, to see what they can leave where others may find it.
• Similarly, if you walk into a shop (or perhaps the Tourist Information Centre, where much of what's on display is free) and try to leave with their display stands, they'll probably stop you. But online, there are again a lot of automated programs that are looking to get things off your server which are the fixures and fittings rather than the goods you have on offer. That's so that they can learn about your systems and come back to leave their advertising material later, via a back door rather than through the front.
Web site and web server security is a huge subject ... to give you an idea, we had 110,000 requests made to our front door yesterday. I estimate that around 45% of those requests are from search engines indexing our pages (these are benign automated programs that will help get our message out to the world), and around 40% of the requests are from real users looking for a resource that we've made available for them. Another few percent can be accounted for by people "hotlinking" images off our web site (see [here]) and that leaves just over 10% of requests being of "security concern". Not a high percentage, but just one request that penetrates a hole in our system would be one too many.
Malicious Automata tend to look and see if certain files / URLs exist on your server. They'll speculatively try some common names, and also names of files that they know exist in standard software packages to see if you have those loaded. Off course, 999 times out of 1000 you won't have that software loaded, and in the remaining case you'll probably have fixed the problem / set a password / not have the right setup to be vulnerable.
For the 999 out of 1000 failed malicious requests, we want to respond as quickly, negatively, and efficiently as we can. Incoming requests for pages that have names which aren't even close to what's on our site are, therefore, met with a very simple "page not found" response - with a header code that clearly says the resource does not exist. The page doesn't follow the format of the rest of the content of our site; I'm all for uniformity, but really I don't want my staff daemon to be spending a lot of time dealing with these rogues, nor do I want to have my landlord getting upset as I ship them loads of information in response to their request which they'll never use. That page can be found [here] if you want to see what it looks like, and it includes a link to this article so that the occasional real visitor who gets it and is interested can read up. The page may be short, but it IS polite ... so that majority of real users who get to it - perhaps because of a broken link on someone else's site, or because of a mistyped link, will simply follow one of the offered links to really help them find the resource.
P.S. Yes - we're a training company and consultancy. We cover some web security on courses such as deploying LAMP, and welcome other questions and enquiries. We may be able to help you ourselves, or point you in the right direction if not.
Anyone who advertises a public facing web server / web site is telling everyone about a door to their resources ... and even those people who have web sites which they don't advertise are likely to be talked about and so discovered by a potentially wide audience. So it's rather important for the people who are looking after the web server and web site to be very careful about that door:
In the "real" world, there are checks and controls and common sense on all of these issues. Think of a shop, of a train, of a private house (or a public house) and you'll see how we administer each issue in day to day life. It may be through signage and good common sense, as in these instructions from the railway carriage telling you how to open the door. And people can make mesaured decisions too, based on how long a queue they find when they turn up, or they can come back later if they're advised that something will take a bit of a while.