Create Your Own Web Proxy Server
By Dark Politricks
If you have read my blog lately you will have seen a number of articles about the Internet and the loss of privacy and freedom that is being rapidly expanded by big brother and which will eventually turn the net into nothing more than a network for advertisers and a way for a government and big corporates to keep track of you.
If the powers that be succeed in removing or limiting the things that make people use the web in the first place then business will suffer as you will have a large network of sellers with no-one around to buy. It reminds me of a Twitter bot I came across the other day that was just retweeting other peoples messages by following certain hash tags. Looking at its profile you could see that it was following thousands of people and had thousands of followers itself a cursory check showed a funny case of Bots following Bots.
In fact the majority of all Internet traffic is bot traffic, search engine crawlers, content scrapers, RSS feed aggregators and so on. I reckon that the actual true percentage of web requests carried out by real humans at the end of a keyboard is actually less than 10%. This 10% are mostly made up of people social networking, downloading movies and music and most of all watching porn. These are you average Internet users and in all 3 areas porn, music and social networks they are being spied on for signs of copyright infringement, illegal activity e.g pedophilia, and the security networks have admitted scanning social networks for signs of malcontent by possible terrorists or subversionists.
Therefore Internet privacy is a major issue and has driven numerous people into the arms of proxy servers, web proxies, darknets and the like and not just because they have something to hide.
One of the major issues with the huge number of web proxies that have sprung up over the last few years is that they are run by hackers who are hoping you are stupid enough to use them to pass important data across such as logons and passwords or even bank details.
If you are not using a secure proxy then any information you pass through it will be available to the proxy server especially because most proxies only handle GET requests which means the data is visible not only in the address bar but also the log file on the web server. You can log POSTS but it takes a bit of work and the log files would be huge.
A recent Radio One interview with school children who were using web proxies to surf Facebook all day whilst they were supposed to be doing school work revealed a very worrying attitude to safety regarding proxies and didn’t seem to care that their logons were being passed into the hands of people who would use them for phishing and other scams.
If like me you are interested in computers and hacking (not the illegal sense) then its not too hard to create your own web proxy.
This could be hosted in a country that you wanted to be able to access e.g the USA so that you could access content banned by IP or it could be hosted on your own server so that at work or school if you wanted to view a site on a blacklist you could by accessing the content through the proxy. In fact there are many reasons for creating your own simple proxy and for that reason I have created an example that you can download the code for and use if you so wished.
The web proxy is pretty simple but has the following features:
- Written in PHP, a popular scripting language.
- The code is all in one page with comments and debug statements that can be switched on or off.
- It offers you the ability to choose a different user-agent IE 7 or FireFox 3 or keep your existing one.
- Once the content has been retrieved it parses the content and changed any links inside the HTML so that they are also re-routed through the proxy. Therefore you only need to use the initial search page to view your first page.
- It contains a built in white-list of allowed domains which can easily be turned into a blacklist. If you allow other people to use it then you can control which domains they can or cannot view.
Anyone who is checking your outbound Internet traffic will most likely be blocking certain sites such as YouTube, Facebook and so on especially if you are at school or work. Therefore you could locate this page on a domain that is not blocked either by purchasing a hosting package or by turning your home computer into a server see this or this. Then instead of accessing the blocked sites directly you can go to your own web proxy and access them that way.
The benefit of using your own web proxy over those on the web are:
- You know it’s secure as you control the code so no worries about hackers inserting malicious scripts into the source code or scanning your request for personal details.
- You can control who uses it. You could put it at a location only you know about or add a simple password check to prevent unauthorised access.
- You can expand the code to add further features and its a good way of getting into computer programming!
This example proxy is just something I have knocked up very quickly so please don’t complain about the code or functionality.
It has been done to show how easy it would be to create your own web proxy and any proper programmers out there would know how to do this anyway. For those of you who wish to use the code which can be downloaded here. I would recommend doing the following tweaks.
- Modify the Crawler Method mycrawler_single so that it handles POST’s as well as GET’s.
- Modify the Crawler Method mycrawler_single so that it can make requests through a proxy server itself. Tunneling will slow down content retrieval but will also make it more secure as long as you trust the proxy it’s going through.
- Add more user-agents to the list of choices.
- Modify the regular expressions that change the URLs so that they are scrambled in a certain way e.g by shifting the characters a couple to the left or using a hash function. You can then unscramble in the crawler method. This would prevent firewalls that scan for web addresses as parameters in other URLs as well as the main URL.
- The function reformat_links currently reformats all src and href attributes so that they pass through the proxy. The code currently handles absolute and relative links but not formats such as ..\.\ I don’t like this format of referencing resources which is why I didn’t bother handling them. If you want to update this function then you can.
- Reformat the code e.g put the code into a class, separate out the HTML and CSS etc. Its all in one file at the moment which makes it simple and easy to set up. Feel free to modify it.
- Once you are happy with the code you should remove or comment out all calls to the ShowDebug function. Even when $DEBUG = False which prevents debug messages from showing its still an overhead to make these unnecessary function calls.
You can download the web proxy source code here: www.darkpolitricks.com/downloads/webproxy.zip
I did have an example running but too many naughty people were using it! So I have removed it.
Please read some more of my articles on Internet Privacy, How to avoid being logged and how to increase the performance of your computer and browser.