reverse proxy URL

Asked

Viewed 576 times

2

My URL requires information on a server (Youtube). But for me to have the desired answer, the request should be made with the IP of a user, instead of mine.

How do I deliver the content directly to the user without my server having to intermediate the download?

It would be possible to do this with proxy, or it would have to be another solution?

I edited the Curl of Script Youtube-Downloader but it didn’t work out.

function curlGet($URL) {

    $proxy=$_SERVER['REMOTE_ADDR'].":80";
    $ch = curl_init();
    $timeout = 3;

    curl_setopt($ch, CURLOPT_URL , $URL );
    curl_setopt($ch, CURLOPT_PROXY, $proxy);
    curl_setopt($ch, CURLOPT_HEADER, 0);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER , 1 );
    curl_setopt($ch, CURLOPT_CONNECTTIMEOUT , $timeout );
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
    curl_setopt($ch, CURLOPT_MAXREDIRS, 2);
    curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7");
    curl_setopt($ch, CURLOPT_COOKIEJAR, "cookies.txt"); 
    curl_setopt($ch, CURLOPT_COOKIEFILE, "cookies.txt");
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); 
    curl_setopt($ch, CURLOPT_ENCODING, "gzip");

    /* if you want to force to ipv6, uncomment the following line */ 
    //curl_setopt( $ch , CURLOPT_IPRESOLVE , 'CURLOPT_IPRESOLVE_V6');
    $tmp = curl_exec( $ch );
    curl_close( $ch );
    return $tmp;
} 

What would be wrong?

  • Post the error that occurs, your doubt may be clear to you that is with the machine that has the proxy, but is not clear to other people, the only way to make understand the problem is to say what exactly is failing friend. Edit the question and provide the details to understand the problem. I’m sure you’ll understand this as a constructive criticism.

  • 2

    This question is being discussed at the goal: http://meta.pt.stackoverflow.com/q/4520

1 answer

5

Curl will perform an HTTP request to the server, and get its response. Note that this is a two-way street, because in each connection some data is sent and then received. Curl uses HTTP, which in turn uses TCP and which in turn uses IP.

In TCP, a connection known as three-way Handshake (in free translation, this would be like "three-way handshake"). In this procedure, the client sends a package to connect to the server (SYN), the server sends a package confirming the accepted connection (SYN-ACK) and the client also sends a package confirming everything (ACK) to the server. This procedure is only possible if there is a two-way communication.

After the three-way Handshake, according to TCP, both client and server can send or receive packages at will. However, in the case of HTTP, this package exchange occurs in such a way that the client sends a request (consisting of a header and in some cases a body) and the server, after receiving the request and deciding what to do with it, sends a reply (also consisting of a heading and a body). After the server sends the response, it can terminate the connection with the client or wait for further requests.

The whole process of three-way Handshake, the sending of the client request and the server response occurs through IP packets. However, in order for the connection to work, the server will need the IP address of the person establishing the connection in order to be able to complete the three-way hand-shake and then send the response to the request (and it will possibly look at the IP address of the sender of the request at the time of formulating the response). If you forge the packets of the TCP connection using an IP address that is not yours (which is possible and is something that many hackers do), the process of three-way hand-shake cannot be successfully completed as the server will send SYN-ACK and attempt to establish the connection to an address other than yours.

In theory, you could carry out the process by forging your user’s IP address and then intercepting the packets the server sends in order to establish the connection and get the answer properly, by pretending to be the user. But in this case, you would have to have control of some of the packet routing points between the server and its user in order to intercept them. And in this case, you’ve already created a kind of proxy or NAT to solve your problem.


Edited.

In response to these comments:

That’s exactly what I want, only in case I will not intercept the data, the user will receive direct data.

To be clearer take a look at this site convert2mp3.net It downloads youtube videos, but somehow that I do not know, it authenticates with the user ip.

If what you want is to simply start the connection in place of the user and make her receive the response without having connected to the server or without being aware of it in any other way, then forget it, it won’t work. The reason is that the user will not have the connection opened with the server, so the TCP packets received by it will be dropped. What happens is more or less the following:

  1. You who have IP 1.2.3.4 that connects to server 5.6.7.8 on port 80. However, you use the forged IP 9.10.11.12 in the packets, and this IP belongs to the user. In the TCP protocol, a TCP port will be allocated to the client for this. Let’s assume that the port that your operating system has chosen (or that you yourself have chosen when forging the package) is 9876. Thus you will have an IP package of type SYN having as origin the port 9876 of IP 9.10.11.12 and as desstino the port 80 of IP 5.6.7.8.

  2. The server receives the SYN and will designate some free TCP port on its side. Let’s say port 43210. Thus the server sends SYN-ACK from port 43210 of IP 5.6.7.8 and as destination port 9876 of IP 9.10.11.12. In this process, the server will reserve port 43210 for exclusive use of port 9876 of IP 9.10.11.12, discarding any other packets arriving at this port from anywhere other than port 9876 of 9.10.11.12. When sending SYN-ACK, the server also expects port 9876 of 9.10.11.12 to be reserved for the exclusive use of port 43210.

  3. If the user receives a SYN-ACK on port 9876 from port 43210 of IP 5.6.7.8, there will be an error in TCP, as there will be no process in the operating system there waiting for something on that port. And if there is, something listening using port 9876, this something will not be waiting SYN-ACK packets from port 43210 of IP 5.6.7.8. As a result, SYN-ACK will be discarded.

  4. If you want, you can intercept this SYN-ACK (regardless of whether it was also forwarded to the user or not) and send an ACK to port 43210 of IP 5.6.7.8, and then send the contents of the request right away.

  5. The server receives the ACK and the packet(s) with the request, processes the data, and sends the response to port 9876 of IP 9.10.11.12.

  6. The user receives some data packets on port 9876 from port 43210 of IP 5.6.7.8. Since there will be nothing on this port (or anything that is waiting for packages from port 43210 of 5.6.7.8), then the packages are discarded.

In this case, your options are:

  • Make a more complicated system, where there is a service running on the user’s host and when you make the Curl for the server, the server makes a Curl for the user sending it data. However, in this case you have to be able to leave a program waiting for connections on the user and the server has to be able to reach it. You cannot do this with convert2mp3.net.

  • Abandon TCP and use UDP instead. In UDP, any packets arriving at a given port are accepted, regardless of origin or content. However, again both the server and the user will need a special program for this and you will have to implement security measures yourself to prevent an attacker from taking advantage of it. You cannot do this with convert2mp3.net.

  • Have the user periodically perform a Polling on the server looking for something that has been posted by you. You cannot do this with convert2mp3.net.

  • Establish the connection with the server and the user and redirect what is received from the server to the user. In this case you will have created something similar to a proxy or NAT.

  • Put some process in the user’s operating system that under its command, establishes itself all the steps of the TCP connection process, including the download of all response packages.

  • Make the user’s system accept packages coming from the server. However, in this case, it might be easier to get her to initiate the connection herself, as in the previous alternative.

And finally, the process that authenticates the user’s IP is relatively simple: The source IP is embedded in the IP package and the source port in the TCP package (which is within the IP package). When the server receives this IP package, it has access to all its contents, including the source IP address. The server can use the source IP address to decide whether or not to fulfill the request and how to handle it. The server has no way of knowing if this source IP is true, but will respond to it and if the response packets end up being routed to someone who is not waiting for them, it will not work.

Recommended reading: TCP and IP.

  • That’s exactly what I want, only in case I will not intercept the data, the user will receive direct data.

  • To be clearer take a look at this site http://convert2mp3.net/ It downloads youtube videos, but somehow I don’t know, it authenticates with the user’s ip.

  • @Rafaeldeoliveira Edited response

  • Well, for you to understand better what I want.

  • https://github.com/jeckman/YouTube-Downloader &#Xa youtube will limit downloads. What do you think of some solution for this, independent of proxy, Curl, anyway, something that works like convert2mp3.net and not with it itself.

  • @Rafaeldeoliveira Now your question has become something entirely different, and I recommend posting another question. However, my idea would be to use Youtube Downloader to download the video directly to your machine, upload an HTTP server on your machine that provides the newly downloaded video, access the convert2mp3.net via Curl or via Selenium and put the URL of your local server, let the convert2mp3.net download from your machine while you download them. The bad part is that the video ends up dealing twice on your network and the music three times.

  • @Rafaeldeoliveira Wait, what do you want is to publish a website like youtube Downloader where the user of your site downloads the videos locally without all the traffic having to go through your site? That’s it?

  • Yes that’s right.

  • @Rafaeldeoliveira Post another question making it clear. In this one you asked to make a request on Curl and asked if you could avoid using a proxy, and what you really wanted was to make a Youtube Downloader that delivered the content directly to the user without your server needing to intermediary the download. Notice that you ended up asking a question that wasn’t the one you wanted and in the end it wasn’t even similar!

  • 1

    OK understood, right now I’m going to ask another question. Thank you.

  • I’m sorry to bother you, but do you have a solution for that? or a starting point for me to research, I’m already without ideas, I’ve been researching for days and I can’t find a solution.

  • @Rafaeldeoliveira Well, I don’t know. The only viable suggestion I could give you is to make the Downloader run on the user’s machine and you make this Downloader available. As for PHP, I really don’t know.

Show 7 more comments

Browser other questions tagged

You are not signed in. Login or sign up in order to post.