2
I am performing FTP connection, with the FtpWebResponse
of c#. So far so good, I’m listing the directories as per this answer.
When I simulate an FTP server locally with Filezilla Server included in XAMPP, I list the directories and come one by one on each line from ResponseStream
, as in the example:
config/
app/
public/
file.xml
But I tested today on two remote servers and comes a html gigantic.
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd
">
<!-- HTML listing generated by Squid 2.6.STABLE21 -->
<!-- Wed, 27 May 2015 17:42:13 GMT -->
<HTML><HEAD><TITLE>
FTP Directory: ftp://[email protected]/
</TITLE>
<STYLE type="text/css"><!--BODY{background-color:#ffffff;font-family:verdana,sans-serif}--></STYLE>
</HEAD><BODY>
<PRE>
--------- Welcome to Pure-FTPd [privsep] [TLS] ----------
You are user number 2 of 50 allowed.
Local time is now 18:43. Server port: 21.
This is a private system - No anonymous login
IPv6 connections are also welcome on this server.
</PRE>
<HR noshade size="1px">
<H2>
FTP Directory: <A HREF="/">ftp://[email protected]</A>/</H2>
<PRE>
<A HREF="etc/"><IMG border="0" SRC="http://proxy.domain.local:8080/squid-internal-static/ic
ons/anthony-dir.gif" ALT="[DIR] "></A> <A HREF="etc/">etc</A>. . . . . . . . . . . . . . . Jan 13 20
:39
<A HREF="logs/"><IMG border="0" SRC="http://proxy.domain.local:8080/squid-internal-static/i
cons/anthony-dir.gif" ALT="[DIR] "></A> <A HREF="logs/">logs</A> . . . . . . . . . . . . . . May 14
19:06
<A HREF="mail/"><IMG border="0" SRC="http://proxy.domain.local:8080/squid-internal-static/i
cons/anthony-dir.gif" ALT="[DIR] "></A> <A HREF="mail/">mail</A> . . . . . . . . . . . . . . Dec 16
20:53
<A HREF="public_ftp/"><IMG border="0" SRC="http://proxy.domain.local:8080/squid-internal-st
atic/icons/anthony-dir.gif" ALT="[DIR] "></A> <A HREF="public_ftp/">public_ftp</A> . . . . . . . . .
. . Aug 4 2014
<A HREF="public_html/"><IMG border="0" SRC="http://proxy.domain.local:8080/squid-internal-s
tatic/icons/anthony-dir.gif" ALT="[DIR] "></A> <A HREF="public_html/">public_html</A>. . . . . . . .
. . . May 25 17:21
<A HREF="ssl/"><IMG border="0" SRC="http://proxy.domain.local:8080/squid-internal-static/ic
ons/anthony-dir.gif" ALT="[DIR] "></A> <A HREF="ssl/">ssl</A>. . . . . . . . . . . . . . . Aug 5 2
014
<A HREF="tmp/"><IMG border="0" SRC="http://proxy.domain.local:8080/squid-internal-static/ic
ons/anthony-dir.gif" ALT="[DIR] "></A> <A HREF="tmp/">tmp</A>. . . . . . . . . . . . . . . May 5 12
:57
<A HREF="www"><IMG border="0" SRC="http://proxy.domain.local:8080/squid-internal-static/ico
ns/anthony-link.gif" ALT="[LINK]"></A> <A HREF="www">www</A>. . . . . . . . . . . . . . . Sep 30 20
14 <A HREF="www;type=a"><IMG border="0" SRC="http://proxy.domain.local:8080/squid-i
nternal-static/icons/anthony-text.gif" ALT="[VIEW]"></A> <A HREF="www;type=i"><IMG border="0" SRC="h
ttp://proxy.domain.local:8080/squid-internal-static/icons/anthony-box.gif" ALT="[DOWNLOAD]"
></A> -> <A HREF="public_html">public_html</A>
</PRE>
<HR noshade size="1px">
<ADDRESS>
Generated Wed, 27 May 2015 17:42:13 GMT by proxy.domain.local (squid/2.6.STABLE21)
</ADDRESS></BODY></HTML>
I removed some parts so it wouldn’t get too big and also some confidential information...
How to force the answer to be only directories and files line by line, or at least one XML?
Edit
I inspected the requisition with the Fiddler Web Debugger, and in Inspector > Raw contains the following:
GET ftp://user:[email protected]/ HTTP/1.1
Host: domain.com
Proxy-Connection: Keep-Alive
Obs.: I don’t know why, Fiddler is interrupting my request and my application doesn’t complete it, stands still until it’s too late, only concludes when I close Fiddler.
Edit 2
I tested the program at home and it worked normally, returning only the list of directories. As discussed in the comments the problem is probably the company Proxy.
Strange, can you identify what HTML is about? Do you use any Proxy?
– stderr
Yes proxy use, at first html seems to be the directory listing, I will post a part of it...
– KaduAmaral
That’s the problem. Add your code to the line:
ftpRequest.Proxy = null;
. =)– stderr
I put it, it doesn’t connect...
– KaduAmaral
So define him with a proxy:
ftpRequest.Proxy = "TeuProxyAqui";
.– stderr
The FTP server is understanding that the request was made by HTTP and not by FTP so it is responding with an HTML file with the formatted listing of directories and files. Try to verify using Finddler2 or Charles or Wireshark exactly how your request is being made.
– Guilherme Branco Stracini
When I don’t define the proxy, or define in the following ways:
Request.Proxy = WebRequest.GetSystemWebProxy();
orRequest.Proxy = new WebProxy("proxy.dominio.local", 8080);
. It works and brings the result like [tag:HTML]– KaduAmaral
You are not getting past Squid, one way to solve this problem would be to analyze the HTML and extract the information you want. You can use the HTML Agile Pack, and select the files like this:
document.DocumentNode.SelectNodes("//A/text()")
.– stderr
@Guilhermebrancostracini tried to use Fiddle but it is not capturing the application request, only from the browsers... :/
– KaduAmaral
@Guilhermebrancostracini managed to set up here, but he’s hijacking my request is not letting it be completed... Anyway, the Inspector tab on the Raw option contains the following:
GET ftp://user:[email protected]/ HTTP/1.1
Host: domain.com
Proxy-Connection: Keep-Alive
– KaduAmaral
Use the property
KeepAlive
ofFtpWebRequest
. As a last resort, if not successful, consider parse html. Using the example of the question I managed that result.– stderr
You can configure this Squid?
– Leonel Sanches da Silva
No @Ciganomorrisonmendez, the proxy is company, I do not have access to settings, I surround the home application where there is no proxy and worked normally. So the problem isn’t in the configuration of my application? I have to parse as @qmechanik said?
– KaduAmaral
You have, but I don’t know if it’s worth it.
– Leonel Sanches da Silva
@qmechanik can post the example parser as answer? I think this will be the most viable solution... :/ Then I’ll test later.
– KaduAmaral
The proxy is routing your FTP request to HTTP, this is the problem, in this case you will have to "parse" the HTML returned by FTP and extract the information. You can use the Html Agilepack as already recommended. the CS Query or make your own reader/parser.
– Guilherme Branco Stracini