Request by browser works, but Java gives error 301

Question

When I make an HTTP request to that URL, It works and I get a CSV file. However, when calling the same URL via Java (by get or post), I’m getting code 301.

Could someone help me? Here’s my code:

public InputStream sendGet(String url) throws Exception {

    String USER_AGENT = "Mozilla/5.0";
    URL obj = new URL(url);
    HttpURLConnection con = (HttpURLConnection) obj.openConnection();

    // optional default is GET
    con.setRequestMethod("GET");

    //add request header
    con.setRequestProperty("User-Agent", USER_AGENT);

    int responseCode = con.getResponseCode();
    System.out.println("\nSending 'GET' request to URL : " + url);
    System.out.println("Response Code : " + responseCode);

    InputStream in = con.getInputStream ( ) ;
    return in;
}

public InputStream sendPost() throws Exception {

    String Url = "http://www.oanda.com/currency/historical-rates/download";
    URL obj = new URL(Url);
    //HttpsURLConnection con = (HttpsURLConnection) obj.openConnection();
    HttpURLConnection con2 =  (HttpURLConnection) obj.openConnection();
    String USER_AGENT = "Mozilla/5.0";
    //add reuqest header
    con2.setRequestMethod("POST");
    con2.setRequestProperty("User-Agent", USER_AGENT);
    con2.setRequestProperty("Accept-Language", "en-US,en;q=0.5");

    String urlParameters = "quote_currency=EUR&end_date=2016-3-26&start_date=2016-3-24&period=daily&display=absolute&rate=0&data_range=c&price=mid&view=graph&base_currency_0=USD&base_currency_1=&base_currency_2=&base_currency_3=&base_currency_4=&download=csv";

    // Send post request
    con2.setDoOutput(true);
    DataOutputStream wr = new DataOutputStream(con2.getOutputStream());
    wr.writeBytes(urlParameters);
    wr.flush();
    wr.close();

    int responseCode = con2.getResponseCode();
    System.out.println("\nSending 'POST' request to URL : " + Url);
    System.out.println("Post parameters : " + urlParameters);
    System.out.println("Response Code : " + responseCode);

    InputStream in = con2.getInputStream() ;
    return in;
}

Answer 1

Do you know what 301 is? It’s not exactly a mistake, but rather the permanent redirection indicator code. The server is warning the HTTP client that the content has been moved to another URL. When the server sends this response, it probably also sent an HTTP header called Location which contains the new URL. Your browser is reading this header and redirecting it automatically. Your HTTP client in Java does not.

I took a look at your case and saw that what’s happening is that it’s redirecting to the HTTPS page. In this case, a simple solution is to simply exchange your URL for the HTTPS version.

In a more general case, you need to manually extract the URL and start another request. I’ll give you basic insight into how to do this to make it a little more complete:

Function indicating if redirection should occur:

public boolean devemosRedirecionar(int codigoDeRespostaHTTP) {
    return codigoDeRespostaHTTP == HttpURLConnection.HTTP_MOVED_TEMP
        || codigoDeRespostaHTTP == HttpURLConnection.HTTP_MOVED_PERM
        || codigoDeRespostaHTTP == HttpURLConnection.HTTP_SEE_OTHER
}

How to extract the URL that should be used for redirect:

public String obterURLDeRedirecionamento(HttpUrlConnection conexao) {
    return conexao.getHeaderField("Location");
}

With this data, just reassemble your request and send again.