1
I am trying to make an application to log into the college website, and through Crawler I work the information returned by httpclient. Note: I am doing for learning, I started recently in java.
I do several posts because the college site has redirect during login.
The error is that I authenticate myself, but when I do the Post of the last page this appearing in the console the expired session text.
Statements:
private AsyncHttpClient client = new AsyncHttpClient(true, 80, 443);
private PersistentCookieStore myCookieStore;
Code:
RequestParams params = new RequestParams();
params.put("tipo", "aluno");
params.put("unidade", "12");
params.put("login", "2013200111");
params.put("senha", "???");
client.post("http://www.unicarioca.edu.br/gvcollege_submit.php",
params, new AsyncHttpResponseHandler() {
@Override
public void onSuccess(int arg0, Header[] arg1, byte[] arg2) {
// TODO Auto-generated method stub
String str = getString(arg2);
System.out.println(str);
}
@Override
public void onFailure(int arg0, Header[] arg1, byte[] arg2,
Throwable arg3) {
// TODO Auto-generated method stub
}
});
params = new RequestParams();
params.add("lstUnidades", "1,12");
params.add("ViewLoginXmlXsl[method]", "btnLogin_click");
params.add("acao", "login");
params.add("usr", "2013200111");
params.add("passwd", "???");
client.post(
"https://portal.unicarioca.edu.br/index.php5?irPage=login&irModulo=aluno",
params, new AsyncHttpResponseHandler() {
@Override
public void onSuccess(int arg0, Header[] arg1, byte[] arg2) {
// TODO Auto-generated method stub
String str = getString(arg2);
System.out.println(arg2);
}
@Override
public void onFailure(int arg0, Header[] arg1, byte[] arg2,
Throwable arg3) {
// TODO Auto-generated method stub
}
});
client.get(
"https://portal.unicarioca.edu.br/modulos/aluno/login.php5?",
new AsyncHttpResponseHandler() {
@Override
public void onSuccess(int arg0, Header[] arg1, byte[] arg2) {
// TODO Auto-generated method stub
String str = getString(arg2);
System.out.println(str);
}
@Override
public void onFailure(int arg0, Header[] arg1, byte[] arg2,
Throwable arg3) {
// TODO Auto-generated method stub
}
});
I will describe step by step.
1° Cookie sent by Chrome- page gvcollege -- no return of cookies
ys-path_menu_aluno=s%3A/ynode-7/ynode-36/ynode-38; _ga=GA1.3.1594079648.1403628555; __utma=127282249.1594079648.1403628555.1409943723.1410190271.40; __utmb=127282249.1.10.1410190271; __utmc=127282249; __utmz=127282249.1408118488.24.5.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided); PHPSESSID=17d78ba1c38aafae711f916838baa997
After sending you have a redirect, to index.php5? irPage=login....
2° cookie sent by Chrome --
Cookie:ys-path_menu_aluno=s%3A/ynode-7/ynode-36/ynode-38; _ga=GA1.3.1594079648.1403628555; __utma=127282249.1594079648.1403628555.1409943723.1410190271.40; __utmb=127282249.1.10.1410190271; __utmc=127282249; __utmz=127282249.1408118488.24.5.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided)
The page returns the cookie set
3° Cookie set
Set-Cookie:PHPSESSID=2e1dd7378b3ef2a5a6d443c9b0128f99; path=/; HttpOnly
Set-Cookie:PHPSESSID=17d78ba1c38aafae711f916838baa997; path=/; HttpOnly
And redirect to login.php5
4° cookie sent by Chrome to the college website
ys-path_menu_aluno=s%3A/ynode-7/ynode-36/ynode-38; _ga=GA1.3.1594079648.1403628555; __utma=127282249.1594079648.1403628555.1409943723.1410190271.40; __utmb=127282249.1.10.1410190271; __utmc=127282249; __utmz=127282249.1408118488.24.5.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided); PHPSESSID=17d78ba1c38aafae711f916838baa997
5° Cookie set
Set-Cookie:PHPSESSID=b73d66a5f917ff9615aefd469a6fe758; path=/; HttpOnly
6° step it redirects to the student’s home page.
What are these ys-path_menu_student cookies and others that appear ? I couldn’t identify where they were set.
Lucas, is that code updated with the Igor tip? Could you include the configuration part of
CookieStore
?– Wakim
Lucas, I looked at your college website and noticed that the first two requests do not need to be made (I tested an invalid login/password and he threw me straight to the last page). The issue of the expired session is why a session cookie
PHPSESSID
(domain portal.unicarioca.edu.br) is lost. I believe only having the last request works, but could you check if in the first two requests which cookies are returned? You could also include in your question (clearing personal data if you have) their content for each one?– Wakim
Updated with the yes tip. I will add cookies here.
– Lucas Duca
The problem is that in the last request no parameter is passed, as I will debug the site login in Chrome.
– Lucas Duca
Using the system by the browser, which screen does it redirect after logging into
https://portal.unicarioca.edu.br/modulos/aluno/login.php5?
using the form? It’s the same page?– Wakim
https://portal.unicarioca.edu.br/modulos/aluno/index.php5 Cookie:ys-path_menu_aluno=s%3A/ynode-7/ynode-36/ynode-38; _ga=GA1.3.1594079648.1403628555; __utma=127282249.1594079648.1403628555.1409943723.1410190271.40; __utmb=127282249.1.10.1410190271; __utmc=127282249; __utmz=127282249.1408118488.24.5.utmcsr=google|utmccn=(Organic)|utmcmd=Organic|utmctr=(not%20provided); PHPSESSID=b73d66a5f917ff9615aefd469a6fe758
– Lucas Duca
They were probably set via javascript. You are setting this setting to Persistentcookiestore with
AsyncHttpClient.setCookieStore
? I believe the last request is not necessary. Because atPOST
it redirected to the student’s home, print the result of the third parameterbyte[]
to see.– Wakim
myCookieStore = new Persistentcookiestore(this); client.setCookieStore(myCookieStore); myCookieStore.clear();
– Lucas Duca
I’m thinking that my problem is being these cookies that are set via javascript.
– Lucas Duca
Hmmm, good finding... It may be that there is some javascript processing that in your case via
HttpClient
not being done and then breaks the flow of the system... not only Cookies, but validation/ requests ajax and etc... An alternative would be to invest in aWebView
"invisible" (allowing javascript), make the navigations from it and parse the HTML withJSoup
for example.– Wakim
Good idea I’ll try here and put, thank you.
– Lucas Duca