0
My code has this bug. But I’ve tried:
- Increase memory in VM settings:
# custom Intellij IDEA VM options
-Xms512m -Xmx1024m -XX:Reservedcodecachesize=480m -XX:+Useconcmarksweepgc -XX:Softreflrupolicymspermb=50 -ea -Dsun.io.useCanonCaches=false -Djava.net.preferIPv4Stack=true -XX:+Heapdumponoutofmemoryerror -XX:-Omitstacktraceinfastthrow
The method in which this burst happens is this:
public List<SearchByCity> getSearchesByCity(){
List<SearchByCity> searches = new ArrayList<>();
Arrays.asList(Estado.values()).forEach(uf -> {
System.out.println(String.format("COLETANDO CIDADES EM %s", uf.getNome().toUpperCase()));
try {
List<String> cityNames = new ArrayList<>();
HtmlPage page = WebClientFactory.getInstance().getPage(String.format(SEARCH_CITIES_URL, uf.toString().toLowerCase()));
cityNames.addAll(Optional.ofNullable(page.<HtmlAnchor>getByXPath(CITYS_URLS_XPATH).stream()
.map(anc -> anc.getAttribute("title")).collect(Collectors.toList())).orElse(null));
final JsonResponseByCity[] jsonResponseByCity = {null};
if(Objects.nonNull(cityNames)){
cityNames.forEach(city -> {
String cityName = removeAccents(city.replaceAll(" ", "-").toLowerCase().replaceAll("'", ""));
String url = String.format(SEARCH_BY_CITY_URL, uf.getNome(), cargoPolitico.getDescription(), cityName, uf.toString().toLowerCase(), cityName, cargoPolitico.getDescription());
try {
jsonResponseByCity[0] = ObjectMapperFactory.getInstance().readValue(WebClientFactory.getInstance().getPage(url).getWebResponse().getContentAsString(), JsonResponseByCity.class);
} catch (IOException e) {
throw new UncheckedIOException(e);
}
searches.add(new SearchByCity(uf.getNome().toUpperCase(), cityName, cargoPolitico, jsonResponseByCity[0]));
});
}
} catch (IOException e) {
throw new UncheckedIOException(e);
}
});
if(searches.size() > 0)
return searches;
return null;
}
The overflow happens on the line of value assignment to the jsonResponseByCity[0].
java.lang.Outofmemoryerror: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3332) at java.lang.Abstractstringbuilder.ensureCapacityInternal(Abstractstringbuilder.java:124) at java.lang.Abstractstringbuilder.append(Abstractstringbuilder.java:596) at java.lang.Stringbuilder.append(Stringbuilder.java:190) at org.apache.Commons.io.output.StringBuilderWriter.write(Stringbuilderwriter.java:143) at org.apache.Commons.io.Ioutils.copyLarge(Ioutils.java:2370) at org.apache.Commons.io.Ioutils.copyLarge(Ioutils.java:2348) at org.apache.Commons.io.Ioutils.copy(Ioutils.java:2325) at org.apache.Commons.io.Ioutils.copy(Ioutils.java:2273) at org.apache.Commons.io.Ioutils.toString(Ioutils.java:1041) at com.gargoylesoftware.htmlunit.WebResponse.getContentAsString(Webresponse.java:246) at com.gargoylesoftware.htmlunit.WebResponse.getContentAsString(Webresponse.java:191) at br.com.Crawler.impl.eleicoes.page.Searchbuilder.lambda$null$2(Searchbuilder.java:71)
I will not try to analyze the code, because it may be even elsewhere, but fix the programming error. The problem is not the size of the heap.
– Maniero
I’ve already reviewed the code, but the way I see it, there’s no programming error. From what I understand json is assigned to only one location in memory, since I am defining its position. Is that it? I don’t understand this error
– laaf
If we’re talking about json interpretation and it’s a memory problem, without looking at your code (which I haven’t looked at), I’d start by seeing if there are no circular references in your json and/or interpretation of it. This could be a reason to loop around creating objects endlessly.
– Isac