Java problem n + 1 query with Map

Asked

Viewed 91 times

1

I have a performance problem where I have 10 rental cars and need to popular a graph of lines with these 10 cars however it is taking a long time to open the screen. Follow the logic I used to make There would be a way to improve this code for the screen to open faster because, it is complicated for the user to have to wait more than 4 minutes for the screen to open. When opening the screen and made the query 10 times (first for cars) and 200 more queries inside the while What would be the best solution to this problem?

public List<CarroVO> carrosAlugadosPorMes() {

List<CarroVO> listaCarrosAlugados = new ArrayList<>();
List<LinkedHashMap<String, List>> carrosAlugados = mapCarros();

// obterCarrosAlugados = id = marcaCarro 
for (LinkedHashMap carroAlugado : carrosAlugados) {

LinkedHashMap<String, Integer> carroMesTotal = new LinkedHashMap<>();
LocalDate dataAtual = LocalDate.now();
LocalDate ultimoMes = dataAtual.minusMonths(11);
LocalDate mesBalanco = LocalDate.of(ultimoMes.getYear(),
ultimoMes.getMonth(), ultimoMes.getMonth().length(ultimoMes.isLeapYear()));
LocalDate ultimoDiaMes = LocalDate.of(dataAtual.getYear(), dataAtual.getMonth(),
dataAtual.getMonth().length(dataAtual.isLeapYear()));

while ( mesBalanco.isBefore(ultimoDiaMes) ||
mesBalanco.isEqual(ultimoDiaMes)) {

Month mes = mesBalanco.getMonth();
String noMesAtual = mes.getDisplayName(TextStyle.SHORT, new Locale("pt", "BR"));
int numeroDias = mes.length(mesBalanco.isLeapYear());
Calendar calendar = ultimoDiaMes(mesBalanco.getYear(), mesBalanco.getMonthValue(), numeroDias);
//Aqui faz a consulta para popular o gráfico de linhas onde utilizo nativeQuery contendo duas pesquisas uma para tipo 1 de carro e outra para o tipo 2 de carro 
Integer totalCarros = carroDao.consultaBanco(calendar, new ArrayList( carroAlugado.values() ));

carroMesTotal.put(noMesAtual, totalCarros);

mesBalanco = mesBalanco.plusMonths(1);
}

CarroVO carroVO = new CarroVO();
carroAlugado.forEach((k,v) -> carroVO.setMarcaCarro(k.toString()) );
List<LinkedHashMap<String, Integer>> totalCarros = new ArrayList<>();
totalCarros.add(carroMesTotal);
carroVO.setTotalCarro(totalCarros);
listaCarrosAlugados.add(carroVO);
}

return listaCarrosAlugados;
}

1 answer

0

There are some critical points in this code snippet. I made a very simple but functional sample to test (If someone else wants to analyze, it’s easier). I believe that a good refactoring is the most important step and, I also believe, that if you arrived at this level of 4 minutes this refactor is urgent.

I noticed that well in the core of for-while we have the consultation at the base and this is very worrying. What is the performance of this consultation on an exponential basis?

Perhaps the section below can be refactored and extracted, for a single execution or a more punctual access to the base. Being so, it seems to me that it will be necessary to refactoring the structure. For this it is necessary to understand well what is behind the carsAlugadosPorMes().

  Calendar calendar = ultimoDiaMes(mesBalanco.getYear(), mesBalanco.getMonthValue(), numeroDias);
  //Aqui faz a consulta para popular o gráfico de linhas onde utilizo nativeQuery contendo duas pesquisas uma para tipo 1 de carro e outra para o tipo 2 de carro 
  >>>>>>> CRÍTICO <<<<<<
  carroDao.consultaBanco(calendar, new ArrayList( carroAlugado.values() ));
  >>>>>>>>>>>><<<<<<<<<<
  carroMesTotal.put(noMesAtual, totalCarros);

[ EDITED ]

So, Rodrigo, I’ve made some adjustments and refactored a few points that might help you. Remembering that I’m in the dark about everything that’s involved in your project. I’m taking into consideration that you can create a database process.

Amendments: Creation of a filter by Tag. (The creation of this filter may involve changing some more structure) As you can notice the while was abstracted to the mercy get CarrosMesTotalProc(...) that must have its logic delegated to a previous.

    public List<CarroVO> carrosAlugadosPorMesBy(String marca) {

            List<CarroVO> listaCarrosAlugados = new ArrayList<>();
            List<LinkedHashMap<String, List<CarroVO>>> carrosAlugados = mapCarrosPor(marca);

            LocalDate dataAtual = LocalDate.now();
            LocalDate ultimoMes = dataAtual.minusMonths(11);
            LocalDate mesBalanco    = LocalDate.of(ultimoMes.getYear(), ultimoMes.getMonth(), ultimoMes.getMonth().length(ultimoMes.isLeapYear()));
            LocalDate ultimoDiaMes  = LocalDate.of(dataAtual.getYear(), dataAtual.getMonth(), dataAtual.getMonth().length(dataAtual.isLeapYear()));

            for (LinkedHashMap<?, ?> carroAlugado : carrosAlugados) {

                CarroVO carroVO = new CarroVO();
                carroAlugado.forEach((k,v) -> carroVO.setMarcaCarro(k.toString()));

                LinkedHashMap<String, Integer> carroMesTotal = new LinkedHashMap<>();
                List<LinkedHashMap<String, Integer>> totalCarros = new ArrayList<>();

                mesBalanco = obterCarrosMesTotalProc(mesBalanco, ultimoDiaMes, carroAlugado, carroMesTotal, totalCarros);
                carroVO.setTotalCarros(totalCarros);

                listaCarrosAlugados.add(carroVO);
            }

            return listaCarrosAlugados;
        }

Another point to be highlighted is the mapCarro() method that now receives a filter by passing to mapCarrosPore(Brand).

 List<LinkedHashMap<String, List<CarroVO>>> carrosAlugados = mapCarrosPor(marca);

I believe that if you follow this line the performance should improve significantly. Below is a new sample with the proposed amendments.

import java.time.LocalDate;
import java.time.format.TextStyle;
import java.util.ArrayList;
import java.util.Calendar;
import java.util.Collection;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Locale;
import java.util.Random;

public class AppTest {

    public static void main(String[] args) {

        Aluguel aluguel = new Aluguel();

        List<CarroVO> listaVolks = aluguel.carrosAlugadosPorMesBy(Marca.VOLKS);
        List<CarroVO> listaToyota = aluguel.carrosAlugadosPorMesBy(Marca.TOYOTA);

        listaVolks.forEach(val -> System.out.println(val.getMarcaCarro() + ": " + val.totalCarros));
        listaToyota.forEach(val -> System.out.println(val.getMarcaCarro() + ": " + val.totalCarros));
    }
}

class Aluguel {

    //MOCK - Dao mockado para simulação
    CarroDao carroDao = new CarroDao();

    public List<CarroVO> carrosAlugadosPorMesBy(String marca) {

        List<CarroVO> listaCarrosAlugados = new ArrayList<>();
        List<LinkedHashMap<String, List<CarroVO>>> carrosAlugados = mapCarrosPor(marca);

        LocalDate dataAtual = LocalDate.now();
        LocalDate ultimoMes = dataAtual.minusMonths(11);
        LocalDate mesBalanco    = LocalDate.of(ultimoMes.getYear(), ultimoMes.getMonth(), ultimoMes.getMonth().length(ultimoMes.isLeapYear()));
        LocalDate ultimoDiaMes  = LocalDate.of(dataAtual.getYear(), dataAtual.getMonth(), dataAtual.getMonth().length(dataAtual.isLeapYear()));

        for (LinkedHashMap<?, ?> carroAlugado : carrosAlugados) {

            CarroVO carroVO = new CarroVO();
            carroAlugado.forEach((k,v) -> carroVO.setMarcaCarro(k.toString()));

            LinkedHashMap<String, Integer> carroMesTotal = new LinkedHashMap<>();
            List<LinkedHashMap<String, Integer>> totalCarros = new ArrayList<>();

            //Delegado a uma procedure
            mesBalanco = obterCarrosMesTotalProc(mesBalanco, ultimoDiaMes, carroAlugado, carroMesTotal, totalCarros);
            carroVO.setTotalCarros(totalCarros);

            listaCarrosAlugados.add(carroVO);
        }

        return listaCarrosAlugados;
    }

    /*
     * MOCK - ESSE CONTEÚDO DEVE SER DELEGADO A UMA PROCEDURE
     * NA BASE DE DADOS
     */
    private LocalDate obterCarrosMesTotalProc(
            LocalDate mesBalanco, 
            LocalDate ultimoDiaMes,
            LinkedHashMap<?, ?> carroAlugado, 
            LinkedHashMap<String, Integer> carroMesTotal,
            List<LinkedHashMap<String, Integer>> totalCarros) {

        String noMesAtual;
        int numeroDias;

        while (mesBalanco.isBefore(ultimoDiaMes) || mesBalanco.isEqual(ultimoDiaMes)) {

            noMesAtual = mesBalanco.getMonth().getDisplayName(TextStyle.SHORT, new Locale("pt", "BR"));
            numeroDias = mesBalanco.getMonth().length(mesBalanco.isLeapYear());

            Integer totalCarrosBase = carroDao.consultaBanco(ultimoDiaMes(mesBalanco, numeroDias), new ArrayList<>(carroAlugado.values()));
            carroMesTotal.put(noMesAtual, totalCarrosBase);

            mesBalanco = mesBalanco.plusMonths(1);
        }

        totalCarros.add(carroMesTotal);
        return mesBalanco;
    }

    public List<LinkedHashMap<String, List<CarroVO>>> mapCarrosPor(String filtro){

        //MOCK
        List<LinkedHashMap<String, List<CarroVO>>> returnList = new ArrayList<>();
        LinkedHashMap<String, List<CarroVO>> lhm = new LinkedHashMap<>(); 
        List<CarroVO> listaCarros = new ArrayList<>();

        if(Marca.TOYOTA.equals(filtro)) {

            CarroVO carroVO = new CarroVO("Hatch",2018);
            listaCarros.add(carroVO);

            carroVO = new CarroVO("Sedan",2019);
            listaCarros.add(carroVO);

            lhm.put("Tyota", listaCarros);
        }

        if(Marca.VOLKS.equals(filtro)) {

            CarroVO carroVO = new CarroVO("Hatch",2018);
            listaCarros.add(carroVO);

            carroVO = new CarroVO("Sedan",2019);
            listaCarros.add(carroVO);

            lhm.put("Volks", listaCarros);
        }

        returnList.add(lhm);

        return returnList;
    }


    //MOCK
    public Calendar ultimoDiaMes(LocalDate mesBalanco, int numeroDias){
        Calendar cal = Calendar.getInstance();
        cal.set(Calendar.MONTH, mesBalanco.getMonthValue());
        cal.set(Calendar.YEAR, mesBalanco.getYear());
        cal.set(Calendar.DAY_OF_MONTH, numeroDias);
        cal.set(Calendar.DATE, cal.getActualMaximum(Calendar.DATE));
        return cal;
    } 

}

class Marca {
    public static String VOLKS = "VolksWagen";
    public static String TOYOTA = "Toyota";
}

class CarroVO {
    private String modelo;
    private String marcaCarro;
    private Integer ano;
    List<LinkedHashMap<String, Integer>> totalCarros;

    public CarroVO() {}
    public CarroVO(String modelo, Integer ano) {
        super();
        this.modelo = modelo;
        this.ano = ano;
    }
    public String getModelo() {
        return modelo;
    }
    public void setModelo(String modelo) {
        this.modelo = modelo;
    }
    public Integer getAno() {
        return ano;
    }
    public void setAno(Integer ano) {
        this.ano = ano;
    }
    public List<LinkedHashMap<String, Integer>> getTotalCarros() {
        return totalCarros;
    }
    public void setTotalCarros(List<LinkedHashMap<String, Integer>> totalCarros) {
        this.totalCarros = totalCarros;
    }
    public String getMarcaCarro() {
        return marcaCarro;
    }
    public void setMarcaCarro(String marcaCarro) {
        this.marcaCarro = marcaCarro;
    }
}

//MOCK 
class CarroDao {
    public Integer consultaBanco(Calendar calendar, Collection<Object> carrosAlugados){
        Random r = new Random();
        return r.nextInt((500 - 1) + 1) + 1;
    }
}

Return obtained after executing the above code:

Volks: [{jun=500, jul=362, ago=324, set=97, out=202, nov=290, dez=479, jan=310, fev=98, mar=129, abr=294, mai=295}]
Tyota: [{jun=215, jul=90, ago=395, set=15, out=384, nov=458, dez=124, jan=20, fev=324, mar=374, abr=204, mai=184}]
  • And how would you look to improve the performance of that code?

  • Not knowing what is involved is difficult, but I think some steps could be taken: 1. Evaluate the query consultBanco(Arraylist, new Arraylist( carroAlugado.values() )); What is the performance of this query? Some part of the logic could be delegated to a precedent? 2. Anticipate some of the information by decoupling the while code. Perhaps for this you need to create more punctual queries. 3. Somehow the data information could become an object? How are the integration tests, present the same performance problem?

  • I could not add in a VO the result of the 4 queries?

  • I edited the above code and scored two segments. Using filters and abstracting the while for a project. Check there, please and tell me if it worked. Another line to be followed would be the use of threads.

  • What the process would look like in the private Localdate method(&#xA; LocalDate mesBalanco, &#xA; LocalDate ultimoDiaMes,&#xA; LinkedHashMap<?, ?> carroAlugado, &#xA; LinkedHashMap<String, Integer> carroMesTotal,&#xA; List<LinkedHashMap<String, Integer>> totalCarros) I’m a database beginner and would like a help

Browser other questions tagged

You are not signed in. Login or sign up in order to post.