Multithread in web application

Question

I have an app that does some server processing and then delivers it to the client. Something like:

    http://aplicacao:8080/app/videos/extrair/{id}.

Where the "id" is the item reference for processing.

If user repeats the request 10 times with different items, he is creating 10 simultaneous threads. but I’d like to create only three simultaneous threads for each user, and when those three are done, start the next three in line. Not to overload the server.

I’m not able to implement with Threadpoolexecutor on the web server, because whenever a new request is made it instantiates the class again. I thought of implementing a Singleton, but I would have to create threads per user, or different IP.

Any suggestions?

Answer 1

Not to recreate the ThreadPoolExecutor each request, place it in the user’s session.

Example:

//valida requisição
//...

ThreadPoolExecutor tpe = null;

//evita problemas com duas requisições simultâneas do mesmo usuário
synchonized (session) {
    tpe = session.get("userThreadPool");
    if (tpe == null) {
        tpe = MeuThreadPoolBuilder.criarNovoThreadPool();
        session.put("userThreadPool", tpe);
    }
}

//adiciona processo no pool
//...

The problem with this approach is that it doesn’t work in clusters, i.e., if there is more than one server with the application.

Ideal for running processes in queues is to have some kind of Scheduler, like the Quartz, which is a scheduler with distributed environment support.

In addition, define threads per user can easily overload the server and become an issue if the number of users increases. In short: your application will not scale.

The ideal is to set a maximum number of threads and then queue all processing requests in this queue.

Answer 2

The best solution would be to create a stack for each user, and limit all of them to run only 3 threads at once, and put the rest on ThreadPoolExecutor. Thus, as the stack will not cease to exist until all its threads are executed, the ThreadPoolExecutor will not be called again when a new request is made.

Answer 3

This need has a lot of "Actors" like Erlang, clojure, scala or Akka (now scala standard but with java implementation as well).

There are other competing models too, you can go to apache Storm, have an application using and is very fast, in this case you can control the number of Spouts ( data generators ) for the bolts or have a single Spout and the amount of bolts for each "customer" as you like. In my case I have 10 bolts that keep recording in the database in the last layer so Storm. The possibilities of Storm are infinite but it is recommended for clusters, recommended, use in a single machine and have a problem with the data of his zookeeper that clog my tmpfs.

If it does not matter how many threads you want to use and its application is in 1 server, then you can think of the Lmax disruptor, practically the disruptor tries to keep the data hot or in the processor’s cache memory. By chance it will be the next evolution of my application that I mentioned.

Study hard, really hard all the possibilities I mentioned, just because they contain less accidental complexity than trying to manipulate threads, there’s no problem with that, but the less we need to go to the lowest level of everything, with a great value for money ratio, it is better.

Take it easy on the possibilities of everything in life, keep your application running the way it is, make the patchs just to avoid your fear of overloading the server and jump to a higher-level solution, "maybe" until a new language if any... the best solution is the one that is best for you and the business.