Find via SSH on multiple servers

Question

One of my activities in the company I work with is to check Jvms log when we have errors in the application.

The way this check is done is through SSH (we have a "tunnel" configured through VPN directly configured in SONICWALL).

It works that way:

$ ssh [servidor interno] (utilizo ALIAS, o host está configurado na minha chave)
$ssh [servidor cliente] (não temos mais acesso direto)

Inside the client’s server we have 2 Jvms (we create links from the log folders so we don’t have to keep browsing them), but to access the folders we need to go back a directory (I don’t know why the company’s user doesn’t log in to the root directory), getting:

$ cd

Summarizing the routine:

$ ssh [servidor empresa]
$ ssh [servidor cliente]
$ cd
$ cd [log_vm_1]
$ find ./* -type f -exec grep -l COD_ERRO_JVM  {} \;

It is not a very long process, the problem is that we have 5 servers with 2 Vms each, and the application is managed by a Weblogic (we have no way of knowing where the error occurred), IE, I have to search VM by VM.

My question is:

It is possible to create a script where I run this find on all servers/Vms at the same time?

Answer 1

First you can create a script that searches the two Jvms of the same server:

#!/bin/bash

cd
for dir in log_vm_1 log_vm_2
do
    echo "procurando em" $(hostname) $dir
    find $dir -type f -exec grep -l COD_ERRO_JVM  {} \;
done

Notice that you don’t have to cd dir and then find, it is possible to pass the directory directly to the find. By the way, if you know the complete path of the directory, not even the cd at first it is necessary, you could just do find /caminho/completo/log_vm.

In fact, if you want, you can pass several directories at once: find log_vm_1 log_vm_2, so I wouldn’t need the for. I made the for up just to have a echo between one directory and another, so that the respective outputs are visually more separate.

---

Assuming the above script is saved to the file busca.sh (in some folder of your internal server), just connect to each of the client’s servers passing the name of the script as parameter:

ssh servidor_interno
for serv in  servidor_cliente_1  servidor_cliente_2  servidor_cliente_3
do
    ssh $serv "bash -s" < /pasta/do/arquivo/busca.sh
done

"bash -s" < arquivo causes the commands to be read from the file. Then, they are executed on the client server, without the script needing to be there.

With this the commands of the script busca.sh run on each of the client’s servers. The line with echo prints the hostname and the directory, so you will have an output similar to that:

procurando em servidor_cliente_1 log_vm_1
... saída do find 
procurando em servidor_cliente_1 log_vm_2
... saída do find
procurando em servidor_cliente_2 log_vm_1
... saída do find 
procurando em servidor_cliente_2 log_vm_2
... saída do find
... e assim por diante

Of course you can save the for serv ... above in another script, if you like. For example, if I save in /pasta/busca_todos.sh (on the internal server) and give permission to execute (chmod u+x busca_todos.sh), I can run it directly by SSH:

ssh servidor_interno /pasta/busca_todos.sh

Thus, when logging into the internal server, the script busca_todos.sh is executed. And within it there is the for that makes a loop by all the client’s servers, running the search on each of them.

---

Another alternative is to copy the script busca.sh for each of the client’s servers:

scp busca.sh servidor_cliente_1:/pasta
scp busca.sh servidor_cliente_2:/pasta
... copiar para todos os servidores do cliente

Since there is now a copy of the file on each client server, just pass the full path of it while doing SSH:

ssh servidor_interno
for serv in  servidor_cliente_1  servidor_cliente_2  servidor_cliente_3
do
    ssh $serv /pasta/busca.sh
done

Remembering that the script must have execution permission (it is necessary to execute chmod u+x busca.sh on each client server so that it can be run).