How to calculate the amount of connections on a Deep Network?

Question

I have exactly this scenario and I need to know how many connections this set has. I’ve looked in several places and I’m not sure of the answer. I mean, I don’t know how to calculate the number of connections in my network, this is not yet clear to me. What I have is exactly the following:

**Having bias on everything but input - Input: 784 - First Hidden layer - Output: 400 - Second Hidden layer - Output: 200 - Output layer - Output: 10

I would calculate this as follows: (784 * 400) + bias) + ((400 * 200) + bias) + ((200 * 10) + bias) = XXX

I don’t know if that’s right. I need help understanding how to solve this, and if it’s not just something mathematical, what’s the theory for doing this calculation?

Thank you.

Answer 1

If you write this as matrix multiplication - in my view - it is easier to calculate the number of parameters (or if you want to call connections) of your network.

Suppose your X input is a Matrix X[n, 784] in which n is the number of observations.

A network with 2 layers one with 200 neurons and one with 10. It would be something +- like this:

Y[n, 10] = ativacao(X[n,784]*P1[784, 200] + B1[200])*P2[200,10] + B2[10]

In which P1, P2, B1, B2 are matrices of parameters. Now you can check if your account is correct, just count the number of elements of the parameter matrices.

In the case: 784*200 + 200 + 200*10 + 10