multivariate regression in Keras

Question

I have two models created in Keras to solve a multivariate regression problem. The first one has 1 input and 5 outputs. The second has an additional entrance, that is, it has two entrances and the same 5 exits. Intuitively, the second model would have an MSE lower than the first due to additional input, however, it is not what happens. The first model has the lower MSE than the second, IE, has a better performance. Do you think that intuition is correct? How do I ensure the second model has a smaller MSE?

Answer 1

It makes sense to think that the more information you put into the model, the better your hit will be. But this is not always true when we are talking about forecasting error.

It may be that this information has left its more specific model for the training base and less general for the validation base.

Answer 2

The complex mathematics of neural networks can create confusion in the model. If this input does not significantly contribute to a better result, it may be that it disrupts and "consumes" part of the neurons in calculations that lead to nothing.

It’s not uncommon to do this: add not very significant inputs and make models worse.

But there could always be something else going on, it’s good to check the bugs, see if the final activations are compatible, etc.

And consider also the "overfitting", case mentioned by @Daniel Falbel in his reply.