Is there a package that does stepwise and multicollinearity analysis simultaneously?

Question

I’m developing a model and I wonder if anyone knows any packages that do stepwise and already measure the correlation between dependent variables, seeking the best model without high collinearity.

I’m using the package caret with the functions findCorrelation and developing the model with train(..., method = "glmStepAIC").

But I wanted a function that already measured everything at once and found the best model thinking first of the variables that best discriminate rather than first taking multicollinearity. I thought of a strategy to do this by running the glmStepAIC and then test the collinearity, and I myself will remove the correlated variables prioritizing the most significant, run again the glmStepAIC without the variables I removed, rechecking the correlation, once again removing the highly correlated ones prioritizing the power of discrimination and so on, but this process is very long/slow/repetitive. Does anyone know another medium or a function/package that does this automatically?

Answer 1

Hard to have a function that does exactly what you want (I don’t know). When you do an AIC stepwise, it will first select the model with the smallest AIC (it does not look for multicollinearity, but it is evident that estimates of Betas and variance are somehow affected or so), and after that, you do multicollinearity analysis via car::vif, for example.

But instead, you could adjust a regression with penalty. It on its own can already accommodate multicollinearity and p > n problems. The most common are:

• Regression with penalty Lasso (Selects variables by setting some parameter equal to 0);
• Regression with penalty Ridge (All initial variables remain in the model, however some with parameters close to 0);
• Regression with penalty Elastic Net (is in the 'middle' term of the 2 from above);

They are implemented in the package glmnet, whose documentation can be found here