Best value for span/f parameter in lowess/loess function in R

Asked

Viewed 243 times

3

I’m facing a problem with smoothing curves adjusted by functions lowess or loess in R, for the choice of the smoothing parameter f in the lowess or span in the loess

I looked for some tips on web and found the resolution of a similar problem for function smooth.spline() as follows

tuneSpline = function(x,y,span.vals=seq(0.1,1,by=0.05),fold=10){
require(bootstrap)
fun.fit <- function(x,y,span) {smooth.spline(x = x,y = y,spar = span)}
fun.predict <- function(fit,x0) {predict(fit,x0)$y}

mae <- sapply(span.vals, function(span){
  y.cv <- bootstrap::crossval(
    x,y,fun.fit,fun.predict,span=span,ngroup = fold
    )$cv.fit
  fltr <- which(!is.na(y.cv))
  mean(abs(y[fltr]-y.cv[fltr]))
  })
span.vals[which.min(mae)] 

}


attach(cars)
tuneSpline(speed,dist,fold = length(dist))

# 0.75

I tried to modify this routine as follows:

tuneSpline = function(x,y,span.vals=seq(0.1,1,by=0.05),fold=10){
  require(bootstrap)
  fun.fit <- function(x,y,f) {lowess(x = x,y = y, f= f)}
  fun.predict <- function(fit,x0) {predict(fit,x0)$y}

  mae <- sapply(span.vals, function(f){
    y.cv <- bootstrap::crossval(
      x, y, fun.fit, fun.predict, f=f, ngroup = fold
    )$cv.fit
    fltr <- which(!is.na(y.cv))
    mean(abs(y[fltr]-y.cv[fltr]))
  })
  span.vals[which.min(mae)]
}

However when I run code on the same data set, it gives the following error:

attach(cars)
    tuneSpline(speed,dist,fold = length(dist))

# Error in UseMethod("predict") : 
# método não aplicável para 'predict' aplicado a um objeto de classe "list"

I’d appreciate it if you could help, thank you

Ps: I haven’t posted my own data because I don’t have it yet

thanks in advance

r
  • 1

    The error is caused because lowess returns a list of x and y, while smooth.spline returns a class object smooth.spline. The function crossvall You know what to do with the result of the second, but not the first. Are you sure this solution fits your problem? Perhaps the ideal is to post your original problem and not your attempt to adapt another solution to it.

  • Thank you friend for clarifying the error. My problem is to "simply" find a way to optimize lowess or loess functions for the best fit parameter to be chosen. The problem is q, unfortunately I have no idea how to do this, so looking for similar problems I found this solution for Smooth.Spline function, so I thought I’d adapt it to these two. The fact is that I wanted to use a regression method (curve adjustment) that is not parametric at all (q does not depend on any parameter, or q suffers the minimum action of the user). Very Obrg for the help

  • 1

    Your problem does not seem to be too complex. Perhaps the solution involves putting the loess within a function, and then use optimize for the parameter in question. If you have the input data, the parameter to be optimized (within a range) and the criterion to consider optimal, finding the ideal value should not be complicated.

1 answer

1

It would be good to post what the purpose of doing this, it may be that loess functions are not the best solution. The goal of this type of function is usually not to achieve the greatest adjustment to the data but to better represent the trend in the data, eliminating everything that is "noisy". It is relatively simple to adjust the parameter to minimize some error function, but perhaps the best solution is another model.

Minimizing the quadratic error:

tuneSpline = function(x,y,span.vals=seq(0.01,1,by=0.01)){
  theta <- function(x,y,f){ lowess(x=x,y=y, f= f)$y }
  mqe <- sapply(span.vals, function(f){
    ylow <- theta(x,y,f=f)
    fltr <- which(!is.na(ylow))
    mean((y[fltr]-ylow[fltr])^2)
  })
  list(mspan=span.vals[which.min(mqe)],mqe=cbind(span.vals,mqe))
}

attach(cars)
tuneSpline(speed,dist)$mspan
plot(tuneSpline(speed,dist)$mqe)

[1] 0.12

inserir a descrição da imagem aqui

Browser other questions tagged r

You are not signed in. Login or sign up in order to post.