There are two ways: server-side or client-side.
Server-side
It depends heavily on the language. Some already have well-established libraries for client detection from the HTTP request header. If your pages are generated dynamically, a good approach would be to provide a version without the script statement.
Client-side
If your pages are statically served (i.e., pure HTML), the best option is to serve them without calling the script. You can then detect through a small custom script if the browser is mobile; if not, dynamically add skrollr.js script.
Like, exactly, you will detect if the browser is mobile gets your choice; you can utilize ready libraries, detect keywords in the userAgent
or use services such as http://detectmobilebrowsers.com/.
I take this opportunity to remind you that size detection is not always a good idea; tablets and some laptops have high resolution And touchscreen interactivity, which can frustrate your plans. Still, I only mentioned methods that work crossbrowser; as you yourself mentioned, media query and some other new technologies can be used as well, but at the cost of not working in legacy browsers.
I hope I’ve helped!