G'day mate, what are you shooting? Varmints here are called "small game" rabbits, hares etc... They are very light skinned and 98% of us (kiwis) use a .22 calibre, however we have some enthusiasts that like longer range hunting of these "varmints, .17hmr is a great calibre, super accurate as I am sure you know, I have a mate that has a savage, he loves it and has had no problems whatsoever. I use a ruger 10/22 and would not go past it.
I am (as per) with Puc on a 22-250, it is probably the most versatile calibre you can buy, a huge range of applications due to the projectile-velocity from this brilliant calibre.
.204 is not so popular in my land a it doesn't really "fit" with any of our game, to large for small game and not large enough for big game I am talking projectile. "knock down" power here, oh and "big game in NZ is our red deer, fallow deer, we do have the much larger Wapiti, elk not far from my home however they can only be hunted during the rut (March/April) and large calibres can only be used, I hunt with a 7mm08 Sako Finnlight topped with a Swarvoski scope and a suppressor, I have literally just walked in from hunting today, a nice wee surprise to see this post!!! I use Federal Premium 139g ammo, I can get 3/4" Moa at 400 yards, which is way more than I need, i predominantly bush hunt.
Sorry long winded on a favourite subject!!
Firstly decide WHAT type of shooting you are after, then try some calibres out, and remember this question is like asking what type of oil to use in your Venture!!!!!!
Kiwi