Two of those came in as first half dominated by the Taunton forwards, with tight head Vince Carrington also crossing the whitewash.
Wanderers’ work-rate was inspired by Lee Stanley, who despite an ankle injury, fired around the pitch making every effort to close down any attacks.
Wivey hit back early in the second half after a mistake in the Taunton line-out but the home side soon got back on track.
Centre Chris Hurley scored from close range before Wiltshire dotted down for his hat trick.
Taunton: 1 Wayne Wiltshire, 2 Andy Farley, 3 Vince Carrington, 4 Ben Goddard, 5 Steve Harris, 6 Paul Dungy, 7 Lee Stanley, 8 Chris Dennehy, 9 Mark Winter, 10 Moses Tuisawau, 11 Adam Harvey, 12 Chris Hurley, 13 Rob Carter, 14 Pete Newguy, 15 Alex Pride. Replacements: James Dean, Steve Frost, Toby Everleigh.