Turns out the Taskbar Thumbnails Tuner link somebody else posted works by changing the following 2 registry keys:
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Advanced
Create new DWORD: "ExtendedUIHoverTime" and set it to a really high number (it's in milliseconds) to delay the preview popup virtually forever (over 7 weeks, anyway). Apparently it doesn't like it if you set it to exactly 0xffffffff, I think it treats it as 0. But 0xfffffffe works just fine. Kind of weird.
and
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Taskband
Create new DWORD: "NumThumbnails" and leave it at 0 to switch the previews to a list when you actually click on a stacked icon.
You'll need to restart Windows Explorer for it to take effect, easiest way is from the Task Manager.
source:
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Advanced
Create new DWORD: "ExtendedUIHoverTime" and set it to a really high number (it's in milliseconds) to delay the preview popup virtually forever (over 7 weeks, anyway). Apparently it doesn't like it if you set it to exactly 0xffffffff, I think it treats it as 0. But 0xfffffffe works just fine. Kind of weird.
and
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Taskband
Create new DWORD: "NumThumbnails" and leave it at 0 to switch the previews to a list when you actually click on a stacked icon.
You'll need to restart Windows Explorer for it to take effect, easiest way is from the Task Manager.
source: