Deleting files with Shift+Del in Windows 95/98

Over all those years using Windows 95/98/98SE, I had issues with permanently deleting files with the Shift+Del key combination: Either I'd get the vanilla deletion dialog (after which I had to retry) or sometimes I'd hit lucky with a real permanent file deletion dialog.

This Summer I discovered that the trick was to hold the Shift key until the permanent deletion dialog appeared, which then seemed to solve my problem with all this.