Turn on the thumbnails.

Once a video has been added to the menu, you can open the button's settings by selecting it from the tree view on the left side panel. You can also hover over the button and select the settings icon.

In the lower-right of the button settings, select your preferred option for the thumbnail (Upload Image, Use Current Frame, Reset to Default). The default image is the poster frame of the video. If you choose "Use Current Frame," it will capture a frame from the current time in the player and use it as the thumbnail.