Go to Home Depot or Lowes and pick-up some Loctite. It's used to keep screws and bolts/nuts from backing out. I'm sure it would work for this to hold it in tight without being as permanent as glue or silicone. Good luck