In a saucepan over medium heat, combine butter, brown sugar, corn syrup, and sweetened condensed milk. Stir constantly until mixture begins to bubble, then reduce heat to low and continue stirring for 5–7 minutes until thickened and golden. Remove from heat, stir in vanilla extract, and immediately pour caramel over cooled shortbread. Spread evenly. Let it sit at room temperature for 1 hour or in the fridge for 30 minutes.