Preheat the oven to 450°F. In a small bowl, whisk 1/4 cup of the olive oil with the rosemary, lavender, and garlic. Season the lamb all over with salt and pepper.