1. Preheat the oven to 325˚ and arrange a rack in the middle. Coat a 9-by-5-inch loaf pan with nonstick cooking spray.
2. Make the cake: In a medium bowl, whisk the flour with the baking powder and salt and set aside. In a large bowl, whisk the eggs with the sugar until combined; add the lemon zest, olive oil, milk, brandy and orange juice and whisk again until combined. Add the dry ingredients to the wet ingredients and whisk until combined.
3. Transfer the batter to the prepared baking pan. Bake the cake, turning halfway through, until golden brown and a toothpick inserted into the middle comes out clean, about 1 hour 15 minutes. Let the cake cool on a rack for 30 minutes, then run a knife around the edge of the loaf and invert it onto the rack and let cool completely.
4. While the cake cools, make the apples: In a medium skillet, cook the sugar and water over medium heat, swirling the pan (do not stir the sugar and water with a spoon), until dark amber, about 8 minutes. Fold the apples into the caramelized sugar. Add the cider and Calvados and cook over medium heat until the apples are tender and translucent, about 8 minutes.
5. Cut the loaf into 8 slices and divide among 8 plates. Garnish with the glazed apples and walnuts and serve immediately.