Preheat oven to 350°F (175°C). Line baking sheets with parchment paper.
In a medium bowl, whisk together flour, baking powder, baking soda, salt, cinnamon, nutmeg, ginger, and cloves.
In a large mixing bowl, beat butter with granulated sugar and brown sugar until light and fluffy.
Add egg, vanilla extract, and pumpkin puree, mixing until combined.
Gradually add dry ingredients to wet ingredients, alternating with the milk, and mix until just combined.
Drop heaping tablespoons of dough onto prepared baking sheets, spacing about 2 inches apart.
Bake 10–12 minutes, or until cookies are set and lightly golden. Let cool completely on wire racks.
Meanwhile, prepare the frosting: beat softened butter until creamy, then add powdered sugar, vanilla, cinnamon, and 2 tablespoons milk. Mix until smooth, adding more milk as needed for spreadable consistency.
Frost cooled cookies generously with cinnamon frosting.
Optional: sprinkle with a light dusting of cinnamon or drizzle with caramel for extra flavor.