Capacitors are relatively low-cost and cheap components. Unless they are specialised and designed for a specific electrical circuit or system they are low-cost and cheap to replace. Capacitors come in a variety of different types, sizes, and operating voltage ranges.
High stability with temperature and time similarly is not so expensive. On the other hand, precision capacitors are not cheap and generally unavailable at a sensible price at better than 1% tolerance. If you can use NP0 ceramic of relatively low value they are not too bad in stability and price, provided you don't demand tight tolerance.
The advantages of using capacitors are: When a voltage is applied to a capacitor they start storing the charge instantly. This is useful in applications where speed is key. The amount of time it takes to fully charge the capacitor depends on its type and how much voltage that they can store.
On the other hand, precision capacitors are not cheap and generally unavailable at a sensible price at better than 1% tolerance. If you can use NP0 ceramic of relatively low value they are not too bad in stability and price, provided you don't demand tight tolerance. Inductors tend to be at least as bad as capacitors.
Since many resistor applications involve RATIO's (i.e. voltage dividers, etc), all the resistors are matched and they have the same temperature effects, they'll make for a very stable circuit. Generally it is much cheaper to get tight tolerance resistors than capacitors.
Like any component that we use in the world of electrical circuitry and machinery, capacitors have some certain drawbacks and disadvantages. The disadvantages of using capacitors are: Capacitors have a much lower capacity of energy when compared to batteries.