With the circuit you have shown above, the capacitor will delay the LED from lighting for a very short time after you apply power, an may keep it lit briefly after you remove the power, but as shown, with no switching, the capacitor has no effect.
As the capacitor's stored charge builds, it "sucks" more and more available current from the lightbulb, causing it to dim. Feel free to poke any holes in my theory. Youre second guess is correct. A capacitor will not discharge itself unless the battery is taken out of the circuit and a complete closed path remains.
Since capacitor values are not very precise, and the LED doesn't produce much light at low currents, we can estimate the amount of time the capacitor can power the LED as the time constant of the RC circuit. It's not the exact time, but it will give you a ballpark estimate of the decay time for the fade out.
That's why you aren't seeing the LED fade off, because it happens too quickly. There are two things you can do. The first thing you might try is increasing the size of the resistor. This will make the time constant longer, but it will also limit the current, making the LED dimmer. The other thing you might try is using a larger capacitor.
The bulb will therefore glow, but as the charge accumulates on the plates of the capacitor a voltage builds up over it. This voltage opposes that of the battery. The current in the circuit will then decrease as the voltage builds up over the capacitor and eventually stop when the capacitor is charged up to the same voltage as the battery.
Lamps with power rating of 300W or more tend to more or less acoustic noise when dimmed. If this acoustic noise is a problem can be removed by adding a series coil which limits the current rise time to around 1 millisecond. In providing those filtering functions, the chokes themselves can generate a slight buzz.