When working on high voltage projects the first think you will need is a power supply, the type of supply will depend on what you are doing.

There are two things to consider with power supplies, the output voltage and the current it can deliver.
The voltage determines how far the spark can travel in the air and the current decides how much energy that spark has.
When we talk about high voltage we mean voltages from about 1000v to 100,000v and up, and currents up to about 100mA, any more current than this and things will get very dangerous. It only takes 30mA across your heart to stop it.

There are many sources of power supplies. I had trouble getting hold of one until I got board one day and started looking at the things on eBay; I bought a Neon Sign Transformer (NST) on eBay for £30 this outputs 15kV at 20mA (reasonably safe!), this is big enough to power small projects such as a Jacobs ladder.

Neon Sign Transformer (NST)

A Neon Sign Transformer (NST)

As you would expect from the name neon sign transformers come from neon signs (those signs made from a long glowing tube) they produce a voltage of around 10 to 20kV, this excites the gas in the tubes causing it to emit light.
These power supplies are very simple and easy to use, you connect the primary to the mains and a hi voltage appears between the two think wires.
These transformers are available with various different power ratings, the higher the power the bigger the light it can run.

Flyback Transformer

A modified flyback transformer

Fly back transformers are the devices used in CRT's (TV's) to produce the high voltages required to accelerate the electrons from the back of the screen to the front. The electrons are drawn because of the electric field set up by the voltage difference between the front and the back, this doesn't require much current just a high voltage so the use of these power supplies can be limited.
Like a lot of these things they aren't the easiest thing to get hold of, most modern screens have small flybacks which have the coils wound on top of each other or sealed up tight, this makes them very hard to use as you have to utilise the existing windings. I got my flyback transformer from a TV repair shop, it was an old flyback with one half of the C core uncovered, this allowed me to wind my own primary and feedback winding .
There are many websites which go into the construction in lots of detail so I am not going to tell you what to do here.
I managed to get mine running but it didn't last long, the transistor I was using to drive it kept blowing and when it did work the output voltage was very low producing an arc of 4 or 5 mm.

Ignition Coil

A common method for producing a high voltage is an ignition coil, these are the devices that produce the high voltages that the spark plugs in a car requires. The ignition coil is made up of a primary and a secondary sealed inside a container, unlike the flyback with its feedback winding you will require an oscillator to get it to work. A simple 555 timer chip connected to a high current transistor is all that is required, the 555 needs to oscillate at a high frequency in order to produce the high voltage pulse. I haven't built one of these so I can't comment on its performance, but from what I have seen you can get sparks of up to a foot. I plan to build one at some point just to see what it is like.

Microwave Oven Transformer (MOT)

A microwave transformer is as you would expect a transformer for a microwave; these produce a lot of power and can be very dangerous, they only output about 2 to 4 kV but they output a lot of current, several hundred mili amps. As they only output a relatively low voltage there use on there own is limited, although if you connect several of them up you can get a decent high voltage high current supply. If you do decide to go down this route you will need to watch out as these transformers aren't current limited, that means that the output isn't fixed at 100mA or 500mA it will go up as you draw more current, eventually leading to the destruction of the transformer. You can current limit these by building an inductor; these are commonly built out of microwave transformers! If you don't have another transformer or don't want to make an inductor you can connect an electric heater or light bulb in line.
A microwave contains several other useful parts, high voltage diodes, capacitors, and a magnetron so if your interested in high voltage stuff and you lay your hands on one keep it.

My MOT power supply

Pig Pole (utility pole transformer)

These have to be the biggest of all supplies; these transformers are used in rural areas to supply power to groups of houses (residential areas use a much larger transformer to supply large numbers of houses).
If you want to build something very large you will need one of these, like the microwave transformers these aren't current limited so you will need to use a decent inductor.
Apparently they can be obtained through your electricity provider (bought as scrap).