Level 1/2/3.. does not say anything on where it is located. The number just refers to its relative position in the cache hierarchy. L1 is closest to the core, typically smallest and fastest. L2 is typically bigger and slower, and 'further away' (both physical, distance, and in latency). L3 is even bigger, slower and further, etc. Some Xeon's even have L4 caches, like on the chipset, and if you like, you could consider main memory (RAM) as a level 4 or 5 cache to cache your harddisk.
L1 stores data and instructions for immediate use and is the fastest but also fairly small, generally measured in KB, L2 is slower but significantly larger, sometimes shared between cores, sometimes only used by a single core it serves as a much faster access than the main system memory, L3 is the slowest but also the largest on CPUs that have it, it too is much faster than having to call the system memory and is shared among all of the cores on a CPU. Within a certain family line of CPUs more L2 or L3 will generally add a slight performance increase, but it the performance effect is hard to tell just by looking at specs especially when comparing between lines, because the performance is effected by architecture significantly more.
It just depends on what the computer will be used for; if it is just for browsing, playing 3D games, and playing media, a CPU without the L3 cache will do.
If you do a lot of encoding, programming, video editing, etc, then the CPU with a L3 cache will help significantly.
