Agree with digihub00.
Also depends on: type of music you are playing, whether you intend to sequence or not, and the type of keyboard.
As digihub00 said, if one patch has more than one element per voice, then your polyphony is going to drop. But keyboards with 32 notes polyphony don't usually do that - they will program them to maximise polyphony. This also means the patches will sound "thinner".
For digital pianos, usually polyphony will not be an issue unless you decide to layer sounds. Digital pianos, unlike synths, usually have one element per patch (except perhaps the later ones). So you don't usually run out of polyphony, except when you layer 2 patches, then the polyphony drops. But it is usually not a big problem. If you layer 2 patches with short decay (eg piano and epiano), the short decay will compensate for the need for more polyphony. If you layer patches with long decay or sustain (eg piano and strings), you can get into trouble. But then again, you don't step on the sustain pedal too much or your pads/strings will become mushy. In fact, you'll hardly use your sustain pedal in this case, which will mean you'll compensate yourself in your playing and polyphony should not be an issue. But for layering, I would recommend 64 notes polyphony. The difference between 64 ane 128 polyphony for a digital piano is not huge, but 32 and 64 can be significant if you layer.
Synths are different - they almost always have more than 1 element per patch (Yamaha being the worst of the lot - in terms of elements per patch). So 64 notes polyphony is the bare minimum. Latest synths must have 128 because of their complex programming.
If you do sequencing, then 32 notes polyphony is almost certainly not sufficient. But again, if you are doing very simple sequencing, you may just about make it, but I wouldn't count on that.
Type of music you play is also important. Which is why you find analog synths (which we use for leads and some pads) to have very low polyphony (8-16) - and that is actually sufficient.
In your case, Casio don't usually have complex programming. If you are just going to play piano sounds, 32 notes polyphony is enough. If you want to layer, get at least 64. Long-winded answer to your question...