In many rpgs, spellcasting is an extremely powerful option that is difficult to curtail without shutting it down entirely (or at least creating a risk of shutting it down entirely). While it’s often fairly straightforward to make life more difficult for weapon-users without making them entirely ineffective, that can be harder for spellcasters. Especially when spells are a very limited resource (such as using spell slots or prepared spells), even things that can be used to put a weapon-wielder at a disadvantage (such as a penalty to attack rolls)
The following options are specifically designed for Pathfinder 1st ed, Pathfinder 2nd Ed, Starfinder, and 5e, but could certainly be expanded to a wider range of games by an experienced GM.
(Art by DM7)
You CAN use antimagic shells as minor hindrances if you make them very small, and spread them out. And don’t allow the main villain to just sit in one and be immune to all magic. A few small areas where magic works but spells cannot be cast (perhaps strange metoric iron disrupts the act of conjuring the power for a spell, but doesn’t negate magic in general) can become a form of battlefield terrain spellcasters just need to work around.
Rather than make it impossible to cast spells or highly likely that efforts to do so will result in failure, you can make spellcasting take additional effort. In Pathfinder and Starfinder, casting times of 1 standard action become full rounds. In 5e, you cannot move or take a bonus action or reaction in a round you cast a spell with a casting time of 1 action. In Pathfinder 2e, add one action to any spell with a casting time listed in actions.
This option forces a spellcaster to make more tactical decisions, but doesn’t make it any more likely their precious resources are wasted if they take the needed extra time.
Rather than make spellcasting more difficult, you can just slap some consequence on it that hinders or damages the caster for using spells. This can be as minor as one point of damage per level of spell cast, or a minor penalty to saving throws and attack rolls for 1-4 rounds after casting a spell (perhaps that stacks if you rapid-fire spells every round), to more major neative erffects depending on how harsh you want your penalties to be. You could also simply add a risk of penalties, such as forcing the caster to make a Constitution or Fortitude save every time they cast a spell or gain a level of fatigue.
Increased Spell Cost
A much more impactful options it to increase the cost of spellcasting. Perhaps casting a spell requires additional eldritch power, which must come from somewhere. A character could be required to use multiple spell slots, or sacrifice an additional prepared spell.
You could also require the expenditure of some additional resource beyond additional spells. For example in Starfinder you could require a Resolve Point be spend, or in 5e a Hit Die. Pathfinder 2nd edition could require a focus point (though not all characters have focus points). These are pretty steep costs, so it might be smart to have the additional cost only be needed once every 2d4 rounds or so, or even just once per ten minutes, as the spellcaster “attunes” themselves to some specific circumstance.
If you found this helpful or informative, and you’d like to support the creation of more such content, check out my Patreon!