An “option select” is any technique that takes advantage of ambiguous behavior from a game system. This is not really on the same level as a bug (certain option selects may, in fact, be encouraged by the developers), but utilizing it may be considered cheap in some overwhelmingly bad cases, to the level of being buggy.
One example of an option select comes from Street Fighter III: 3rd Strike (and is available in the exact same form in Street Fighter IV), where a normal throw attempt cannot be performed crouching, but a throw escape can. Therefore, where one suspects the opponent may attempt a normal throw, one can crouch and input the throw command; this will come out as whatever normal move attack button has input priority, or it will escape the throw. It will not attempt a throw.
The ambiguity lies in that the throw command, when crouching, could be interpreted as a throw escape or a normal move attempt. This ambiguity does not exist when standing; the throw command will lead to a throw attempt, or a throw escape if already being thrown. Thus, a crouching throw escape attempt is a much better choice when a throw seems imminent: if your suspicion is wrong, you will not have whiffed a throw attempt, and you may even land a hit if you are lucky. Of course, the opponent also has options to press: if they decide to stuff anything you try, then it doesn’t matter which way you attempt a throw escape, because any ideal stuffing move will stuff the poke and smash the (whiffed) throw attempt. Therefore, this particular option select is advantageous, but it is not unfair.