One hopes this myth is no longer subscribed to by anybody, but just in case...

In the early days of RL, much of the research in the field was focused on developing new algorithms and

1. analyzing them for basic kinds of soundness such as asymptotic convergence and for this assuming lookup tables sufficed, and
2. empirically testing them for basic correctness and again for this experiments with small problems in which one can use lookup tables sufficed. Thus many viewers of early RL talks got treated to many research papers with RL and lookup tables. This may account for the myth say in the early 1990's. It should have frittered away given that all of the success stories in RL use function approximation. Indeed, the two basic text books on RL (Sutton & Barto's Reinforcement Learning: An Introduction and Bertsekas & Tsitsiklis' Neuro Dynamic Programming) present many results and much discussion of RL and function approximation.

Of course, even today when someone develops a new RL algorithmic idea they often test it first on small lookup table like problems. This may account for the few holdouts on this otherwise untenable myth.