This criticism may be interesting, but I don't think it is sound. After all, a vegetarian lifestyle, while not perfect, is still better, that is, morally preferable, than a non-vegetarian one.
Another answer to this criticism comes from the poetry of William Blake, famous for its distinction between innocence and experience - embodied in his collection of poems entitled "Songs of Innocence" and "Songs of Experience". In his earlier poetry, Blake appeared to believe that innocence was irretrievably lost and existed in a dynamic tension with experience. Thus, every poem in "Songs of Innocence" has a counterpart in "Songs of Experience" and they have to be read with each other in mind.
However, later, it appears, Blake did come to believe in a type of return to innocence, not a simple return, but a more nuanced one, summed up in the slightly paradoxical phrase "organized innocence". I doubt that Blake would have applied this to vegetarianism, but I think his concept helpfully provides an answer to the "impossible return to innocence" criticism. Through their lifestyle choices, vegetarians are creating a state of organized innocence, one in which they rationally organize their lives (i.e. using experience) so that they can live more peacefully with the rest of nature (i.e. innocence). In a sense, they are planning for compassion to play a larger role in their lives, providing for what Hindus call "ahimsa".