Since the puns in the ivory and horn gates were Greek I thought that
the use of these symbols must have pre-dated Virgil so I poked around
the internet a bit more and found this:
"Hear, then, I said, my own dream; whether coming through the horn
or the ivory gate, I cannot tell."
Charmides, or Temperance
Written 380 B.C.E
Translated by Benjamin Jowett
Then that seem to looked like the symbol of the gates had been been
around awhile so I searched some more and found this.
"This dream, Madam," replied Ulysses, "can admit but of one
interpretation, for had not Ulysses himself told you how it shall
be fulfilled? The death of the suitors is portended, and not one
single one of them will escape."
And Penelope answered, "Stranger, dreams are very curious and
unaccountable things, and they do not by any means invariably come
true. There are two gates through which these unsubstantial fancies
proceed; the one is of horn, and the other ivory. Those that come
through the gate of ivory are fatuous, but those from the gate of
horn mean something to those that see them. I do not think,
however, that my own dream came through the gate of horn, though I
and my son should be most thankful if it proves to have done so.
Furthermore I say- and lay my saying to your heart- the coming dawn
will usher in the ill-omened day that is to sever me from the house
of Ulysses, for I am about to hold a tournament of axes. ...
Written 800 B.C.E
Translated by Samuel Butler
Even that looks like it might not have been the first source but I
don't care any more. I give up.