I can't providde specific explanations of the passaage cited, but it
seems to me that the poem as a whole is generated by sexual
frustration. Prufrock grows old (at least fears he will) still a virgin,
and obsesses over occastions that might have led to sexual experience
but (perhaps due to his inability to force the moment to its possible
end) remained unfulfilled: coffee spoons but no going to bed. The You is
a general term for the woman (or women) with whom he has associated.