Somehow, "finite" must exist within infinite, not only because, as has already been said, infinite is infinite, meaning that it contains or encompasses all there is (and all there is not!), but also because if there were no such thing as "finite" how could we be talking about it? Where would the words come from? The concept itself?
Now, we can say that finite is an illusion. But that does not mean that finite is not real. Clearly, illusions are real; we all live and die by them every day.
So long as we believe ourselves to be the bodies we seem to be living in, then we are caught in the separative universe each of us calls "my life". And it is not helpful, in my opinion, to say "it's not real" because as long as I think I am "me, not you", all of the ramifications of that perspective (birth, disease, disappointment, disaster, death, etc. as well as the "happier" aspects) are "real", as real as necessary for me to feel them, experience them, enjoy them, dread them. What would be the point, for example, of telling the relatives of the 200,000 dead in South Asia that the earthquake/tsunami was not real?.
What distinguishes illusions is not that they are not real, but that they are not binding. That's the definitive characteristic of illusions, and particularly of "the world".
We are bound by an illusion, any illusion, only as long as we accept its limitations as our own.
As for Einstein, that a mind that reached so successfully so far into the unknown could still allow itself to proclaim "this far and no further" is most interesting, not to say confusing. This is a clear example (in my opinion) of a mind saying, "I like it here". It is, of course, perfectly okay for us to do that; but in doing so, we must remember, choices have consequences.