Muslims Don't Worship a Black Box
What we should point out here is that our prayer facing the direction of the Ka’bah has nothing to do with the structure of the Ka’bah, rather it has to do with its location. If it were to be destroyed, we would still pray in that direction, not to that structure.
Hence we find that nowadays Muslims pray on the second storey and on the roof of the Grand Mosque in the Sanctuary of Makkah, facing the direction of the Ka’bah even though it is not directly in front of them. This is what millions of Muslims do throughout the world, praying in the direction of the Ka’bah even though they cannot see it. This demonstrates the great difference between the laws of Islam and the actions of the mushrikeen (polytheists) whose worship of their idols, stones and trees ceases when these objects cease to exist. Therefore if the mushrik cannot see his object of worship or idol, he does not face the direction of that thing.
Kissing any place on earth is bid’ah (an innovation), apart from the Black Stone. Were it not that this is following the example of the Prophet (peace and blessings of Allaah be upon him), kissing the Black Stone would be bid’ah too. ‘Umar (may Allaah be pleased with him) used to say: “I know that you are only a stone and can neither cause harm nor bring benefit. Were it not for the fact that the Prophet (peace and blessings of Allaah be upon him) kissed you, I would not have kissed you.” Hence it is not permissible to kiss the covers of the Ka’bah or its stones...or to touch them with the intent of seeking blessing from them.
(Extracts from IslamQA.info)