Responding to the adhan holds great merit and reward in Shariah. It is not proper for one to be engaging in futile talks while the adhan is being called.

Have not found a Hadith with the exact words as in your question, however the following is a very similar narration.

Nabi (sallallahu ‘alayhi wa sallam) is reported to have said:“One who speaks during the adhan, it is feared that he will lose his Iman”‘. Allamah Saghani (rahimahullah) has classified the above Hadith as a fabrication. (See Mawdu’atus Saghani, Hadith: 145 and Kashful Khafa, Hadith: 2437)

The narration stating that one who speaks during the Adhaan will not be able to recite the Kalimah at the time of death is a fabrication, as mentioned in Kashful-Khifaa and San’aani.

However, in spite of the narration being a fabrication, it is still necessary and highly emphasized that one should stop all work and listen to the Adhaan as well as reply to its wordings.

And Allah Ta’ala Knows best

3 Fatwa links below 

https://hadithanswers.com/talking-during-the-adhan/
http://islamhelpline.net/answer/6033/the-holy-prophet-saw-said-stop-doing-everything-during-the-azaan-even-reading-the-quran-the-person-who-talks-during-the-azaan-will-not-be-able-to-say-the-kalima-e-shahada-on-his-her-death-bed
https://islamqa.org/hanafi/askmufti/45280/hadith-on-talking-during-athan/