You Can Hear Them Too If You Pay Attention