Indeed. I feel I can't stop an audio either without losing track slightly of what's happened when I start it again, unlike a book.
But mostly, I just can't stand the voice acting, the sound effects in the background and can just never get to grips with something being described to me like they do. The last one I listened to was Sword of Truth. Garro sounds constipated, Hakim just sounds awful as some generic Asian and sometimes quite hard to understand, and lets not forget Varren, who sounds like some London gangster.