I was looking back and trying to work out what were the significant steps in my Mandarin learning over the past two years or so. Maybe the most significant step was simply spending a lot of time listening to real Mandarin even though at the time I didn't understand it.
It is not easy to remember the exact details, but for the first three months or so of studying a large part was simply listening to real world Chinese at full speed. I was studying on my own so it was up to me to decide what I did, and against all advice this seemed the obvious thing to do. I don't speak French, Spanish, German, Italian, Russian etc. etc. but at that time I coould confidently identify all of these language, simply because of exposure over my lifetime. If I decided to study any of these languages I would know roughly what I was getting into, Asian languages all sounded similar though (might as well be Martian). Certainly I also listened to a few podcasts, and certainly I did some background reading about the language and picked up a few simple words. Somehow these preparation meant that I ended up listening to about 15 of the Chinesepod Newbie podcasts and a similar amount of the elementary before I started tackling the Intermediate. Even though my vocabulary was limited I seemed to have a huge headstart on speed, coping with different accents etc.
So what happened? via attentive listening the mush of sounds became a stream of syllables, and I was increaingly able to determine differences in sounds that originally sounded the same to my Western ears. I started hearing a few simple words that I had learned (中国 zhong1guo2 china) being an obvious one. All the time I was attempting to be attentive, When do I think sentances are starting and finishing, are there sounds that are common before a pause? can I learn any words from context? what common sounds can I learn so that when I finally learn the meaing it will be obvious. Even at that time I could hear the significance of 个 and was anticipating finding out the meaning of 这个 and 那个.
Finally I enlisted the help of my sons, they dug up some videos from Youtube and played the sounds to me, firstly (because I wasn't actually sure) I proved to myself that yes it was fairly easy for me to identify European languages even vaguely similar ones such as Dutch and German, Spanish and Portuguese. Then I discovered that now I could distinguish Mandarin from Cantonese, and Thai and Vietnamese and Korean and Japanese with ease. Even more interesting the Cantonese apart from having sounds different to Mandarin had at least one tone that was obviously very different (the low level one) so somewhere my brain had already started on the long arduous journey to get a feel for tonal meaning.
What followed was a mastery of Pinyin and relating the sounds to words I heard, the aim was to get to point where I could reliably hear a word in a sentance of real Chinese and look it up in dictionary to find the meaning.
Sound is where it all starts and what you gain from first getting an ear for a language is hard to measure and test (so is unlikely to be emphasized in a formal course). Gaining an ear for language is what you do naturally as a baby with your mother tongue, it seems obvious to me that you should start this way when you study a new languge (and actually because of the prevalance of English media and culture, many foreigners have had this exposure already before they come to formally study English).
I know this view is not popular but it seems to have helped me.