you really need to be smarter and have much better grades than an average med school applicant.
I am not sure that is absolutely true, and wouldn't take that as a discouragement from applying.
Admissions is full of mysteries and myths, and while it can be informative to ask others (including other students, rankings, US News & World Report, etc), take all you hear (especially all discouragement!) with a grain of salt. In the end, statistics and odds (whatever they are for people in your situation) won't matter-- there are just three things that can happen to you from applying, and those are 1) Accepted, 2) Waitlist, 3) Rejected. Whether the odds are 99% in your favor or 99% against you, you'll await the three possibilities with just as much trepidation.
I have known non-US citizens (international students, permanent address is outside the country) going through US med schools. I would not worry too much about whether it is possible to get accepted into a school-- the bigger issue is whether you can pay for it, since US education is primarily self-funded rather than government-funded as it is in many countries. Students pay for med school by getting loans.
It is difficult to get a student loan from the US government if you are not a US citizen-- the money simply is not available in most cases. International students are generally not eligible for government student loans (which are more low-interest); they are eligible for private bank loans (not funded by the government). But that will cost you lots and lots of interest. There may be some cases where you could access private (non-government, i.e. non-tax-raised) money to help you out-- loans, grants, and scholarships offered by the school itself if it's private, by foundations, etc.