Before 2010, the ATM was the best source for least amount of fees - was easy to find a bank with less than 1% tacked on fees.
No more - now just like credit cards, most banks typically charge 3% currency conversion fee in addition to a fixed foreign ATM use fee.
For the infrequent traveler, any choice is fine - even with 3% costs plus fees you lose just 3-4 cents per dollar, so exchange cash (comparison shop before settling on a place), or use a credit card (no fee for Capital One, 3% for nearly all others), or use a ATM (0-3% fees) - whatever is more convenient.
Credit Cards: The Capital One credit cards do not yet charge any extra fees for credit card transactions. Now that the free ride on ATMs is history, credit cards may be a good deal even for smaller transactions of $20 or more, certainly credit cards are now the better choice for larger transactions such as hotel stays - no need to lug around too much cash. Other than Capital One, all other credit cards seem to charge at least 3% in fees.
See Foreign Exchange Fees Going Up for more details on both ATM and credit cards.
ATM: Daily limits are usually $400 to $500 and this depends both on the bank owning the ATM machine as well as the bank where the account is kept. And most banks charge a $2 fee or more for foreign ATM cards. Carry two ATM cards and have a backup plan since the likelihood of ATM card use denials is quite high.
Best option is to use ATM cards from credit unions or investment accounts such as E*Trade (0-1% or so actual fee charged) and they may also provide ATM fee refunds. This may end up being better than the so-called 0% conversion fee cards.
While some 0% currency conversion fee cards show up in many travel discussion websites, the two top cards are actually quite problematic and in the end not good deals at all. Capital One 360 ATM card claims 0% conversion fees - but it rarely works overseas - for "security reasons" they disallowed most transactions over two visits to Canada. And they do not refund any ATM fees. And even after calling customer support, they said they cannot manually fix the security software which kept denying ATM transactions! Charles Schwab also claims low fees but using their website is very cumbersome and they have very poor customer service (difficult to link accounts, they reject mobile app check deposit applications, they require too many phone calls to do anything).
Of the big banks, Bank of America was the best way to get cash, since its ATM holders can use the Scotiabank ATM machines without foreign ATM fees and they had the low currency conversion fees of just 1%. After November 2013 the currency conversion fee is now 3%. Visit the Bank of America site for names of countries and partner banks. Note that using a bank not in that network will result in even more fees.
Cash: In Montreal, with a bit of shopping around, one can find very good exchange rates at some places. Easy to find some that beat the 3% charges of ATM and credit cards.
After having looked around at many exchange shops - many are on Ste Catherine, 4-5 blocks on either site of McGill street. Prices vary a lot - but no need to look around too much. There is an easy trick to find a if a specific exchange house has best rate - look at the spread between their posted buy and sell rate for a single currency, say USD. If the spread is 2% or close, that is an excellent rate for converting your USD to CAD. Some big exchange shops have spreads of 6-8% - that is an indication of costly exchange. Just walk a block or two more, there will be another place with better rates. If in the Chinatown area, try 1112 St Laurent just below Rene Levesque. The rate is very good and commission is the standard CAD$2.50. Of course, if you can mostly use a good credit card like Capital One for large transactions, there may be no need to get USD changed into CAD.