Everybody's had great suggestions so far! You can also go out to Frenchmen Street pretty much any night of the week to hear awesome music, for free. Obviously you'll have to pay for drinks, but only the Spotted Cat has a one drink minimum; otherwise you can pop in and check out the band at DBA, Apple Barrel, Maison de la Musique and other venues for free. Sometimes you'll find places that are charging a cover, but there are always free shows on Frenchmen Street, which you can find simply by walking across the street or going next door.
Definitely check out Wednesday at the Square, the free concert series in Lafayette Square, but you'll have to wait until the 2011 season. The concerts take place over 12 weeks in the spring, and they ended about a month ago. However, there's a new concert series at Lafayette Square that happens in the fall, so keep an eye out for Harvest the Music. Same location, but I believe the fall concerts are on a different day. Possibly Thursday.
Walking around the French Quarter during the day, particularly Royal Street, is another free source of entertainment. There are tons of art galleries and antique shops that can occupy you for days, plus there are always musicians playing outside on the streets. If you've got a few bucks to tip them, then do it, but if not there's no reason not to enjoy the music while you're there.
City Park, the Sculpture Garden, Audubon Park and New Orleans Museum of Art (NOMA) are all worth checking out, and won't cost you a dime. If you're a Louisiana resident, you can go to NOMA for free, but I can't remember if it's all the time or just certain days of the week. It's a bit hot right now for the park, so the museum is a good indoor activity.
Welcome to New Orleans! Hope this helps. :)
