Get a PS2 or at least make sure your computer is capable of emulating the PS2. And a PSP. Below is a list of MegaTen (and spin offs) games that are available in english.
Standard MegaTen
Shin Megami Tensei and Shin Megami Tensei II (SNES) Aeon Genesis made translation patches for these two. I've only partially played SMT. Apparently SMT II is better. There's also Shin Megami Tensei If which if I remember correctly is presently being translated. Both are played in the First person perspective.
SMT III Nocturne or SMT Lucifer's Call if you live in Europe (PS2). IT got translated and is a very good game. IT gives you a good challenge and visuals are very nice. The story is typical MegaTen awesome. It is rather minimalist as you don't buy armour and that type of shit or use a contemporary party system. You can also get Dante off DMC in your party as well which is pretty cool.
SMT Strange Journey (DS). Spiritually it is SMTIV and but the gameplay fits with the other MegaTen titles. First person dungeon crawler, no traditional party system. Fucking awesome game.
Persona
Persona (PS1/PSP). Personally I love Persona. I like the story, the characters and the setting. The game is flawed as fuck. Yet I still love it. The battle system is the greatest but its still more than playable. There's a conventional party system. Though play the PSP version. The PS1 version is pretty fucked up regarding game balance, has the Snow Queen quest, translation and the fact they changed a character's ethnicity.
Persona 2 Innocent Sin (PS1/PSP). Good game. I liked it. The PS1 version you can emulate it but you have to apply a translation patch by Gemini. The PSP version was translated by Atlus. It has a better menu interface which it makes better than the original in my opinion.
Persona 2 Eternal Punishment (PS1/PSP). The PSP version isn't in english so your stuck with the PS1 version which is.
Persona and the Persona 2 duology feel similar to the nominal MEgaTen titles. Persona 3 onwards, shit changes into the life simulator/ dungeon crawler.
Persona 3 (PS2/PSP/PSN) Persona 3 FES is the best version imo. As it contains an epilogue which concludes the story of the game. P3P is a good version particularly as you can play as a girl and have complete control of you party. Whilst in the PS2 version, you give out broad order i.e. you tell a party member she has to heal and the AI plays on that.
Persona 4 (PS2/PSV) AWESOME. Essentially the same structure to P3 but with an improved battle system.
Spin Offs
Devil Summoner games (PS2) Can't get my hands on them so no comment. They cost a few dabloons. More action orientated than previous MegaTen titles.
Digital Devil Saga 1 and 2 (PS2). I've only played DDS1 and I rather like it. It uses the same engine as SMT Noctune. Quite a few people absolutely love it.
Devil Survivor 1 and 2 (DS/3DS). Good games but completely different as its a SRPG. Story is good MegaTen WTFs.
Personally I suggest Noctune, Strange Journey or P3/P4 as a better way to get into MegaTen. The former two are quintessential MegaTen in terms of gameplay mechanics (dungeon crawling). Whilst Persona 3 and 4 are really accessible in comparison to other titles. Personally I don't think the Devil Survivor games is the best way to start as the gameplay and story structure is different though it has the accessibility of Person 3 and 4 as you a (relatively)conventional party system.