Reference for Lectures

1. Lecture notes by Milne(click)

[MilCF] class field theory
[MilMF] modular functions and modular forms
[MilEC] elliptic curves
[MilAV] abelian varieties

2. Some useful books and lecture notes
[CF67] J. Cassels and A. Frohlich (editors), Algebraic number theory, Proceedings of an instructional conference organized by the London Mathematical Society (a NATO advanced study institute) with the support of the International Mathematical Union. New York, NY: Academic Press, 1967.
[Antwerp3] Modular functions of one variable, III (Proc. Internat. Summer School, Univ. Antwerp, Antwerp, 1972), Lecture Notes in Mathematics, Vol. 350, Springer, Berlin, 1973.
 [Hid93] Hida, Haruzo Elementary theory of $L$-functions and Eisenstein series. London Mathematical Society Student Texts, 26. Cambridge University Press, Cambridge, 1993.
[Bum97] D. Bump, Automorphic Forms and Representations, Cambridge Studies in Advanced Math., Cambridge Univ Press, 1996.
[Ed97] Bas Edixhoven, The modular curves X_0(N), Trieste (1997)
[Iwa97] Henryk Iwaniec, Topics in Classical Automorphic forms, Graduate studies in mathematics, Volume 17, 1997
[Jac97] H. Jacquet, Principal L-functions for GL(n), Representation theory and automorphic forms (Edingburgh, 1996), 321-329, Proc. Sympos. Pure Math., 61, Amer. Math. Soc., Providence RI. 1997
 [IAS01] B. Conrad and K. Rubin (editors), IAS/Park City Math. Ser., 9, Arithmetic algebraic geometry (Park City, UT, 1999), 233--406, Amer. Math. Soc., Providence, RI, 2001.
[LGS03] D. Bump, J. Cogdell, E. de Shalit, D. Gaitsgory, E. Kowalski, S. Kudla, An introduction to the Langlands program. Lectures presented at the Hebrew University of Jerusalem, Jerusalem, March 12--16, 2001. Edited by Joseph Bernstein and Stephen Gelbart. Birkhauser Boston, Inc., Boston, MA, 2003. x+281 pp. ISBN: 0-8176-3211-5
[Gold09] Dorian Goldfeld, lecture notes, Spring 2009 & Spring 2010

3. Comprehensive references (which cover elliptic curves, modular forms and more)
[FLT1] V. Kumar Murty (editor), Seminar on Fermat's Last Theorem, CMS Conference proceedings, Vol 17, 1995
[FLT2] J. Coates and S.-T. Yau (editors), Elliptic Curves, Modular Forms, and Fermat's Last Theorem (2nd ed), International Press, 1997
[FLT3] G. Cornell, J. Silverman and G. Stevens (editors), Modular Forms and Fermat's Last Theorem, Springer, 1997
[DS06] Diamond-Shurman, A First Course in Modular Forms, Springer-Verlag, Graduate Texts in Mathematics 228, 2006

The topics

References

Review of class field theory

[CF67], chapters 1, 2, 6, 7
[MilCF]

Tate's thesis

[CF67], chapter 15
[LGS03], Kudla's article.
[Bum97], chapter 3.1

Various approaches to modular forms and automorphic representations

J.-P. Serre, A First Course in Modular Forms (last chapter), Springer-Verlag, Graduate Texts in Mathematics 228, 2006
[Bum97], all chapters
William Casselman, On some results of Atkin and Lehner, Math.Ann. 201, 301-314 (1973)
[Ed97], [Iwa97], [Jac97]
Dorian Goldfeld, lecture notes, Spring 2009 & Spring 2010 (will be available on the web soon).

Abelian varieties

[MilEC], [MilAV]
Joseph H. Silverman, The Arithmetic of Elliptic Curves (2nd Edition), Springer-Verlag, Graduate Texts in Mathematics 106, 2009
D. Mumford, Abelian varieties, Oxford Univ Press, 1970 (2nd ed 1974)

Eichler-Shimura-Deligne construction of Galois representations attached to modular forms

B. Conrad, Appendix to Ribet and Stein: The Shimura Construction in Weight 2, [IAS01]
T. Saito, Galois representations and modular forms
P. Deligne, Formes modulaires et representations l-adiques. Seminaire Bourbaki 355; Lecture Notes in Math. 179 (Springer-Verlag 1971) pp. 139-172  

Introduction to Serre conjecture

H. Swinnerton-Dyer, On l-adic representations and congruences for coefficients of modular forms, pp. 1--55, [Antwerp3]
K. Ribet and W. Stein, Lectures on Serre's conjectures, [IAS01]
J.-P. Serre, Sur les representations modulaires de degre 2 de Gal(Qbar/Q), Duke Math. J. 54 (1987), 179-230
C. Khare, Serre's conjecture and its consequences, Japan. J. Math. 5, 103-125 (2010)

Ribet's converse of Herbrand's theorem

K. Ribet, A modular construction of unramified $p$-extensions of $Q(mu{p})$. Invent. Math. 34 (1976), no. 3, 151--162.

Theory of p-adic modular forms following Katz and Hida

N. Katz, $p$-adic properties of modular schemes and modular forms. pp. 69--190, [Antwerp3]
[Hid93], chapter 7.
N. Katz, Higher congruences between modular forms. Ann. of Math. (2) 101 (1975), 332--367.

Introduction to Modularity Lifting Theorem

M. Kisin, What is a Galois representation? AMS Notices (2007)
R. Taylor, Galois representations. Ann. Fac. Sci. Toulouse Math. (6) 13 (2004), no. 1, 73--119.
M. Harris, Arithmetic applications of the Langlands program, Japan. J. Math. 5, 1-71 (2010)

Introduction to the Langlands correspondence between Galois representations and automorphic representations

[LGS03]
A. Knapp, Introduction to the Langlands program
S. Gelbart,An elementary introduction to the Langlands program