You could make a case for either of these, but I will argue that the Roman Empire really fell earlier than 476 AD.
I would argue that the "fall" of the Western Roman Empire came as Rome lost its prestige and much of its power. This can be seen happening as early as the death of Constantine in 337. This led to civil wars that only ever ended for a very short time. This, along with the split in the Empire, reduced the importance and prestige of Rome.
This reduction in prestige can also be seen in 404 when the capital of the Empire is moved to Ravenna. The need to move the capital to a more defensible spot is a clear indication of decline. Rome itself was sacked in 410 and again in 455. Both of these seem to be indications that the Western Roman Empire had "fallen."
I would see the fall of Rome as a process, not a single event. I would say that it started in earnest long before 476.
Your question as well as the above answer deal with semantics more than with actual history. Unquestionably the Western Roman Empire came to an end and unquestionably this was a process, like most important events in history. Whether or not it actually "fell," or when it "fell" is of little consequence.
Historians generally accept that the process which saw the end of the Roman Empire commenced in 376 C.E. when the Eastern Roman Emperor, Valens, allowed tribes of Visigoths to cross the Danube River to escape the on-coming Hunnish invaders. Hostile treatment by the Romans caused the Visigoths to revolt, and at the ensuing Battle of Adrianople, the Romans were defeated and Valens killed. Thereafter, the Germanic Tribes maintained an existence within the Empire. At the same time, a number of dynastic disputes within the government and the army further weakened the Empire. Ultimately, the Empire came to an end with the forced abdication of Romulus II Augustulus in 476 C.E. I often tell my students that the empire ended, in the words of T.S. Eliot, "not with a bang, but a whimper."