Hmmm… I disagree. I think A:AoU is reliant on the other movies far more than Avengers 1 is… Avengers 1 essentially hand waves away origin stories “look, there are superheroes. We are coming together to fight this super villain who has a magic stick.” Nothing else matters. Where did they get their powers? Somewhere. Where did Loki get the magic stick? Who cares? Everything you need to know is right there.
A:AoU however is entirely reliant on the previous film and Tony’s journey through his 3 film franchise (and especially IM3). The driving plot to the movie is “Tony has had enough of being reactive, and so due to his paranoia from the previous films, he is harnessing the alien tech of Avengers 1, he’s going to build his own robot cop force. But since he didn’t understand alien tech… the robot goes bad and still the magic rock that was inside of Loki’s magic stick from the first movie to make a super robot, who turns on him because of the humanity of the programming that was installed in it from IM1-IM3.
I don’t think it works on its own at all.