The standard objection I've heard to this line of reasoning is that Jesus seems to show concern for his mother from the cross: "When Jesus then saw his mother, and the disciple whom he loved standing nearby, he said to his mother, 'Woman, behold, your son!'" (John 19:26).
To my mind, both interactions stem from an early and widespread impression of Jesus' preaching career. Many of Jesus' followers carried forward the message that biological families were secondary to the metaphorical family of those who "hear the word of God and obey it" (either secondary, or altogether contrary to the the new kingdom of God). In short, the words from John's crucifix contribute to the general impression that Jesus encouraged his followers to form a metaphorical family, one not based on biological ties or patrilineage.
So, the question: What would Jesus do on Mother's Day?