It is NOT the case that the Church teaches that we KNOW all unabaptized people go to hell.
Rather, the Church has always taught that we DO NOT KNOW any way, through our work here on earth, any way by which we can bring to those who die without baptism the grace they need to enter heaven. So, from our point of view, we cannot do a thing to help them avoid hell.
However, the Church DOES NOT teach that souls who have not received water baptism descend into Limbo (the outer fringe of hell). Limbo is not mentioned in either the Catechism of Trent nor in the Universal Catechism that followed Vatican II. It is NOT a concept taught by the Church in a formal way.
A Catholic can believe in Limbo if s/he wants, but no Catholic is under obligation to accept Limbo because the Church does not formally teach that such a place exists.
Catechism of the Catholic Church 1261: "As regards children who have died without Baptism, the Church can only entrust them to the mercy of God, as she does in her funeral rites for them. Indeed, the great mercy of God who desires that all men should be saved, and Jesus' tenderness toward children which caused him to say: "Let the children come to me, do not hinder them,"64 allow us to hope that there is a way of salvation for children who have died without Baptism. All the more urgent is the Church's call not to prevent little children coming to Christ through the gift of holy Baptism."We hope, not with human hope, but with DIVINE hope, that these children enter heaven. Human hope accomplishes nothing at all in terms of salvation, so having human hope alone is stupid and pointless.
Liturgy is not a human work, it is a divine work done by human hands. Liturgy instills divine hope, not human hope. Only divine hope holds out the possibility of heaven to the unbaptized, and that divine hope is instantiated in the Church's liturgy, as the Catechism attests. So, if that hope is present in the Church's own liturgy, then it must be present in us as well.
We do not pray for the devil because he cannot benefit from our prayers - no one can be prayed out of hell. But we do pray for the unbaptized, that God grants them the grace to enter heaven. The fact that this prayer is not only permitted, but enshrined in the Church's own liturgical prayers demonstrates that those who die unbaptized die with the possibility of entering heaven through means known only to God.
This grace is all the more pertinent on the Feast of the Holy Innocents. Their parents knew nothing of Christ, they knew nothing of Christ, they knew nothing about dying for Christ, yet they DID die for Christ as mute witnesses to the goodness of God. And who can look on an infant without seeing the goodness and glory of the life-giving God? Every infant is a testimony to the Life-giving God Who is infinitely Good.
We cannot know with certitude that these children enter heaven, but we can hope with divine hope that they do, and the Church requires us, through Her liturgy, to maintain in prayer this divinely-founded hope.
The FSSP teaching on this is quite, quite wrong.
Pray for every FSSP priest, that they may one day begin to teach what the Church teaches on this point.