The way the conversation has played out, though, suggests that it is not a well-posed question. "Justice" has two distinct, though analogous, meanings, and different people assume different meanings in answering the question.
The Catechism (naturally following St. Thomas, who naturally followed Publius Iuventius Celsus) defines the virtue of justice as "the moral virtue that consists in the constant and firm will to give their due to God and neighbor." But we also use the word "justice" analogously, to refer to the object of the virtue, to what is objectively due to God and neighbor. (St. Thomas uses the word ius to refer to the object of justice, but "just" has basically the same dual meaning as "justice." The English Dominican translation of the ST uses "right" for "ius," which probably wouldn't be a helpful substitute.)
So I think the question might be rephrased and generalized into something like this: In what sense, if any, can it be said that a man acts in accordance with the virtue of justice when he mistakenly fails to give to another his due?