Buenos días,
"By the end of the afternoon" in English means "at the point at which the afternoon ended", and would normally be followed by "we *had* eaten..." i.e. looking back at what *had* already happened before that point, not "we ate...". It looks as though this has been translated as "Hacía el final de la tarde" or "Al final de la tarde" which would backtranslate as "Towards the end of the afternoon" or "At the end of the afternoon, we ate ..." i.e. looking back at what actually happened during the later part of the afternoon -- which makes more sense. I suggest tweaking the English to "Towards the end" or "At the end" to avoid this confusion.
¡Muchas gracias!