[TurboGears] UTF-8 locale encoding: recommended approach?

[Thomas De Schampheleire]

Hello,

In Kallithea we sometimes have users reporting problems that are
caused by them not starting the application from an UTF-8 environment.
This can normally be solved by setting 'LANG' and 'LC_ALL' to a UTF-8
value but this can be done in several places.

Please see https://bitbucket.org/conservancy/kallithea/issues/310/accentuated-characters-improperly-rendered
as an example.

Here it is reported that the issue can be solved by changing Apache
config, and that Gearbox did not have the problem to begin with.
A suggestion from a Kallithea developer to change app_cfg.py to set
environment variables did not seem to have effect on the Apache
situation.

My question is now: what is the recommended way to handle this?
Does Turbogears2/Gearbox already does something automatically?
Any other feedback on this topic?

Thanks a lot,
Thomas

[Alessandro Molina]

TurboGears itself doesn't usually get in the way with encoding/decoding, I
guess some encoding/decoding is happening when reading outputs for HG
repositories.

The system encoding ( locale.getpreferredencoding ) frequently dictates
what should happen when crossing the boundary between bytes/strings, as by
default apache (when started from init scripts) sets locale to "C" ( as
documented at
http://modwsgi.readthedocs.io/en/develop/configuration-directives/WSGIDaemonProcess.html
) my guess is that strings read from hg might be encoded/decoded with that
locale resulting in some characters being impossible to decode.
I guess that the only "special" thing that gearbox is doing is just not
messing with the locale.

There was an article about this issue from Graham Dumpleton himself which I
guess can be helpful ->
http://blog.dscpl.com.au/2014/09/setting-lang-and-lcall-when-using.html

If my guess that the improper decoding/encoding is happening at "hg layer"
I'm not sure it's something where TG can intervene.

On Sat, Mar 17, 2018 at 10:52 PM, Thomas De Schampheleire <

Post by Thomas De Schampheleire
Hello,
In Kallithea we sometimes have users reporting problems that are
caused by them not starting the application from an UTF-8 environment.
This can normally be solved by setting 'LANG' and 'LC_ALL' to a UTF-8
value but this can be done in several places.
Please see https://bitbucket.org/conservancy/kallithea/issues/
310/accentuated-characters-improperly-rendered
as an example.
Here it is reported that the issue can be solved by changing Apache
config, and that Gearbox did not have the problem to begin with.
A suggestion from a Kallithea developer to change app_cfg.py to set
environment variables did not seem to have effect on the Apache
situation.
My question is now: what is the recommended way to handle this?
Does Turbogears2/Gearbox already does something automatically?
Any other feedback on this topic?
Thanks a lot,
Thomas
--
You received this message because you are subscribed to the Google Groups
"TurboGears" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/turbogears.
For more options, visit https://groups.google.com/d/optout.