Where is the Nautilus/file sorting algorithm defined?

When I look at files as listed in Nautilus (sort by name), if there are both English and non-ASCII (in my case Japanese) characters, some characters (kana) are mixed with the ASCII characters out of order. (Windows segregates them properly.)

I have checked many related questions here, and it seems like the sort order is hard-coded in Nautilus, and so can't be changed.

But my question here is not how to change it, but where the sort algorithm is actually defined, relating to non-ASCII characters. I'm using 18.04 if that matters. My locale is US-English.

Terminal ls command seems to give the same order, so this may be a system setting rather than specific to Nautilus, but again I don't know where to look for it.

2 Answers

Prerequisites: First check that the locale of your language (in your case, the Japanese locale) is installed in your system, using the command:

$ grep -v '^#' /etc/locale.gen

If the Japanese locale (ja_JP.UTF8) is not listed, you have to add it to your system using:

$ sudo dpkg-reconfigure locales

Generally, sorting order is a system setting and is related to your current locale. You mentioned that you are using the "US-English" locale. I assume it is en_US.UTF-8 and probably this is the reason you see file names sorted in US sort order.

First, test from command line the following. Go to the directory containing your English and Japanese named files and enter the command:

$ LC_ALL=ja_JP.UTF8 ls

If this command displays the files in the order you want, then you may try Nautilus also with this command:

$ LC_ALL=ja_JP.UTF8 nautilus .

If you get an error message like:

... Locale not supported by C library. Using the fallback 'C' locale.

check once more:

$ grep -v '^#' /etc/locale.gen

to see which locales are enabled in your system. If the Japanese locale is not enabled, you have to add it to your system using:

$ sudo dpkg-reconfigure locales

Note: In the above commands (ls and nautilus) I have prefixed with an environment variable that with affect only that command invocation. If you want to change your locale permanently, you will have to change the "default locale" also with the sudo dpkg-reconfigure locales command.

The Nautilus/File sorting algorithm seems to be defined in GLib:

GLib is the low-level core library that forms the basis for projects such as GTK and GNOME.

The official download locations are:

And more specifically in gunicollate.c and it's g_utf8_collate_key_for_filename() function.

It is discussed in various issues like

"g_utf8_collate_key_for_filename() corner cases with digits"

or this old bug on Launchpad:

"Sorting by name doesn't work well with digits or special chars"

where comment #10 suggest using this hack to solve the sorting problem :