[Thunar-dev] Thumblerd and large files collections

[Grigory Tuboltsev]

Hi,

Thank you all devs for developing Thunar. It is small, lightweight and
easy to use.

I've got some problems with Thunar trying to work with very large photo
or PDF collections though.

I've got problems starting with 30 PDF files in a directory, not to
speak about Really Large Collections: 3000 pdf files, 2000 raw photos,
5000 (15Mbite large) jpegs and so on. Sometimes I even have to work with
photo or PDF collections consisting of 30 000 + files.

I have "show thumbnails" setting switched on by default for everyday
use. Every time i try to browse a directory full of photo or PDFs or
video files Thunar (thumblerd) starts thumbnailing them. It takes about
60-90 of CPU time and probably the same amount of disk I/O capacity so
So my PC gets unresponsive, slow etc.

So, i see following problems:

1) There is NO way to quickly switch thumbnailing off and on again. No
button on the toolbar, no hotkey, NOTHING. I need to open settings
window every time, than click on checkbox, than OK. FOUR clicks just to
switch thumbnailing off or on.

2) Even if i make all this clicks (after navigating to a directory with
a large file collection) thumblerd does not stop working. It just
continues to use 90% of CPU time. So I need to kill thumblerd manually
every time i make this navigation error.

3) Thumblerd does not have limits (or GUI-way to change them) like "Does
not start thumbnailing in folders with over 150 objects" or "does not
make thumbnails for files over 3Mb large" and so on.

4) (more a FR than a problem) There is no way to tell thumblerd make
thumbnails just for selected files. This would be a killer feature :)

Working with 3000 or 5000 jpeg files is a usual everyday usecase for all
us photographers or photo editors. Thousands of PDF files are often used
by ebook-readers users and so on. So Thunar (or thumblerd) should be
ready for such usecases.

Thanks for reading and sorry for my broken English :)

With best regards,
Grigory Tuboltsev

[Grigory Tuboltsev]

Should i file a bug or FR on "thumblerd" component ?

Post by Grigory Tuboltsev
I've got some problems with Thunar trying to work with very large photo
or PDF collections though.

[Jannis Pohlmann]

Hey,

On Fri, 19 Nov 2010 19:26:25 +0300

Post by Grigory Tuboltsev
Hi,
Thank you all devs for developing Thunar. It is small, lightweight
and easy to use.
I've got some problems with Thunar trying to work with very large
photo or PDF collections though.
I've got problems starting with 30 PDF files in a directory, not to
speak about Really Large Collections: 3000 pdf files, 2000 raw
photos, 5000 (15Mbite large) jpegs and so on. Sometimes I even have
to work with photo or PDF collections consisting of 30 000 + files.
I have "show thumbnails" setting switched on by default for everyday
use. Every time i try to browse a directory full of photo or PDFs or
video files Thunar (thumblerd) starts thumbnailing them. It takes
about 60-90 of CPU time and probably the same amount of disk I/O
capacity so So my PC gets unresponsive, slow etc.

One note about that: Thunar sends thumbnail requests only for those
files that are currently visible or were visible while you scrolled
up/down in a folder. So not for all files thumbnails are created.

Post by Grigory Tuboltsev
1) There is NO way to quickly switch thumbnailing off and on again.
No button on the toolbar, no hotkey, NOTHING. I need to open settings
window every time, than click on checkbox, than OK. FOUR clicks just
to switch thumbnailing off or on.

Creating thumbnails is a standard feature that should to be configured
very often. That's why there is no simple way to switch it on and off.

BTW, this is in no way different to previous versions of Thunar. There
Thunar would have taken up to 60-90% of the CPU time. The UI should not
become less responsive as most of the stuff is performed in ways that
don't block the UI.

Post by Grigory Tuboltsev
2) Even if i make all this clicks (after navigating to a directory
with a large file collection) thumblerd does not stop working. It
just continues to use 90% of CPU time. So I need to kill thumblerd
manually every time i make this navigation error.

This is an issue inside Thunar. At the moment, thumbnail requests
already sent to tumbler are not canceled when turning off thumbnails
or when switching folders. But no new requests should be sent (if
everything works correctly). This definitely can be improved but it's
not easy.

Post by Grigory Tuboltsev
3) Thumblerd does not have limits (or GUI-way to change them) like
"Does not start thumbnailing in folders with over 150 objects" or
"does not make thumbnails for files over 3Mb large" and so on.

I agree, a file size limit option would be a good idea. But it does
not make sense for all file types. A large PNG is different from a
large PDF where only the first page has to be loaded for the thumbnail.

Post by Grigory Tuboltsev
4) (more a FR than a problem) There is no way to tell thumblerd make
thumbnails just for selected files. This would be a killer feature :)

Tumbler has no idea what "selected files" are. That's entirely handled
in Thunar. I'm sorry to say this a feature request I'd reject.
Again, creating thumbnails for all files is a standard feature in
modern file managers.

Post by Grigory Tuboltsev
Working with 3000 or 5000 jpeg files is a usual everyday usecase for
all us photographers or photo editors. Thousands of PDF files are
often used by ebook-readers users and so on. So Thunar (or thumblerd)
should be ready for such usecases.

We can speed things up a little bit by validating existing thumbnails
to avoid requesting thumbnails through tumbler. But the initial
generation will take some time, that's kinda normal. If I zoom out so
that I see 100+ files at once my UI will slow down no matter if
thumbnails are created in the background or not.

- Jannis
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://mail.xfce.org/pipermail/thunar-dev/attachments/20101119/21bd8901/attachment.pgp>

[Grigory Tuboltsev]

Post by Jannis Pohlmann
One note about that: Thunar sends thumbnail requests only for those
files that are currently visible or were visible while you scrolled
up/down in a folder. So not for all files thumbnails are created.

I alsways use a list view to work with collections. So, opening a
directory with pdf files makes over 100 of them visible. More than
enough to make thumbler eat 90% of my CPU and i/o.

Post by Jannis Pohlmann

Post by Grigory Tuboltsev
1) There is NO way to quickly switch thumbnailing off and on again.
No button on the toolbar, no hotkey, NOTHING. I need to open settings
window every time, than click on checkbox, than OK. FOUR clicks just
to switch thumbnailing off or on.

Creating thumbnails is a standard feature that should to be configured
very often. That's why there is no simple way to switch it on and off.

Thats what i am trying to explain: there SHOULD be a way to quickly
configure it. Otherwise one cannot work with really large file archives.
Is it SO difficuilt to make a simple hotkey to switch thumbnailing on
and off ? Sorry i am no progammer so i can not judge if it's an easy
task or not.

Post by Jannis Pohlmann
BTW, this is in no way different to previous versions of Thunar. There
Thunar would have taken up to 60-90% of the CPU time. The UI should not
become less responsive as most of the stuff is performed in ways that
don't block the UI.

It does become slow and\or unresponsive. Not sure if its has to do with
CPU or disk i/o.

Post by Jannis Pohlmann
This is an issue inside Thunar. At the moment, thumbnail requests
already sent to tumbler are not canceled when turning off thumbnails
or when switching folders. But no new requests should be sent (if
everything works correctly). This definitely can be improved but it's
not easy.

That would be a greate feature.

Post by Jannis Pohlmann
I agree, a file size limit option would be a good idea. But it does
not make sense for all file types. A large PNG is different from a
large PDF where only the first page has to be loaded for the thumbnail.

He-he :) You have probably never seen Really Large PDF Files. All pdf
files i am working with are scanned books with hundreds of
high-resolution TIFFs inside. First pages in such PDF files are 5-8 Mb
TIFFs :)

30 000 of small jpeg files make Thunar (+thumblerd) just as slow as 100
of large PDFs. I think that thumblerd gets so much requests because I
always scroll in such folders. So, when i open the directory, thumblerd
gets first 300 requests (list view!), than i start to scroll and it gets
other 15 000 requestst and eats 90 % of my CPU.

Post by Jannis Pohlmann

Post by Grigory Tuboltsev
4) (more a FR than a problem) There is no way to tell thumblerd make
thumbnails just for selected files. This would be a killer feature :)

Tumbler has no idea what "selected files" are. That's entirely handled
in Thunar. I'm sorry to say this a feature request I'd reject.

Well i thought thunar sends thumbnailing requests to thumbnailerd. So it
could send this requests just for selected files, not for all files in
the directory (or just for selected files after pressing some hotkey).

Post by Jannis Pohlmann
Again, creating thumbnails for all files is a standard feature in
modern file managers.

Yes, eating 90 % of CPU and i/o badwidth seem to be a feature of modern
file managers. I just thought Thunar is kinda lightweight file manager
and shouldn't have this feature. No offence, just asking...

Post by Jannis Pohlmann
We can speed things up a little bit by validating existing thumbnails
to avoid requesting thumbnails through tumbler. But the initial
generation will take some time, that's kinda normal. If I zoom out so
that I see 100+ files at once my UI will slow down no matter if
thumbnails are created in the background or not.

Viewing 100+ files at once does not slow UI down if thumbnailing is
switched off.



Anyway, thank you for your answers.


Grigory

[Erlend Davidson]

Post by Grigory Tuboltsev

Post by Jannis Pohlmann
One note about that: Thunar sends thumbnail requests only for those
files that are currently visible or were visible while you scrolled
up/down in a folder. So not for all files thumbnails are created.

I alsways use a list view to work with collections. So, opening a
directory with pdf files makes over 100 of them visible. More than
enough to make thumbler eat 90% of my CPU and i/o.

I think Thunar only generates one thumbnail at a time, in sequence. So
just because you're viewing 100 PDFs does not mean 100 thumbnails are
generated at the same time so the system load should be the same as for
one PDF (but should last for a longer time). Are you working on a
single-core computer?

Erlend

[TW]

Post by Jannis Pohlmann
I'm sorry to say this a feature request I'd reject.
Again, creating thumbnails for all files is a standard feature in
modern file managers.

Sorry to bring up that old topic again (and kind of hijack the
thread): Tabs are a standard feature in modern file managers as well.
Nautilus has them, PCMan FM, Dolphin, Konqueror, Krusader, Windows
Explorer, Cubic Explorer, Proto, Finder (can at least have tabs via
plugin), Path Finder etc. Only file managers that have panes
sometimes don't have tabs (many of them still do). The only exception
I'm aware of is ROX-Filer which neither has tabs nor panes. It's been
requested several times, and I still don't quite understand why they
won't be in Thunar.

Anyway, keep up the good work!

Thomas W.

[Nick Schermer]

Post by TW

Post by Jannis Pohlmann
I'm sorry to say this a feature request I'd reject.
Again, creating thumbnails for all files is a standard feature in
modern file managers.

Sorry to bring up that old topic again (and kind of hijack the
thread): ?Tabs are a standard feature in modern file managers as well.
?Nautilus has them, PCMan FM, Dolphin, Konqueror, Krusader, Windows
Explorer, Cubic Explorer, Proto, Finder (can at least have tabs via
plugin), Path Finder etc. ?Only file managers that have panes
sometimes don't have tabs (many of them still do). ?The only exception
I'm aware of is ROX-Filer which neither has tabs nor panes. ?It's been
requested several times, and I still don't quite understand why they
won't be in Thunar.

Design philosophy; and enough alternatives *with* tabs.

Nick.

[TW]

Post by Nick Schermer

Post by TW

Post by Jannis Pohlmann
I'm sorry to say this a feature request I'd reject.
Again, creating thumbnails for all files is a standard feature in
modern file managers.

Sorry to bring up that old topic again (and kind of hijack the
thread): ?Tabs are a standard feature in modern file managers as well.
?Nautilus has them, PCMan FM, Dolphin, Konqueror, Krusader, Windows
Explorer, Cubic Explorer, Proto, Finder (can at least have tabs via
plugin), Path Finder etc. ?Only file managers that have panes
sometimes don't have tabs (many of them still do). ?The only exception
I'm aware of is ROX-Filer which neither has tabs nor panes. ?It's been
requested several times, and I still don't quite understand why they
won't be in Thunar.

Design philosophy; and enough alternatives *with* tabs.

I'm not really complaining. I've tested many alternatives and found
Thunar is still the best choice for me, even without tabs (thanks for
that!). I just couldn't resist to put in my two cents when a Thunar
developer "defends" a feature because it's a standard in modern file
managers, while it appears to me that another feature for which this
attribute is applicable and which has been requested by a number of
users is consistently rejected. I think the argumentation was not to
add unnecessary complexity to the interface. For me, it adds
unnecessary complexity having to switch between a number of Thunar
windows, and tabs have been becoming so familiar for the user (from
web browsers and maybe other file mangers) that I think they're
perceived as something natural, not something complex.

(I hope *I'm* not perceived as offensive.)

That said, I'm respecting that you have to set priorities, make
consistent design choices and manage limited time resources.

A happy Thunar user (who would nevertheless be even happier with those tabs)