User:Mark Gaiser/KDirchainRebuild

Introduction

This all started a few years ago when i was profiling Dolphin. I wasn't happy with it's general performance and was even less happy when it came to massive folders. Added to that was the - back then - brand new stuff called "QML". So i wanted to create a file browser that is fast, memory efficient and with a GUI done in QML so that i could add those nifty animations. So i set out to restructure all of the directory handling from KIO from as low UDSEntry till the user facing models. I named that "KDirchainRebuild" since it was rebuilding the chain of events when you open a directory.

That's how it all started a few years ago. Fast forward to today and you see an entirely different landscape. Dolphin is now very fast even on large folders. I didn't make any patches for it, but my initial research in why it was performing poorly was apparently a spark for the Dolphin devs to take a good close look at it and improve it greatly. The biggest thing i did here was investigating how entries where being batched in KIO. That is improved to do real batching which in turn allows an application to show data more quickly to the user. A big thank you here for David Faure since the new batching triggered another weird socket issue that he solved.

So is there still a reason to even consider this "KDirchainRebuild" over the existing classes? Well, yes. The new classes are aimed at simplicity, small memory footprint and being as fast as possible (which sometimes conflicts with simplicity), and - via models - being usable in QML.

Why new classes instead of improving the existing ones?

Initially the idea was to take the current classes, remove all function content and re-implement them in my view. Later i abandoned that idea because there seemed to be quite a few functions that i wanted to handle in a different way along with removing methods altogether. I wasn't really interested in marking a dozen functions as "deprecated". Specially not as long as this project was highly experimental and just for my own pet project. So the choice was easiest to just go for my own class names without the burden of having to re-implement the existing classes.

The Classes

Before i present the new classes and the API they have, i first want to take a moment to say that the classes are not done yet! The API is far from finished and code-wise it already can use a bit of cleanup. All the information below is just to indicate it's current state, what is still planned for it and where i see potential issues (and resolutions).

KDirectoryEntry

KDirectoryEntry compares mostly to KFileItem only with a greatly reduced memory footprint (and a lot of missing functions).

Data stored

Inherits from no other classes.

• 1 d pointer (KDirectoryEntryPrivate)
• 1 KIO::UDSEntry
• 1 bool to know if details are loaded (details 0 vs 2). I still like to get rid of this one.

Ideas

You always need an UDSEntry if you want to know something about a file. It might be beneficial for the UDSEntry itself to know (with a bool) if it has all the file details or not. It adds a bool value, but that can be used later on to prevent needless hash lookups if you have no details loaded. I might be worth adding this or at least experiment with it.

For the new classes. I just like to have details with true or false (bool) instead of a string (0 or 2). Easy to implement on my side, but will it cause any other difficulties later on? I don't know.

Functions

Some of the below documentation is a direct copy from KFileItem.

   /**
    * Returns the raw file name with extension as it comes from UDSEntry.
    * @return QString
    */
   const QString name() const;

   /**
    * Returns the owner of the file.
    * @return QString
    */
   const QString user() const;

   /**
    * Returns the group of the file.
    * @return QString
    */
   const QString group() const;

   /**
    * Returns the base name of a file. This is the name without extension.
    * @return QString
    */
   const QString basename() const;

   /**
    * Returns the extension of a file without leading dot.
    * @return QString
    */
   const QString extension() const;

   /**
    * Returns the icon name based on the output of QMimeType
    * @return QString
    */
   const QString iconName() const;

   /**
    * Returns the icon comment based on the output of QMimeType
    * @return QString
    */
   const QString mimeComment() const;

   /**
    * Returns the file size from UDSEntry if details where loaded. 0 for no details or if the current entry is a folder.
    * @return QString
    */
   const KIO::filesize_t size() const;

   /**
    * This is used internally when constructing this object with an UDSEntry, or used externally to update this object. For example when a rename happens. Users using this class cannot use this object to get notified of changes. The KDirectory object containing this instance will notify you of changes.
    */
   virtual void setUDSEntry(const KIO::UDSEntry& entry, const QString& details = "0");

   /**
    * Returns true if this item represents a link in the UNIX sense of
    * a link.
    * @return true if the file is a link
    */
   bool isLink() const;

   /**
    * Returns true if this item represents a directory.
    * @return true if the item is a directory
    */
   bool isDir() const;

   /**
    * Returns true if this item represents a file (and not a a directory)
    * @return true if the item is a file
    */
   bool isFile() const;

   /**
    * Checks whether the file or directory is readable. In some cases
    * (remote files), we may return true even though it can't be read.
    * @return true if the file can be read - more precisely,
    *         false if we know for sure it can't
    */
   // STUB! To be implemented
   bool isReadable() const;

   /**
    * Checks whether the file or directory is writable. In some cases
    * (remote files), we may return true even though it can't be written to.
    * @return true if the file or directory can be written to - more precisely,
    *         false if we know for sure it can't
    */
   // STUB! To be implemented
   bool isWritable() const;

   // STUB! To be implemented
   bool isExecutable() const;

   // STUB! To be implemented
   bool isModified() const;

   // STUB! To be implemented
   bool isSystem() const;

   /**
    * Requests the modification, access or creation time, depending on @p which.
    * @param which the timestamp
    * @return the time asked for, QDateTime() if not available
    * @see FileTimes
    */
   QDateTime time(FileTimes which) const;

   /**
    * Checks whether the file is hidden.
    * @return true if the file is hidden.
    */
   bool isHidden() const;

   /**
    * Checks whether the file details are loaded.
    * @return true if details are loaded, false otherwise.
    */
   bool detailsLoaded() const;

KDirectory

You can't compare KDirectory to anything currently living in KDE (that i know of). It is somewhat inspired by QDir but even there not fully comparable.

KDirectory is basically a data management class. If you open a folder, this class will do a call to KIO::listDir and retrieve it's output. What you as a user can do is set properties how the output should be accessed. You can set filters and flags just like QDir. This class then applies those filters and flags to all incoming data and stores it (internally) in two lists. Once for those that pass, one for those that don't. No data is duplicated there. The number of items in both lists combined is the number of items you would see when you execute "ls -l | wc -l" in the same folder on the command line. This class only gives you access to the items that passed the filters/flags. If you want access to the items that didn't pass then you have to change your filters/flags to make them pass. Or set no filters/flags which makes all items pass. For bookkeeping, this class stores quite a bit of additional data.

Ideas

KDirectory should also manage file/folder changes in it's directory. However, the signals emitted by KDirWatch and QFileSystemWatcher are too generic. I need more detailed signals to let it be useful. Right now adding a file to a directlry simply means re-indexing the entire directory because we don't have enough details in our signals. Details that inotify (the backend) does provide. I plan to revive some old patches for QFileSystemWatcher on the frameworks sprint and make it work again. Once that is in working order, KDirectory can use them. Right now KDirectory just doesn't use any mechanism to monitor a folder for changes. Remember, work in progress!

Functions

   /**
    * Returns all entries that passed the filters and flags.
    * @return QVector<KDirectoryEntry>
    */
   virtual const QVector<KDirectoryEntry>& entries();

   /**
    * Entry returns the KDirectoryEntry object if it's index is in the filteredEntries.
    * @param index
    * @return KDirectoryEntry at index
    */
   virtual const KDirectoryEntry& entry(int index);

   /**
    * String of the full path for this directory.
    * @return QString
    */
   virtual const QString& url();

   /**
    * Number of entries that passed the filters and flags
    * @return int
    */
   virtual int count();

   /**
    * Set the details for the current directory. It can be 0 (no details) or 2 (full details). If the value passed is different then the current details value (and valid) then the directory will be re-scanned according to the new details value.
    * @param QString details. Either 0 or 2.
    * @return KDirectoryEntry at index
    */
   virtual void setDetails(const QString& details);

   // For those, see QDir documentation.
   QDir::Filters filter();
   void setFilter(QDir::Filters filters);
   QDir::SortFlags sorting();
   void setSorting(QDir::SortFlags sort);

   /**
    * Loads the entry details and passes it to the KDirectoryEntry that needs the information.
    * Be aware that this function is executing a (slow) stat call!
    * @param int id the id of the file to load. This is the QVector index id.
    */
   void loadEntryDetails(int id);

Signals

   /**
    * New entries in this folder have been processed.
    * @param KDirectory* directory pointer to the current directory. This pointer is given
    *        because you're likely to use multiple KDirectory objects so you wouldn't easily
    *        know which KDirectory object spawned this signal.
    */
   void entriesProcessed(KDirectory* dir);

   /**
    * Done loading entries.
    * @param KDirectory* directory pointer to the current directory. This pointer is given
    *        because you're likely to use multiple KDirectory objects so you wouldn't easily
    *        know which KDirectory object spawned this signal.
    */
   void completed(KDirectory* dir);

   /**
    * Done loading entries.
    * @param KDirectory* directory pointer to the current directory. This pointer is given
    *        because you're likely to use multiple KDirectory objects so you wouldn't easily
    *        know which KDirectory object spawned this signal.
    * @param int id is the id that you can use to get the KDirectoryEntry object (entry(id)).
    */
   void entryDetailsChanged(KDirectory* dir, int id);