KDE PIM/MS Windows/SQLite Folder Indices/merge

Validation of the SQLite folder indices merge into kdepim trunk (805075), to see whether the mmap mode is affected.

jstaniek 11:38, 8 May 2008 (CEST)

Notes

• "OK" means files or methods inspected in every detail.
• We do not mention changes that only add/remove kDebug(), etc.

Results

• compactionjob.cpp - OK
  • void MboxCompactionJob::done( int rc ): the only addition: result of KDE_rename() is checked, return immediately on failure - OK

• folderstorage.cpp, folderstorage.h - OK
  • QString FolderStorage::indexLocation() const - implemented here to return "*.index" name for mmap mode ("*.index.db" for sqlite mode). Previously it was abstract and implemented in KMFolderIndex. KMFolderSearch reimplements it to return "*.index.search" name - see kmfoldersearch.cpp for details.
  • added virtual QString FolderStorage::location(const QString& suffix) const helper, used in various *location() methods of in KMail, to avoid code duplication; returns full path to .index, .index.ids or .index.sorted file (depending on suffix); the former path (in case of mmap-based implementation) is returned when the suffix is empty.
  • QString FolderStorage::sortedLocation() const - added to avoid constructing paths using indexLocation() + ".sorted" code
  • QString FolderStorage::idsLocation() const - added to avoid constructing paths using indexLocation() + ".ids" code
  • int FolderStorage::rename( const QString &newName, KMFolderDir *newParent ):
    • sortedLocation() is used instead of adding indexLocation()+".sorted" - OK
    • result of KDE_rename( oldIndexLoc, newIndexLoc ) is checked and we return immediately on failure - OK
    • result of KDE_rename( oldSortedLoc, newSortedLoc ) is checked and we return immediately on failure - OK
    • result of KDE_rename( oldIdsLoc, newIdsLoc ) is checked and we return immediately on failure - OK
  • void FolderStorage::remove() - sortedLocation() is used instead of adding indexLocation()+".sorted" - OK
  • void FolderStorage::invalidateFolder():
    • sortedLocation() is used instead of adding indexLocation()+".sorted" - OK
    • idsLocation() is used instead of adding indexLocation()+".ids" - OK
  • int FolderStorage::writeFolderIdsFile() const, int FolderStorage::touchFolderIdsFile(), int FolderStorage::appendToFolderIdsFile( int idx ) all return 0 (success) instead of -1 immediately if mExportsSernums == 0 - OK

• kmailicalifaceimpl.cpp - OK

void KMailICalIfaceImpl::readConfig() - added sanity check: if ( !mCalendar || !mTasks || !mJournals || !mContacts || !mNotes )

 return;

KMFolder* KMailICalIfaceImpl::initFolder( KMail::FolderContentsType contentsType ) - added check for result of open():if ( 0 != folder->open( "ifacefolder" ) ) { ....

• kmfolder.cpp, kmfolder.h - OK
  • QString KMFolder::indexLocation() const - made virtual but implementation is unaffected.
  • QString KMFolder::sortedLocation() const - calls FolderStorage::idsLocation() if storage is presents, else returns empty string; added as a helper for KMHeaders so we don't need to call mFolder->indexLocation() + ".sorted"; gives effectively the same result - OK
  • QString KMFolder::idsLocation() const - like above but for ".ids" names - OK

• kmfolderdir.cpp - OK
  • bool KMFolderDir::reload(void): no changes, as .index.db files are only ignored for sqlite mode - OK

• kmfolderindex.cpp, kmfolderindex.h
  • uchar *indexStreamBasePtr(), size_t indexStreamLength() const, indexSwapByteOrder(), indexSizeOfLong(), bool updateIndexStreamPtr(bool just_close=false) methods and FILE* mIndexStream, off_t mHeaderOffset, uchar *mIndexStreamPtr, size_t mIndexStreamPtrLength, bool mIndexSwapByteOrder, int mIndexSizeOfLong members are unused in sqlite mode - mmap mode is unaffected - OK
  • bool openDatabase( int mode ), int writeMessages( KMMsgBase* msg, WriteMessagesMode mode ) and sqlite3 *mIndexDb member - unused in mmap mode - OK
  • implementation of virtual QString indexLocation() const is moved to FolderStorage (superclass), implementation itself remains unchanged - OK
  • added "kmfolderindex_common.cpp", included from "kmfolderindex.cpp" and "kmfolderindex_sqlite.cpp" - contains common includes and defines for these two files, and the following shared methods:
    • int openInternal()
    • int createInternal()
  • int KMFolderIndex::updateIndex() - minor optimization - OK
  • int KMFolderIndex::writeIndex( bool createEmptyIndex ):
    • returns on unlink( QFile::encodeName( tempName ) ) failure

• • int writeMessages( KMMsgBase* msg, bool flush = true ) added

• kmfoldermaildir.cpp

• kmfoldermbox.cpp

• kmfoldersearch.cpp - OK
  • QString KMFolderSearch::indexLocation() const now just calls FolderStorage::location( "search" ) utility function what has the same effect
  • int KMFolderSearch::writeIndex( bool ): at the very end final, result of KDE_rename() from temp name to the indexLocation() name is checked; on faulure the method fails, what cause that "mDirty = false; mUnlinked = false;" code is not executed. - OK

• kmheaders.cpp, kmheaders.h - OK
  • #define KMAIL_SORT_FILE(x) x->indexLocation() + ".sorted" replaced by #define KMAIL_SORT_FILE(x) x->sortedLocation() - uses a new utility FolderStorage::sortedLocation() method having basically the same source code - OK
  • void KMHeaders::setNestedOverride( bool override ) - like above - OK
  • void KMHeaders::setSubjectThreading( bool aSubjThreading ) - like above - OK
  • void KMHeaders::writeFolderConfig (void): minor optimization: mFolder->getMsgBase( current->msgId() ) is called once instead of twice - OK
  • bool KMHeaders::writeSortOrder() - the only addition: result of KDE_rename() is checked, return immediately on failure
  • QList< Q_UINT32 > KMHeaders::selectedSernums() - sanity check added in case when KMMsgBase *msgBase = mFolder->getMsgBase( item->msgId() ); returns null result; this shouldn't happen - OK
  • QList< Q_UINT32 > KMHeaders::selectedVisibleSernums() - like above

• kmkernel.cpp - OK
  • void KMKernel::slotRunBackgroundTasks(): added sanity checks for null pointers: the_folderMgr, the_imapFolderMgr, the_dimapFolderMgr; this is neutral change

• kmmsgbase.cpp, kmmsgbase.h - OK
  • added ctor
    • KMMsgBase(KMFolder* aParentFolder, char* data, short len, sqlite_int64 id) for sqlite mode
    • KMMsgBase(KMFolder* aParentFolder, off_t off, short len) for mmap mode
    • In both cases the ctor is effectively the same as KMMsgBase(KMFolder* aParentFolder), but also initializes KMMsgBase members - OK
  • KMMsgBase::~KMMsgBase() - frees mData for sqlite mode - no changes for mmap mode - OK
  • void KMMsgBase::assign(const KMMsgBase* other) - for sqlite makes a deep copy of mData, copies mDbId value; no changes for mmap mode - OK
  • setData(char* data), setDbId(sqlite_int64) and sqlite_int64 dbId() const, and sqlite_int64 mDbId member: added only for sqlite mode, won't affect mmap mode - OK
  • off_t mIndexOffset and bool syncIndexString() const unused (ifdef'd) in sqlite mode - OK
    • syncIndexString(): added check: returns false on fwrite( buffer, len, 1, storage()->mIndexStream) failure - OK
  • QString KMMsgBase::getStringPart(MsgPartType t) const:
    • QString ret is moved up above 'retry:' label, nothing is affected because after ret modified (set to not empty), there is no path of return to the 'retry:' label - OK
    • plain rename from quint16 l to quint16 len no changes - OK
    • g_chunk_offset = 0 moved down to an initializer of the for loop, nothing is affected because g_chunk_offset is not used before the loop
    • the while(g_chunk_offset < mIndexLength) loop is replaced by for ( g_chunk_offset = 0; g_chunk_offset < mIndexLength; g_chunk_offset += len ):
      • g_chunk_offset = 0 precondition is unaffected - OK
      • the test expression g_chunk_offset < mIndexLength is unchanged - OK
      • the count command g_chunk_offset += len originally appeared at the end of while loop, this is equivalent as there are no continue command in the loop - OK
  • off_t KMMsgBase::getLongPart(MsgPartType t) const: changes are is similar to those in QString KMMsgBase::getStringPart(MsgPartType t) const:
    • the while loop is replaced by for - OK
    • quint16 len moved outside of the loop, nothing is affected by this, as the value is set at the beginning of each step of the loop - OK

• kmmsgdict.cpp - OK
  • QString KMMsgDict::getFolderIdsLocation( const FolderStorage &storage ): the new FolderStorage::idsLocation() is called which provides the same code as the original impl. of KMMsgDict::getFolderIdsLocation(). - OK

• kmmsginfo.cpp, kmmsginfo.h - OK
  • sqlite mode has different ctor: explicit KMMsgInfo(KMFolder* p, char* data=0, short len=0, sqlite_int64 dbId=0) - mmap mode is unaffected - OK
  • init(): mData is initialized in sqlite mode - mmap mode is unaffected - OK

• messageproperty.cpp - OK
  • minor optimizations only added, for example 1. instead of 2.:
    1. QMap<quint32, QPointer<KMFolder> >::ConstIterator it = sFolders.constFind( serNum );
       return it == sFolders.constEnd() ? 0 : (*it).operator->(); - OK
    2. if (sFolders.contains(serNum))
         return sFolders[serNum].operator->();
       return 0; - OK
  • Affected methods:
    • KMFolder* MessageProperty::filterFolder( quint32 serNum ) - OK
    • ActionScheduler* MessageProperty::filterHandler( quint32 serNum ) - OK
    • bool MessageProperty::transferInProgress( quint32 serNum ) - OK
    • void MessageProperty::setTransferInProgress( quint32 serNum, bool transfer, bool force ) - OK
    • quint32 MessageProperty::serialCache( const KMMsgBase *msgBase ) - OK