Add Checksum Verification and Directory Upload Support for QFieldCloud #6114
+123
−20
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
🎉 Ta-daaa, freshly created APKs are available for 9b3ef68: arm64-android |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
nirvn
reviewed
Mar 19, 2025
harmonize with every variable name around.
mohsenD98
changed the title
nirvn
reviewed
Mar 26, 2025
| @@ -150,23 +152,107 @@ const QMultiMap<QString, QString> QFieldCloudUtils::getPendingAttachments() | |||
| return std::move( files ); | |||
| } | |||
|
|
|||
| void QFieldCloudUtils::addPendingAttachments( const QString &projectId, const QStringList &fileNames ) | |||
| void QFieldCloudUtils::addPendingAttachments( const QString &projectId, const QStringList &fileNames, QFieldCloudConnection *cloudConnection, const bool &checkSumCheck ) | |||
| { | |||
| QLockFile attachmentsLock( QStringLiteral( "%1/attachments.lock" ).arg( QFieldCloudUtils::localCloudDirectory() ) ); | |||
There was a problem hiding this comment.
Move the lock file into writeToAttachmentsFile.
| //! Adds an array of \a fileNames for a given \a projectId to the pending attachments list | ||
| Q_INVOKABLE static void addPendingAttachments( const QString &projectId, const QStringList &fileNames ); | ||
| /** | ||
| * Adds an array of \a fileNames for a given \a projectId to the pending attachments list. |
There was a problem hiding this comment.
Suggested change
| * Adds an array of \a fileNames for a given \a projectId to the pending attachments list. | |
| * Adds an array of files and/or folders for a given cloud project to the pending upload attachments list. |
| * If \a checkSumCheck is true, checks file checksums with the server; otherwise, adds all files without validation. | ||
| * | ||
| * @param projectId The project ID for which files are added. | ||
| * @param fileNames The list of file names to be added. |
There was a problem hiding this comment.
Suggested change
| * @param fileNames The list of file names to be added. | |
| * @param fileNames The list of file and/or folder path(s) to be added. |
| QFileInfo fi( attachmentsFile ); | ||
| if ( checkSumCheck ) | ||
| { | ||
| QVariantMap params; |
There was a problem hiding this comment.
I'd do a null check for the cloud connection here, just in case.
| * @param cloudConnection The cloud connection used to fetch file data. | ||
| * @param checkSumCheck Whether to validate files by comparing checksums with the server. | ||
| */ | ||
| Q_INVOKABLE static void addPendingAttachments( const QString &projectId, const QStringList &fileNames, QFieldCloudConnection *cloudConnection, const bool &checkSumCheck ); |
There was a problem hiding this comment.
Suggested change
| Q_INVOKABLE static void addPendingAttachments( const QString &projectId, const QStringList &fileNames, QFieldCloudConnection *cloudConnection, const bool &checkSumCheck ); | |
| Q_INVOKABLE static void addPendingAttachments( const QString &projectId, const QStringList &fileNames, QFieldCloudConnection *cloudConnection = nullptr, const bool &checkSumCheck = false ); |
| // ? should we also check the checksums of the files being uploaded? they are available at deltaFile->attachmentFileNames()->values() | ||
| QFieldCloudUtils::addPendingAttachments( project->id, { absoluteFilePath } ); | ||
| // ? should we also check the checksums of the files being uploaded? they are available at deltaFile->attachmentFileNames()->values() || Or just send true instead of false! | ||
| QFieldCloudUtils::addPendingAttachments( project->id, { absoluteFilePath }, mCloudConnection, false ); |
There was a problem hiding this comment.
Suggested change
| QFieldCloudUtils::addPendingAttachments( project->id, { absoluteFilePath }, mCloudConnection, false ); | |
| QFieldCloudUtils::addPendingAttachments( project->id, { absoluteFilePath } ); |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull Request Description
Overview:
This pull request introduces several improvements to the file upload functionality, specifically for enhancing efficiency and extending support for pushing entire directories to QFieldCloud. The changes introduced include checksum verification to ensure that unchanged files are not uploaded again, reducing unnecessary data transfer, as well as enabling directory uploads.
Changes:
Checksum Verification for File Uploads:
The file upload process has been optimized by introducing checksum verification. This ensures that files that have not been modified are not uploaded again, leading to significant efficiency improvements and reduced data transfer. The checksum verification mechanism is implemented using ETags for file comparison.
Support for Uploading Entire Directories:
A new functionality has been added to allow uploading entire directories to QFieldCloud. The new code recursively scans a directory and processes its files, ensuring that all files within the directory are correctly handled and uploaded if necessary.
Refactor of the
addPendingAttachmentsFunction:The existing
addPendingAttachmentsfunction has been refactored and renamed towriteToAttachmentsFile. The new function allows both individual files and directories to be handled, and integrates the checksum verification logic.New Helper Functions:
writeFilesFromDirectory: A recursive function to process all files within a directory and push them to QFieldCloud.writeFileDetails: This function writes the file details to the attachments file and checks the checksum for file comparison before uploading.New QML Property for Detecting Project Folder:
A new QML property
itemWithinQFieldCloudProjectFolderhas been added to check if the item is within the QFieldCloud project folder. This property ensures that only items within the project folder are considered for upload.It is now used in the following conditional logic to determine whether a folder is part of the project:
Certainly! Here’s a revised version of your test description, formatted for clarity and professionalism. You can append this text to your pull request.
Test Scenario:
Conclusion:
Both checksum verification and directory upload functionalities are working as intended. The checksum verification is successfully preventing unnecessary uploads of unchanged files, ensuring efficient data transfer.