Snapshot removing process : what triggers and maybe missed snapshot

nipil · November 19, 2025, 1:26pm

Hello, i am using a self-hosted (non-enterprise) grist with cloud storage on a Minio

I did read the following about the topic at hand

Cloud storage - Grist Help Center
the shouldKeepSnapshots() function in DocSnapshots.ts
the /api/docs/:docId/snapshots/remove route un DocApi.ts

As i understand from to the documentation, GRIST_SNAPSHOT_TIME_CAP defines “a maximum number of backups to keep for each time period” and GRIST_SNAPSHOT_KEEP overrides the minimum (total ?) number of revisions to keep (over all time periods ?). The default value for hours is 25, and for days it is 30, and keep is 5.

And as of now (2025-11-19T13:02:11UTC) i have these revisions for one of my document

newer, snapshot suppression possibly not triggered yet
M “lastModified”: "2025-11-19T10:32:32.702Z
L “lastModified”: "2025-11-19T10:30:18.444Z
K “lastModified”: "2025-11-19T10:28:22.814Z
J “lastModified”: "2025-11-19T10:22:42.433Z
H “lastModified”: "2025-11-19T10:06:20.072Z

kept as last from the hour and less than 25 hours
G “lastModified”: "2025-11-19T09:21:39.729Z

kept as last from the hour and last for day
F “lastModified”: "2025-11-18T09:36:00.888Z
B “lastModified”: "2025-11-17T16:00:21.800Z

kept as last from the hour, not last of day, but more than 25 hours ago
A “lastModified”: "2025-11-17T15:40:14.957Z

This morning i had these versions for the same document

G “lastModified”: “2025-11-19T09:21:39.729Z”
F “lastModified”: “2025-11-18T09:36:00.888Z”
E “lastModified”: “2025-11-18T09:35:47.857Z” → auto-removed since
D “lastModified”: “2025-11-18T09:35:22.966Z” → auto-removed since
C “lastModified”: “2025-11-18T09:34:49.952Z” → auto-removed since
B “lastModified”: “2025-11-17T16:00:21.800Z”
A “lastModified”: “2025-11-17T15:40:14.957Z”

As i have/had more than KEEP (5) versions, i think some should have be deleted anyway, which happened and is great, but i think some were either missed, or it is working as intended and i do not understand why they were kept.

So here are a few questions :

candidates for deletion
- B is more than 25 hours ago but was the last of its day, so has been kept. correct ?
- as stated above, i feel that A should have been removed, but was not. Working as intended ?
snapshot deletion process
- when are the snapshot remove triggered ?
- what triggers it ?
- are all documents evaluated ?
- is there any additional setup required (cron job, config ?)
- mainly, how can i ensure i “globally” trim down storage" ?
Does the KEEP parameter work across all time periods or for each time period ?

As always with storage, i am worrying about creeping capacity costs for when i open service to users.

Thanks in advance for any help
Nicolas

paul-grist · November 19, 2025, 5:24pm

Hi Nicolas. The rules are cumulative. For GRIST_SNAPSHOT_KEEP=N, the documentation is:

Number of recent snapshots to retain unconditionally for a document, regardless of when they were made

Grist will attempt to keep the most recent N snapshots, regardless of any other rules.

GRIST_SNAPSHOT_TIME_CAP has unhelpful documentation, but adds MORE reasons why Grist should preserve a snapshot - because it is the most recent in a particular hour, or day, etc.

nipil · November 21, 2025, 8:01am

Ok Paul, thanks, that clarifies the intended inner working of these parameters

But what about the other questions (when/what/how triggers the pruning) ?

paul-grist · November 21, 2025, 4:16pm

Looking here:

github.com/gristlabs/grist-core

app/server/lib/HostedStorageManager.ts

122381a66


      
          delayBeforeOperationMs: 0,  // prune as soon as we've made a first upload.
          minDelayBetweenOperationsMs: secondsBeforePush * 4000,  // ... but wait awhile before
                                                                  // pruning again.

Sounds like the first prune of a document happens pretty quickly after the first new version is stored after the server has opened a document to work on it. And then successive prunes happen at some multiple of a time that traces back in the code to this variable:

github.com/gristlabs/grist-core

app/server/lib/HostedStorageManager.ts

122381a66


      
          import * as fse from 'fs-extra';
          import * as path from 'path';
          import {v4 as uuidv4} from 'uuid';
          import {OpenMode, SQLiteDB} from 'app/server/lib/SQLiteDB';
          import {Features} from "app/common/Features";
          
          // Check for a valid document id.
          const docIdRegex = /^[-=_\w~%]+$/;
          
          // Wait this long after a change to the document before trying to make a backup of it.
          const GRIST_BACKUP_DELAY_SECS = parseInt(process.env.GRIST_BACKUP_DELAY_SECS || '15', 10);
          
          function checkValidDocId(docId: string): void {
            if (!docIdRegex.test(docId)) {
              throw new Error(`Invalid docId ${docId}`);
            }
          }
          
          export interface HostedStorageCallbacks {
            // Saves the given metadata for the specified documents.
            setDocsMetadata: SaveDocsMetadataFunc,

If you wanted to take more detailed control of it, seems like you could. There’s also some snapshot related endpoints:

github.com/gristlabs/grist-core

app/server/lib/DocApi.ts

122381a66


      
          this._app.post('/api/docs/:docId/snapshots/remove', isOwner, withDoc(async (activeDoc, req, res) => {
            const docSession = docSessionFromRequest(req);
            const snapshotIds = req.body.snapshotIds as string[];
            if (snapshotIds) {
              await activeDoc.removeSnapshots(docSession, snapshotIds);
              res.json({snapshotIds});
              return;
            }
            if (req.body.select === 'unlisted') {
              // Remove any snapshots not listed in inventory.  Ideally, there should be no
              // snapshots, and this undocumented feature is just for fixing up problems.
              const full = (await activeDoc.getSnapshots(docSession, true)).snapshots.map(s => s.snapshotId);
              const listed = new Set((await activeDoc.getSnapshots(docSession)).snapshots.map(s => s.snapshotId));
              const unlisted = full.filter(snapshotId => !listed.has(snapshotId));
              await activeDoc.removeSnapshots(docSession, unlisted);
              res.json({snapshotIds: unlisted});
              return;
            }
            if (req.body.select === 'past') {
              // Remove all but the latest snapshot.  Useful for sanitizing history if something

This file has been truncated. show original