Grist Omnibus Attachments Storage

Developers
1

Hi there,

First off, thank you for building a truly fantastic product!
Our company loves Grist and we’ve recently set up Grist omnibus internally.

  1. Wondering how Attachments are stored on the Omnibus image by default and are there performance bottlenecks we should be aware of as the database grows?

  2. Is it recommended to use something like AWS S3 for file storage from initial deployment or can we port our Attachments to the cloud easily at a later stage?

Thanks in advance!
We’d appreciate some guidance :slight_smile:

2

Hi @Onis_Csadi1, glad you are liking Grist!

Grist documents are individual SQLite databases, and attachments are stored within a table called _gristsys_Files within those individual databases. So when you click Download and get a .grist file, that contains all the attachments for that document also. And if you look at the volume mounted at /persist for a Grist container, there should be a docs directory with all the .grist files. There’s a list of the files and what they are for at Self-managed Grist - Grist Help Center

Storing attachments like this has the advantage of being self-contained, and makes it easy to back-up and restore a document. It does mean you are limited by disk space. We don’t yet have an option for storing attachments separately to the document that contains them.

If you hook up snapshot storage, then documents (with their attachments embedded in them) can be stored in S3 and similar. The steps are described in Self-managed Grist - Grist Help Center. I’d say it is worth setting this up early, for the peace of mind of having previous versions of documents available in the UI.

3

Hi @paul-grist

Thank you for the quick response and a clear explanation.
This makes sense and clears up the confusion.

We were under the impression that Attachments were stored on the filesystem and related metadata stored in .grist files along with table data.

The fact that raw Attachment data is also embedded in the .grist file is good news for our use case. Data integrity and portability is far more valuable for us than optimizing for disk space… we can always extend our instance volume as our data grows.

We’re in the process of hooking up snapshot storage to take care of backups and I’m happy to hear Attachments are also backed up for free.

I appreciate your help Paul!
Thanks :+1:

1 Like