High RAM usage when working with large documents

Ask for Help
1

Hello!

We’re experiencing a performance issue on our self-hosted Grist instance and would appreciate some guidance on optimization.

Environment:

  • Grist version: 1.7.12 (grist-core)

  • Deployment: docker-compose (grist-core + redis + nginx)

  • Document storage: MinIO (S3-compatible)

  • Document database: SQLite (default)

  • VM: 4 vCPU, 12 GB RAM

  • Scale: ~10 documents, ~30 users

Problem:
When opening large tables (documents with a large number of records) and importing data from CSV, we see a sharp spike in RAM usage. Logs and monitoring show that all records of a document are loaded entirely into memory. When several users open different “heavy” documents at the same time, RAM usage quickly hits the limit and Grist stops responding (hangs / doesn’t process requests).

Questions for the community:

  1. Is there a way to limit or optimize the amount of data loaded into memory when opening a document (e.g., partial loading, backend-level pagination)?

  2. Are there recommended environment variables (GRIST_*) for tuning memory usage in grist-core?

  3. The bottleneck seems to be in the architecture of loading the document into memory (formula/table engine) — are there ways to work around this or reduce the load without changing the data itself?

  4. Is there any established practice for horizontal scaling — would it help to distribute documents across multiple grist-core instances instead of one, to isolate the load?

  5. What RAM/CPU limits are typically recommended per document/user at this kind of scale (10 documents, 30 users, some of them large)?

  6. Are there any ready-made manifests/Helm charts or recommendations for deploying grist-core on Kubernetes? Does such a deployment offer advantages in scaling and load isolation compared to docker-compose?

I’d be grateful for any configuration advice, or links to relevant issues/discussions on this topic.

Thank you!

2

Thanks for the thorough writeup! What you’re running into is really the limit of a single server. With several large documents and ~30 users, they end up competing for the same box’s memory.

The fix is spreading documents across more than one server so a heavy one can’t stall everyone else. We offer multi-server support for self-hosted Enterprise installations that run Grist as a pool of interchangeable servers behind your load balancer, with documents distributed across them. You scale out by cloning a server you already have. It fits your stack well (you’re already on Redis and MinIO).

Email us at success@getgrist.com and we’ll walk through your setup and sizing. Happy to dig into the details there.